Snowflake发布CoCo一键迁移Spark工作负载的官方技术博客

Migrate Apache Spark to Snowflake with CoCo
Skip to content
Product
Solutions
Why Snowflake
Resources
Developers
Pricing
Language
Languages
English
日本語
한국어
中文（简体）
Português
Deutsch
Français
Español
Italiano
Sign in
CONTACT SALES
start for free
INSIDE THE DATA CLOUD
ENGINEERING BLOG
AI/ML
At Snowflake
Partner & Customer Value
Industry Solutions
Product & Technology
Strategy & Insights
Blog
/
Run Apache Spark™ Pipelines on Snowflake: Faster Performance and Lower Costs with One CoCo Prompt
JUL 30, 2026
1 min read
Copy post link
Open in Claude
Open in ChatGPT
Ash Ubrani +1
Running Spark is not just about writing transformations and business logic. It also means tuning clusters, patching infrastructure and managing dependency environments.
Moving those workloads onto Snowflake addresses that directly, with customers experiencing up to 5.1x faster performance and 42% lower costs
1
.
Snowpark Connect for Apache Spark™
lets your existing Spark code run on Snowflake's engine with minimal changes; no clusters to provision, tune or patch.
Most generic AI coding assistants can fall short when it comes to migration at scale. They can rewrite a snippet, but they lack the compatibility context to move an entire codebase to a new engine: which patterns are unsupported, how to map them to DataFrame equivalents and how to record what changed. The
spark-migration
skill in
Snowflake CoCo
, a data-native AI coding agent, is built to close that gap.
One prompt, one codebase
The clearest way to see the skill is to watch it run. The demo below starts with a directory of data pipelines written in PySpark.
From there, the prompt is simple. Just ask CoCo to migrate a file or directory of files to Snowpark Connect. This will launch the
skill coordinating the
snowpark-connect
migration path that will guide you through making your Spark code compatible with Snowpark Connect. Note that the input file(s) for this skill could be Python, Scala or Java code files (as well as the build files). The skill can also take in notebook files to ensure they are compatible with Snowflake.
The demo shows a large Spark codebase checked for compatibility and migrated to Snowflake from a single prompt.
Figure 1: A directory of PySpark pipelines scanned, rewritten and reported on from one conversational prompt.
How to migrate PySpark to Snowflake with CoCo
Behind that single prompt, CoCo's agentic workflow runs through the steps a migration expert would:
It does an assessment and builds an inventory:
This skill scans every source file for compatibility issues including RDD operations, certain UDF serialization patterns and unsupported file formats, referencing a knowledge base built by Snowflake's engineering team. Each file, dependency and unique API call are inventoried.
It applies targeted fixes:
The core conversion is done by dispatching parallel agents to apply code fixes and verifying the results through multiple gates, including syntax compilation and evidence-based checks. During this phase, incompatible patterns are rewritten to DataFrame equivalents, imports and session creation are updated and anything that needs manual review (such as validating reads and writes) is flagged with a detailed explanation.
It generates predictable reporting that can be consumed by people or agents:
This gives you full visibility into every change made and what may still need attention by generating issue logs, inventory reports and a validation of the overall migration state.
It validates the converted code's functionality:
The validation phase performs an end-to-end validation by running the original PySpark code and the migrated Snowpark Connect code against the same synthetic data and comparing their outputs table-by-table. This catches behavioral differences and validates the code is ready to run. Any fixes discovered during validation are automatically committed back to the deliverable branch, so the result is both a pass/fail verdict and a more complete migration. This validation phase is optional.
Instead of searching across files for incompatible patterns and cross-referencing documentation, the developer stays in one workflow while CoCo handles the scanning, fixing and reporting needed to get Spark code running on Snowflake.
More than a single file
Migration is rarely one clean step, so the skill meets a codebase wherever it is. A few other ways to point it at your work:
Scope the effort before you start:
"Assess the compatibility and level of effort required to migrate this codebase."
Validate before you run:
"Validate that this codebase will run with Snowpark Connect."
Complete another tool's migration:
"Analyze the output of any other migration tool and complete the migration."
The skill auto-activates when you mention Spark, PySpark or code migration; or you can invoke it directly with
. As noted before, it will flag anything that needs manual review, so you stay in control of what changes.
Try it yourself
That is a compelling offer: less manual compatibility testing, less refactoring and a faster path to running Spark workloads natively on Snowflake. What used to take hours of manual effort, even with existing deterministic tools, now takes minutes.
The
skill is bundled with CoCo; no setup required.
Try it:
"Migrate this file to Snowpark Connect."
Get started with CoCo and Spark migration by referring to
this documentation
Based on customer production use cases and proof-of-concept exercises comparing the speed and cost for Snowpark versus managed Spark services between November 2022 and May 2025. All findings summarize actual customer outcomes with real data and do not represent fabricated datasets used for benchmarks.
Learn more about the authors
Ash Ubrani
Senior Product Marketing Manager
Brandon Carver
Analytics and Product Management, Snowflake
Subscribe to our blog newsletter
Get the best, coolest and latest delivered to your inbox each week
By submitting this form, I understand Snowflake will process my personal information in accordance with their Privacy Notice.
Subscribe to our monthly newsletter
Stay up to date on Snowflake’s latest products, expert insights and resources—right in your inbox!
Platform
Snowflake CoWork
Data Engineering
Analytics
AI
Applications & Collaboration
Support
Priority Support
Status
Industries
Advertising, Media & Entertainment
Financial Services
Healthcare & Life Sciences
Manufacturing
Public Sector
Retail & Consumer Goods
Telecom
Technology
Company
About Snowflake
Leadership & Board
Careers
Investor Relations
Trust Center
Brand Guidelines
Contact
Newsroom
Environmental, Social & Governance
Snowflake Ventures
End Data Disparity
Snowflake Summit 26
Learn
Resource Library
Live Demos
Fundamentals
Training
Certifications
Snowflake University
Developer Guides
Documentation
Data Governance
© 2026 Snowflake Inc. All Rights Reserved
Privacy Policy
Site Terms
Communication Preferences
Cookie Settings
Do Not Share My Personal Information
Legal