Snowflake to Databricks Migration: A Practical Guide

A Snowflake to Databricks migration is worth doing when a material share of your workload is machine learning, streaming, or processing of semi-structured and unstructured data that you are currently working around, or when you are already paying for two platforms and two governance models to do one job. It is not worth doing when your estate is mostly SQL analytics and dashboards that Snowflake already serves well. Plan on 6 to 18 months for a full estate, 4 to 12 weeks per significant workload, and a 3 to 9 month window where you pay for both platforms at once. Reserve 15 to 30 percent of the effort for governance and identity mapping, because that is where these programs actually stall.

When the migration makes sense

Four situations justify it on their own. First, machine learning and AI workloads that already pull data out of Snowflake into separate compute, which means you are running two platforms, duplicating data, and paying to move it between them. Second, streaming or near real-time requirements where micro-batch ingestion has quietly become a stack of workarounds with its own failure modes. Third, heavy semi-structured and unstructured processing at a volume where warehouse compute is an expensive way to run transformation over documents, logs, event payloads, and images. Fourth, a mandate to govern files, tables, models, notebooks, and dashboards under one permission model, which is the actual argument for Unity Catalog rather than any query performance claim.

Cost can be a legitimate driver, but only after modeling. Databricks is not universally cheaper, and the platforms have converged enough that price differences on a given workload usually come down to tuning and commitment terms rather than architecture.

When it does not

If 80 percent or more of your workload is BI-serving SQL over structured data with concurrency spikes, you will spend 6 to 18 months arriving somewhere very similar to where you started. If the team is SQL-first with no Spark or Python depth, you are also buying a hiring and training program, so add 3 to 6 months of ramp before productivity returns to baseline. If a Snowflake commitment has 18 months left, dual-run cost usually swamps projected savings. And if the driver is a vendor benchmark or a slide someone showed the board, stop and write down the workload that is actually failing today.

Running both platforms is a legitimate architecture rather than an admission of failure. Plenty of enterprises use Snowflake for serving and Databricks for engineering and ML, and pay for the overlap deliberately. We compare where each one wins in Databricks vs Snowflake.

Triage the workloads before planning anything

Inventory every table, pipeline, and report, then sort each into one of five buckets. This exercise takes two to four weeks and determines the entire shape of the program.

• Lift and shift. Standard SQL transformations, dbt models, and tables using no proprietary features. Cheapest to move, and usually 60 to 75 percent of an estate by object count.
• Rewrite. JavaScript stored procedures, Snowflake Scripting, tasks and streams, proprietary functions. Small by count, large by effort, frequently 30 to 50 percent of total migration hours.
• Improve. Workloads that exist only to work around a platform limit, such as batch jobs simulating streaming or Python running outside the warehouse because it could not run inside. These get redesigned rather than moved, and this is where the return actually comes from.
• Leave. Workloads better served where they are. Naming these explicitly is what keeps the program honest and the timeline real.
• Retire. Every estate carries 10 to 25 percent of tables and pipelines nobody reads. A migration is the cheapest moment you will ever have to find out which.

The phased path

Phase 0, foundation, 3 to 6 weeks. Workspaces, networking, a Unity Catalog metastore, external locations over your object storage, identity federated from your provider, and a cost tagging scheme agreed with finance. Teams that skip straight to data movement rebuild this phase later under time pressure.

Phase 1, data, 4 to 10 weeks per domain. Move bulk history first, then run incremental sync so both platforms hold current data through the transition. Data types are where silent errors live. VARIANT columns need an explicit decision between a typed structure and stored JSON. Snowflake's three timestamp variants do not map cleanly and will shift values if handled carelessly. NUMBER precision and scale need checking against Delta equivalents. Zero-copy clones become shallow clones, and time travel becomes Delta time travel bounded by VACUUM retention, which defaults to a much shorter window than teams expect, so set it deliberately before someone needs to recover a table.

Phase 2, pipelines, 6 to 16 weeks. If your transformations run in dbt, this is mostly adapter, profile, and SQL dialect work, and it is the cheapest possible starting position. If they run as stored procedures with tasks and streams, they are a rewrite into declarative pipelines or Structured Streaming with incremental file ingestion. Orchestration either moves to native jobs or stays in Airflow, and keeping Airflow during migration removes one variable from a phase that already has too many.

Phase 3, BI, 4 to 12 weeks. This phase generates the political risk, because dashboard numbers get compared side by side and any variance becomes a credibility problem. Name one owner for the semantic layer, agree numeric tolerances before cutover, and reconcile a fixed set of executive metrics daily during parallel run. Connection configuration, SQL warehouse sizing, and caching behavior all need tuning before anyone judges performance.

Phase 4, ML and AI, 4 to 12 weeks. Usually the easiest phase and often the reason for the program. Feature tables, the model registry, and serving endpoints move onto governed objects, and training data finally has lineage back to source. For what that layer looks like fully assembled, see the Databricks enterprise AI stack.

Then decommission, on a date written down at the start of the program and defended.

Mapping governance to Unity Catalog

Snowflake's role hierarchy does not translate one to one, and treating it as a rename exercise is the most reliable way to lose a quarter. Roles become account-level groups plus explicit privileges on catalogs, schemas, and objects. The trap that catches nearly everyone: a grant on a table is not sufficient, because the principal also needs usage on the parent catalog and schema. Missing parent privileges is the most common cause of access failures in the week after cutover, and it presents as a permissions bug rather than a design gap.

Masking policies and row access policies become column masks and row filters. Secure views and reader accounts become Delta Sharing. Resource monitors become budget policies plus system tables for chargeback. Produce the permission mapping as a written artifact reviewed by whoever approves access today, before anyone grants anything by hand. Assign groups as object owners rather than individuals, because an object owned by a departed employee becomes a support ticket at the worst possible moment.

Cost modeling before you commit

Do not compare credit prices to DBU prices. Take your top 20 workloads by spend and model each on both platforms using your own queries, your own data volumes, and your own concurrency patterns. Include compute for transformation, serverless SQL for BI, storage, the dual-run period, engineering time, and training. Engineering time is usually the largest line and the one most often left out entirely.

A defensible model shows total cost of ownership over 24 to 36 months rather than a monthly snapshot taken at the moment of comparison. Budget the dual-run period at 40 to 80 percent of one platform's monthly cost for 3 to 9 months. And treat vendor benchmarks as marketing, because the gap between a tuned and an untuned deployment on either platform is larger than the gap between the platforms.

Common failure modes

Governance left for last. Teams move data first, then find that permissions cannot be reconstructed from the old model, and the program stops while someone designs what should have been designed in phase 0.

Lift and shift instead of triage. Moving the workarounds along with the workloads produces the same estate on different infrastructure, plus a migration bill and a year of disruption.

No decommission date. Dual run becomes permanent, and the organization now pays for two platforms while maintaining the governance surface of three.

Underestimating the tail. The last 10 percent of objects consume 30 to 40 percent of the effort, and they are reliably the ones finance closes the month with.

No named owner for BI numbers. Two systems producing slightly different revenue figures for the same quarter does more damage to a program than any technical problem in it.

Where BD Emerson fits

Our Databricks consulting work usually starts with the triage and the cost model, because those two artifacts settle whether the migration should happen at all, and a fair number of assessments end with a recommendation to keep both platforms and fix the integration instead. Where a migration is the right call, our data engineering team runs the phases with your engineers rather than around them, since the people who will operate the estate afterward need to have built it.