Medallion Architecture: Bronze, Silver, and Gold Layers Explained

Medallion architecture is a data design pattern that organizes a lakehouse into three layers of increasing quality: bronze stores data exactly as it arrived from source systems, silver stores cleaned, validated, and deduplicated records, and gold stores business-ready tables for reporting, analytics, and machine learning. Databricks, which named the pattern, states its goal in the Databricks glossary as "incrementally and progressively improving the structure and quality of data as it flows through each layer of the architecture." The layers are a convention with no license attached, so teams run the same pattern on Databricks, Snowflake, and Microsoft Fabric. BD Emerson treats each boundary between layers as a contract: a named owner, a set of quality checks, and an access rule that together decide what may cross.

Bronze, silver, and gold layers compared

The three layers differ in purpose, readers, retention, checks, and access. The purposes and readers below follow Databricks' medallion architecture documentation, and the retention, quality, and access rows are BD Emerson's design defaults.

Where did medallion architecture come from?

Databricks introduced the bronze, silver, and gold names. Its August 14, 2019 post on productionizing machine learning with Delta Lake described "tables that correspond to different quality levels in the data engineering pipeline" and called the set a "multi-hop" architecture. Databricks' current documentation calls following the pattern "a recommended best practice but not a requirement."

The layering idea is older than the name. Warehouse teams ran staging areas and data marts in the Kimball and Inmon traditions, the Databricks glossary places Data Vault style models in silver and star schemas in gold, and dbt Labs recommends a parallel staging, intermediate, and marts structure. Medallion added a shared vocabulary for lakehouse tables and a default that every layer is a governed table with history.

How to implement medallion architecture on Databricks

On Databricks, the pattern runs on Delta Lake tables, Auto Loader for ingestion, Lakeflow pipelines for transformation, and Unity Catalog for access and lineage. Delta Live Tables (DLT) is now called Lakeflow pipelines in Databricks documentation, and older material uses the name Lakeflow Declarative Pipelines. Existing DLT code runs without migration, and the product extends the open source Apache Spark Declarative Pipelines framework in Apache Spark 4.1.

Auto Loader suits bronze because of how it handles schema change. It infers JSON and CSV columns as strings by default, and in its default addNewColumns mode a new source column stops the stream, joins the schema, and resumes on restart. Values that do not fit the schema land in the _rescued_data column as a JSON blob that includes the source file path, so bronze keeps every field. Databricks advises against writing to silver directly from ingestion, because those writes fail when source schemas change or records arrive corrupt.

The AUTO CDC API, which replaces APPLY CHANGES with the same syntax, does the silver merge work. It orders changes by a sequence column, so a late update cannot overwrite a newer one, and it applies source deletes through a condition such as operation = 'DELETE'.

Decide the Unity Catalog layout before the first table lands. BD Emerson's default is one catalog per domain and environment, such as sales_prod, with bronze, silver, and gold schemas inside it, so a grant on sales_prod.gold means the same thing to everyone. Retrofitting permissions onto a populated lakehouse is the expensive path, a point we cover in Databricks as the enterprise AI stack.

How does medallion architecture map to Snowflake and Microsoft Fabric?

Both platforms run the same pattern with different building blocks. On Snowflake, teams land raw data with Snowpipe into tables that keep semi-structured payloads in VARIANT columns, then chain dynamic tables for silver and gold, and Snowflake infers the dependency graph from the queries and refreshes tables in dependency order to meet each table's target lag. Microsoft's Fabric guidance recommends the medallion design with each layer in its own lakehouse, in its own workspace, for "more control and better governance at the layer level."

Teams consolidating from Snowflake onto Databricks often rebuild their layers during the move, and our Snowflake to Databricks migration guide covers the sequencing.

Where should data quality checks run?

Run structural checks in bronze, record-level expectations in silver, and business reconciliation in gold, and pick each failure action by the cost of a wrong number. Lakeflow pipeline expectations offer three actions: warn keeps invalid records and logs metrics, drop removes them before the write and logs the count, and fail stops the update and rolls back the transaction.

Quarantine beats a silent drop. A drop expectation logs a count, but the rows never reach the target, and nobody can fix records they cannot see. Writing rejected rows to a quarantine table with the failed rule name gives data stewards a work queue. Databricks data profiling adds statistics over time, such as null fractions and distribution drift, which catches the slow decay that row-level rules miss.

How should you govern each layer?

Govern bronze most tightly, because it holds raw personal data before any masking or minimization. BD Emerson grants bronze read access to pipeline service principals and a short list of platform engineers, gives analysts silver and gold, and treats any human query against bronze as an exception that gets logged.

Apply masking by policy and tag. Unity Catalog's attribute-based access control attaches row filter and column mask policies at the catalog, schema, or table level, and a policy applies automatically to every object that carries the governed tag it targets, so a tagged email column gets its mask without a per-table grant. Unity Catalog also classifies and tags sensitive data automatically, which catches personal data that arrives in a new source column.

Start deletion requests in bronze. Databricks' guidance on GDPR and CCPA deletes in Delta Lake recommends deleting in bronze first and propagating the change to silver and gold, and it warns that deleted rows persist in table history until maintenance runs. VACUUM removes unreferenced files older than a retention threshold that defaults to 7 days, and tables with deletion vectors also need REORG TABLE ... APPLY (PURGE). Keeping personal data in its own table, keyed by a non-natural ID such as user_id, lets you delete the person and keep the anonymous history.

Protect lineage with two naming rules. Unity Catalog captures column-level lineage automatically for queries run on Databricks, but column lineage requires both source and target to be referenced by table name, and lineage is not preserved for renamed catalogs, schemas, tables, views, or columns. Reference tables by name, never by storage path, and settle names before production. The lineage system tables keep a rolling one-year window, so export what auditors will ask for later. Naming owners and stewards for each domain is part of the wider data governance framework.

Common medallion architecture mistakes

Most medallion rework BD Emerson sees comes from treating layers as folders instead of contracts. Seven mistakes cause most of it:

• Gold sprawl: every dashboard gets its own gold table, metric definitions drift, and two reports disagree on revenue. Map gold tables to business entities and certified metrics.
• Copying data with no contract: a table moves to gold with no stated schema, freshness target, or owner, so nobody knows who fixes it. Write the contract before the pipeline.
• Skipping silver: teams build gold aggregates straight from bronze, and each gold table reimplements deduplication and type casting its own way. Databricks says silver "should always include at least one validated, non-aggregated representation of each record."
• Treating the layers as three physical copies of everything: small reference tables can be views, gold aggregates are a fraction of silver's size, and in the worked example below bronze holds about 97 percent of the bytes.
• Writing silver straight from ingestion: Databricks recommends against it because those writes fail on schema changes and corrupt records.
• Keeping bronze with no retention rule: raw data kept forever by default becomes the largest store of personal data in the company and the hardest one to purge.
• Using layers as environments: layer names describe data quality and dev, test, and prod describe deployment stages, so mixing them produces schemas like gold_dev that blur both.

When does medallion architecture not fit?

Medallion fits batch and near-real-time analytics, and it fits poorly in three situations. The first is streaming-first work that needs results in under a second, such as card fraud scoring, where every extra hop adds latency. A Kappa-style design that reprocesses from a replayable log, the approach Jay Kreps proposed in his 2014 essay Questioning the Lambda Architecture, serves those paths better, with medallion tables downstream for analytics. Snowflake draws a similar line in its documentation, which says dynamic tables do not fit workloads that need data fresher than 60 seconds.

The second is small data. A company with one PostgreSQL database and a dozen dashboards gets the same discipline from a dbt project with staging and marts models. The third is a data mesh, where each domain owns its data products under what Zhamak Dehghani calls federated computational governance. There, medallion runs inside each domain instead of as one central pipeline. The Databricks glossary calls medallion "compatible with the concept of a data mesh," and Microsoft's Fabric guidance places medallion layers inside domains such as sales and marketing.

Worked example: SQL Server orders to a gold revenue table

Here is how BD Emerson would apply the pattern for a distributor whose order management system runs on SQL Server, where finance wants a trusted gold table of daily net revenue by region. The volumes are illustrative assumptions, stated so the arithmetic can be checked.

Bronze. AWS Database Migration Service writes change data capture files for the orders, order_lines, and customers tables to Amazon S3, with a first field that marks each row I, U, or D for insert, update, or delete. Auto Loader reads the files into sales_prod.bronze.order_lines_raw as strings, adds the file name and ingest time, and keeps unexpected fields in _rescued_data. The only checks confirm that files arrived and that hourly row counts sit in range.

Silver. A Lakeflow pipeline uses AUTO CDC to build silver.order_lines as SCD Type 1 and silver.customers as SCD Type 2, sequenced by the source commit timestamp, with rows marked D applied as deletes. Expectations send rows with a null order_line_id to a quarantine table and warn on negative amounts, which are usually credit memos. A governed tag on the customer email column triggers the ABAC mask policy for analysts.

Gold. A materialized view, gold.daily_net_revenue_by_region, sums net amounts by order date and region. Before each refresh publishes, a check compares the day's total with the order system's daily control total and fails the update on a difference above 0.1 percent. Finance queries the table through a Databricks SQL warehouse, and Unity Catalog lineage traces each figure back to its bronze file.

Bronze holds about 97 percent of the storage in this design, 200 of about 206 GB, silver holds about 3 percent, and gold holds almost none. Cutting bronze retention from 400 days to 90 days drops total storage from about 206 GB to about 51 GB. Bronze retention and refresh frequency move the bill far more than the number of layers does.

What does medallion architecture cost?

Medallion costs come mostly from compute and bronze retention, and three design choices set the bill. The first is ingestion mode. Databricks documents three: continuous incremental ingestion has the highest cost and lowest latency, triggered incremental ingestion costs less with more latency, and batch ingestion with manual incremental logic also costs less but has the highest latency. A finance table that refreshes hourly does not need a continuous pipeline.

The second is maintenance. Rewritten and deleted files keep billing as storage until VACUUM removes them, and Delta Lake keeps 30 days of log history by default. Databricks recommends predictive optimization for all Unity Catalog managed tables, which runs VACUUM automatically. The third is refresh scope. An incremental refresh processes only changed rows, and a full refresh recomputes the whole result; Snowflake's dynamic tables expose the choice through INCREMENTAL, FULL, and AUTO refresh modes. A gold table that fully recomputes years of history every hour is the first cost leak to check.

Frequently asked questions

What is medallion architecture in simple terms? Medallion architecture is a way to organize a lakehouse into three layers. Bronze keeps raw data exactly as it arrived, silver keeps cleaned and validated records, and gold keeps business-ready tables for dashboards, reports, and machine learning. Each boundary between layers has its own quality checks, owner, and access rules.

Is medallion architecture only for Databricks? No. Databricks named the pattern, but it is a design convention that works on any platform with tables and pipelines. Snowflake teams build it with Snowpipe and dynamic tables, Microsoft recommends it for Fabric lakehouses, and teams on plain object storage build it from Delta Lake or Apache Iceberg tables.

What is the difference between bronze, silver, and gold data? Bronze data is raw and unchanged, kept for replay and audit, and read mainly by pipelines and data engineers. Silver data is typed, deduplicated, and validated, with one trustworthy version of each record for engineers, analysts, and data scientists. Gold data is aggregated and modeled for specific business questions, and it is the layer most business users should query.

Do you need all three layers? Most lakehouses need all three layers logically, but not every dataset needs three physical tables. A small reference table can be a silver view over bronze, and a simple projection of silver can serve as gold. Keep one rule: silver holds one validated, non-aggregated version of each record, so gold never reads raw data.

Where should personal data live in a medallion architecture? Raw personal data lands in bronze, which is why bronze needs the tightest access. In silver, tag personal data columns so masking policies apply automatically, and keep identifiers in a separate table keyed by a non-natural ID where you can. Gold should aggregate personal data away or mask it. Deletion requests start in bronze and propagate to silver and gold.

What happened to Delta Live Tables? Databricks renamed it. As of October 2026, Databricks documentation calls Delta Live Tables Lakeflow pipelines, older material uses the name Lakeflow Declarative Pipelines, and the classic SKUs still begin with DLT. Existing DLT code runs without migration, and the product extends the open source Apache Spark Declarative Pipelines framework.

If you are planning a lakehouse build or untangling one that grew without contracts, our data engineering team can design the layer contracts, pipelines, and quality gates with you, and our Databricks consulting practice can build the Unity Catalog structure and access policies alongside them.