Data Migration Strategy: Cutover Options, Reconciliation, and How Much History to Move

A data migration strategy decides how data leaves a legacy system and arrives in its replacement: which records move and which go to an archive, how they are mapped and cleansed, how the cutover happens (big bang, phased, or parallel run), and how the team proves the target matches the source before the old system is switched off. In a legacy replacement, the data work usually sets the go-live date, because every full rehearsal exposes mapping and quality defects that must be fixed before the next one. For high-volume transactional systems, the safer default is a phased cutover with change data capture keeping source and target in step. Big-bang weekends suit systems small enough to rehearse end to end.

What does a data migration strategy cover?

A complete strategy answers seven questions in writing before the first mock migration runs.

• Scope: which entities and how many years of history move, and what goes to an archive instead.
• Mapping: the transformation rule for every target field, with a named business owner who approves it.
• Quality: which defects get fixed in the source, which get fixed in flight, and which the business accepts.
• Cutover: the approach, the window, and the freeze on legacy changes before it.
• Reconciliation: the counts, totals, and field comparisons that prove completeness and accuracy, with pass thresholds agreed in advance.
• Rollback: the point of no return in the cutover runbook, and a tested path back before it.
• Decommissioning: the date the legacy store becomes read-only, the date it shuts down, and who signs off on each.

Sponsors tend to fund the first three and underfund the last four, which is backward. Mapping and cleansing determine whether the data is right. Reconciliation, rollback, and decommissioning determine whether anyone can prove it is right, recover when it is wrong, and stop paying for the old system.

When should data quality work start?

Start profiling the legacy data in the first month of the program, well before the target system is ready to receive it. Profiling reads every column of every table in scope and reports nulls, invalid codes, duplicate keys, broken references, and values that violate the business rules the new system will enforce. Legacy systems tolerate bad data for years because their screens and batch jobs were written around it. The replacement rejects the same records at load time, usually during a mock migration, when the schedule has no slack left.

Log every defect class with a count, an owner, and a decision. Fixing in the source is cleanest, because the legacy system runs correctly until cutover and the fix carries over automatically. Fixing in flight, inside the transformation, is faster but leaves source and target permanently different, which complicates reconciliation. Accepting a defect is a legitimate business decision when the records are closed and low value, as long as the owner signs it and the reconciliation thresholds account for it. Duplicate customers or vendors deserve special attention, since merging them changes counts and totals, and the reconciliation logic has to reflect each merge before the first mock runs.

Big bang, phased, or parallel run: which cutover approach fits?

Three cutover approaches cover almost every program. The right one depends on data volume, the length of outage the business can accept, and how expensive a wrong answer would be.

Change data capture (CDC) is what makes phased cutovers practical. CDC reads changes from the source database's transaction log and applies them to the target, so most of the data moves while the legacy system stays in use. AWS's Database Migration Service, for example, runs a full load plus CDC task that migrates existing data and then updates the target based on changes to the source. The same documentation states that DMS CDC "does not provide real-time replication" and that there are no SLAs for CDC latency. Plan a short freeze at cutover so the target can catch up, and verify that it has caught up before users switch.

What does a failed migration look like?

The best-documented example is TSB's. In April 2018 the UK bank migrated the data for its corporate and customer services onto a new IT platform. According to the Financial Conduct Authority, all of TSB's branches and a significant proportion of its 5.2 million customers were affected by the initial problems, and the bank did not return to business as usual until December 2018. In December 2022 the FCA and the Prudential Regulation Authority fined TSB a combined £48.65 million. The regulators found that TSB failed to plan the migration properly, that the project's governance was too weak, and that it failed to manage the operational risks of its outsourcing arrangement with a critical third-party supplier. TSB had also paid £32.7 million in redress to customers.

The lesson for any sponsor concerns decision rights. A big-bang cutover of a full customer base is only as safe as the evidence behind the go decision, so write the go/no-go criteria before anyone announces a date: full-volume rehearsals completed inside the window, reconciliation breaks on material fields at zero, rollback tested, and every critical supplier's readiness shown in a rehearsal.

How do you prove the data moved correctly?

Reconciliation works in layers, and each layer catches a different class of defect. Record counts by entity and status catch dropped or duplicated records. Control totals, such as the sum of balances, invoice amounts, or quantities by business date, catch transformation errors that leave the counts intact. Field-level comparison catches mapping errors; run it on every record for financially material fields and on a statistically sized sample for descriptive ones. Business-rule checks catch what passes every comparison and is still wrong, such as orphaned child records, invalid status combinations, or dates that fall before the account opened. Finally, business users test real scenarios against migrated records, because they know which edge cases matter.

Every layer runs in every mock migration, and each mock is a full run at production volume, timed against the real cutover window. BD Emerson planning range: three to five full mock migrations for a core transactional system, with the last one run as a dress rehearsal of the cutover runbook by the people who will perform it. The trend matters as much as the final number. Breaks should fall with each mock, and a mock that turns up a new class of defect means another mock belongs on the schedule.

How much history should you migrate?

Usually less than stakeholders first ask for. Every year of closed history adds codes for retired products, old formats, and exceptions that someone has to map and validate, and little of that history helps anyone operate the new system. The common pattern moves open records plus a defined window of closed history into the new system and sends the rest to a read-only archive that stays searchable and enforces retention rules. Keeping the whole legacy system alive in read-only mode is the expensive fallback, since it keeps the licenses, infrastructure, and skills the program was meant to retire.

Two obligations constrain the cut. Retention schedules set by regulation or contract decide how long archived records must stay retrievable, and any data under a litigation hold must be preserved in a form counsel can still produce. Confirm both with legal and records management before the scope is final.

Who should own data migration?

The business owns the decisions, and engineering owns the machinery. Named data owners approve mapping rules, accept quality trade-offs, and sign off on reconciliation results. The data governance framework that defines those owners should exist before the program starts, because a migration under deadline is a poor place to invent it. Engineering builds the extraction and load pipelines, the reconciliation tooling, and the runbook automation. A cutover manager owns the runbook and runs the go/no-go meeting against the criteria agreed in advance.

Phased replacements add one more ownership question: who maintains the temporary pieces, such as sync jobs and dual-write interfaces, that let the old and new systems coexist. Martin Fowler calls this transitional architecture, code that will go away once the modernization is complete, and he argues that the reduced risk and earlier value of a gradual approach outweigh its costs. Give that code an owner and a retirement date, or it will outlive the program. Data migration is also one workstream inside a larger replacement, and our application modernization services cover the application side of that program.

BD Emerson's data engineers design migration strategies for legacy replacements and build the pipelines, reconciliation tooling, and cutover runbooks that prove the data arrived intact. Our data engineering services page explains how we scope that work.