7 Overnight Migrations, 850GB, Zero Downtime

In 2023, a major satellite operator completed a nine-figure acquisition. The technical challenge: migrate all customer data and services from the acquired infrastructure onto the buyer’s systems, without disrupting existing broadband customers across Europe.

I was the engineer responsible for building and executing the migration pipeline.

The Constraint

Every migration had to run overnight. Residential broadband customers across Europe couldn’t experience downtime during business hours. This meant each migration had to:

1. Start after midnight in the relevant timezone
2. Complete before morning
3. Succeed fully or roll back cleanly, with no partial states
4. Handle 100GB+ data volumes within the window

The Architecture

AWS Step Functions orchestrated each migration as a state machine. This was the right choice for several reasons:

• Visual debugging: when a migration step fails at 3 AM, you need to see exactly where it stopped and why. Step Functions gives you a visual execution graph.
• Built-in retry logic: transient failures (network blips, API rate limits) retry automatically with exponential backoff.
• Parallel execution: independent data streams (customer records, billing, service configs) migrate concurrently.
• Compensation: each step has a defined rollback. If step 5 of 8 fails, steps 4 through 1 unwind in reverse.

The data moved through a staging layer before hitting production. Every record was validated against the target schema before commit. Mismatches flagged for manual review rather than silently corrupting data.

The Execution

Seven migrations over three months. Each one planned for a week, rehearsed in staging, then executed overnight:

• Total data transferred: 850GB+
• Customer-facing downtime: Zero
• Failed migrations: Zero (two were paused and resumed after fixing upstream data quality issues)
• Rollbacks triggered: None in production

The largest single migration moved 200GB+ in under 6 hours.

What Made It Work

Rehearsal in staging was non-negotiable. Every migration ran against production-like data in staging before the real thing. We caught schema mismatches, encoding issues, and timeout problems in rehearsal, never in production.

Step Functions as the orchestrator meant the migration was the code. No runbooks, no manual steps, no “ssh into this box and run this script.” The entire migration was a versioned, tested state machine. If something went wrong, the execution history told us exactly what happened.

Conservative parallelism. We could have pushed harder on concurrent data streams, but the overnight window was generous enough that we traded speed for safety. Lower parallelism meant lower peak load on both source and target systems.

The Context

The migration infrastructure was purpose-built for this acquisition and ran its course. The system did its job and was decommissioned after the final migration completed.

Enterprise data migration is unglamorous work, but it’s where engineering discipline matters most. There’s no “move fast and break things” when you’re moving real customer data for a nine-figure acquisition.