Cloud Data Migration: A Practitioner's Guide to Moving to AWS, Azure, GCP, Snowflake, and Databricks
Cloud data migration is the planned movement of data and analytic workloads from on-premises or first-generation cloud environments to modern cloud data platforms. Done well, it produces lower fixed cost, better elasticity, faster analytics, and easier integration. Done poorly, it produces a surprise bill and the same reports running slightly slower. This guide covers how to do it well.
Snowflake, Databricks, Azure, AWS fluent
HQ Atlanta, serving US, Canada, Europe
Cloud data migration has five phases: assess, design, migrate, validate, optimize. The most important early decisions are:
- target cloud (AWS, Azure, GCP),
- target data platform (Snowflake, Databricks, native cloud warehouse),
- ingestion pattern (Fivetran, native, custom),
- lift-and-shift versus modernize-in-place,
- decommissioning timeline for the source.
How do you migrate data to the cloud?
A cloud data migration follows a defined sequence: assess the source environment and confirm drivers, design the target architecture and migration approach, execute the migration in waves, validate the target against the source, then optimize and decommission. The work is conceptually identical to any data migration; the cloud-specific complications come from cost modeling, security and identity integration, network topology, and the breadth of target platform choices.
The two early decisions that shape everything else are target cloud provider and target data platform. The cloud provider is often constrained by existing relationships, geographic footprint, and the rest of the enterprise architecture. The data platform decision (Snowflake, Databricks, native cloud warehouse, or a combination) deserves more scrutiny than it usually gets, because it shapes cost, capability, and skill requirements for years.
The single most important framing question is whether to lift-and-shift or modernize-in-place. Lift-and-shift moves what exists with minimal change, accepting that modernization will follow. Modernize-in-place reshapes the data as part of the move. Both are valid. The wrong choice is to commit to one without naming the trade-offs.
- Target cloud (AWS, Azure, GCP) and target data platform (Snowflake, Databricks, native) are the two foundation decisions.
- Lift-and-shift is faster; modernize-in-place captures more value but adds risk.
- Cost surprises usually come from egress, storage, and uncontrolled compute.
- Ingestion patterns: Fivetran for breadth, dbt for transformation, native cloud tools for tight coupling.
- Validation and reconciliation are first-class phases, not afterthoughts.
- Decommissioning the source is part of the migration scope.
Cloud provider and data platform: how to choose
We are vendor-neutral and fluent across AWS, Azure, GCP, Snowflake, and Databricks. Selection depends on the client’s situation, not a partner quota.
- Target cloud (AWS, Azure, GCP). Existing enterprise relationships often dominate. If the business is already on Microsoft 365 and Azure AD, Azure has gravity. AWS tends to win on breadth of services and engineering team familiarity. GCP wins where the analytics workload is the center of gravity and BigQuery is a strong fit. Multi-cloud is a real option for some workloads, expensive for others.
- Target data platform (Snowflake, Databricks). Snowflake is strong for SQL-centric analytics workloads, ease of operation, and predictable cost modeling. Databricks is strong for combined analytics and ML workloads, open-format flexibility (Delta, Iceberg), and engineering-heavy environments. Native cloud warehouses (Redshift, Synapse, BigQuery) are viable, particularly where workload patterns align tightly with the provider’s strengths.
The decision is rarely “best platform.” It is “best fit for this team, this workload, this cost envelope, this five-year direction.”
Pipeline and operating choices
Ingestion.
Fivetran is a strong default for breadth of source connectors and operational simplicity. Native cloud services (AWS DMS, Azure Data Factory, GCP Dataflow) suit tight integration with provider services. Custom pipelines (Airflow, Dagster) win where source systems are unusual or volumes demand precision.
Transformation.
dbt has become the standard for SQL-based transformation, with strong fit across Snowflake, Databricks, and native warehouses. Spark-based transformation (native to Databricks) suits ML and high-volume workloads. Choose based on team skill, not fashion.
Governance.
A modern data platform without governance produces faster sprawl. Collibra and similar catalogs integrate with Snowflake and Databricks for lineage, glossary, and stewardship. Governance work runs alongside the migration, not after.
BI and analytics.
Qlik (our deepest partner expertise), Power BI, Tableau, and Looker all integrate cleanly with modern cloud data platforms. The BI layer often migrates after the data platform stabilizes.
The categories are not always staffed separately, but they are always present. Underinvesting in any one of them creates fragility in the others.
Avoiding the cost and security surprises
Cloud cost surprises follow a pattern. Egress fees on data crossing cloud boundaries. Storage costs growing faster than expected because nothing gets retired. Compute running uncontrolled because there are no guardrails. Reserved capacity left on the table.
We model these in the design phase: storage growth assumptions, compute patterns, egress flow, reserved capacity strategy, autoscaling guardrails, and FinOps review cadence. Cost is a design problem, not an operational discovery.
Security and identity integration is the other recurring surprise. Cloud identity (Entra ID, AWS IAM, GCP IAM), data classification, network topology, and key management need to be specified in the design, not improvised in execution. Penetration testing and compliance review (HIPAA, SOC 2, PCI, where relevant) are part of the migration scope.
How DI Squared sequences a cloud data migration
We follow our standard migration sequence, with cloud-specific tasks layered in.
- Discover. Discover assesses the current pipelines, platform, models, and operating practices, plus the team’s skill and capacity.
- Map. Map documents the target state, the sequencing, and the platform decisions, ending in a plan the team and the CFO can both work from.
- Navigate. Navigate is the build. We prioritize the most valuable first wins (often a pipeline modernization or a finance data mart), deliver them, and run alongside the internal team through change management.
- Adjust. Adjust is the long horizon, where we tune the platform, retire technical debt, and adapt the architecture as the business evolves.
Cloud Data Migration, answered.
Q: How long does a cloud data migration take?
A: Highly variable. A focused workload migration to Snowflake or Databricks can run a few months. A full enterprise migration involving multiple source systems, BI, and governance runs longer. Volume, source documentation, downtime tolerance, and the lift-and-shift versus modernize decision are the main drivers.
Q: Should we lift-and-shift or modernize during the migration?
A: Both are valid. Lift-and-shift is faster and lower risk; it accepts that modernization will follow. Modernize-in-place captures more value but adds complexity and timeline. Most engagements blend: lift-and-shift the bulk, modernize the highest-value workloads as part of the move.
Q: How do we control cloud cost during and after migration?
A: Model cost in the design phase, set storage and compute guardrails, establish a FinOps cadence (often monthly), and retire the source on a defined timeline. Cost discipline is built in, not added later.
Q: Snowflake or Databricks?
A: Both are excellent. Snowflake fits SQL-centric analytics environments seeking operational simplicity and predictable cost. Databricks fits engineering-heavy environments combining analytics and ML, with open-format flexibility. We help clients choose based on workload, team, and direction, not partner quota.
Q: Do we need a multi-cloud strategy?
A: Usually no. Multi-cloud adds complexity, egress cost, and skill requirements. It is justified for specific reasons (regulatory residency, workload-specific provider strengths, M&A integration). A deliberate single-cloud-plus-exceptions strategy is more common and more practical.
Q: When should we migrate the BI tools (Qlik, Power BI, Tableau)?
A: Usually after the data platform stabilizes, in a dedicated wave. The BI migration is its own scope: re-platforming, content rationalization, security model rebuild, and user training. Skipping the rationalization step is the most common BI migration mistake.
Planning a move to AWS, Azure, GCP, Snowflake, or Databricks?
We have run cloud migrations for clients across utilities, healthcare, manufacturing, and financial services. Tell us what you are moving and what the constraints are.