Teradata, Netezza or Hadoop to Snowflake
Cohash moves on-premises Teradata, Netezza and Hadoop warehouses to Snowflake. We start from query logs, so only the data and jobs people still use get moved. SnowConvert AI converts the code it supports and engineers rewrite the rest. Every table is reconciled against the source before reports switch, and warehouses are sized per workload.
- 01Workload inventory from query logs
- 02Code conversion
- 03Data movement
- 04Security and cost model
- 05Parallel run and reconciliation
- 06Cut-over and decommission
The story
Told without the client's name or anything that could identify them.
A logistics company's data team ran its warehouse on a Teradata appliance due for a hardware refresh, alongside a Hadoop cluster still on CDH 6.3, which lost support in March 2022. BTEQ scripts ran from cron and only a few people knew which ones mattered, and analysts had started copying extracts into Snowflake by hand to get around the queue. We began with DBQL and Hive audit logs so that only tables and jobs people still used were moved. SnowConvert AI handled the DDL, views and BTEQ it supports, engineers rewrote the rest as dbt models, and every table was reconciled with row counts, checksums and aggregates before reports switched. Consumers moved in waves, and the appliance was archived and shut down.
why it was urgentPlenty of Hadoop clusters still run on releases Cloudera stopped supporting years ago: HDP 3.1 lost support in December 2021 and CDH 6.3 in March 2022. On the conversion side, SnowConvert AI now handles Teradata tables, views, procedures, BTEQ, MLOAD and TPUMP scripts, and Netezza and Hive tables and views too.
- openThe appliance or cluster is due a hardware refresh, and nobody wants to sign for it.
- recurringThousands of BTEQ or nzsql scripts run from cron, and only a few people know which ones matter.
- blockedThe Hadoop cluster still runs CDH or HDP, on a release out of support for years.
- openAnalysts already copy extracts into Snowflake by hand because the warehouse queue is too long.
The estate, before and after
Before: a Teradata appliance next to a Hadoop cluster out of vendor support
ETL loads fed a Teradata appliance and a Hadoop cluster, with BTEQ scripts on cron and Oozie workflows doing the transforms. BI tools, Excel and SAS read the appliance, and analysts copied extracts to Snowflake by hand.
After: everything runs in Snowflake, with dbt, SCIM roles and a warehouse per workload
History is exported to cloud object storage and loaded with COPY INTO. Changed data keeps flowing in until cut-over. After that, dbt models do what BTEQ and Oozie used to, roles come from the identity provider over SCIM, and Terraform defines the warehouses and resource monitors.
How we run it, phase by phase.
6 phases take you from Teradata, Netezza or Hadoop (on-premises) to Snowflake. Each one ends with something you can check before the next starts. As you scroll, the blocks show the estate moving across.
- phase 01
Workload inventory from query logs
Pull Teradata DBQL, Netezza query history or Hive and Impala audit logs. Rank tables and jobs by recent use, map who reads them, and mark anything untouched for months as a retirement candidate.
- phase 02
Code conversion
Run SnowConvert AI on DDL, views, procedures and Teradata BTEQ, MLOAD and TPUMP scripts, then fix what it flags. Netezza NZPLSQL procedures, Spark jobs and Oozie workflows sit outside its coverage and are rewritten by hand, usually as dbt models plus an orchestrator.
- phase 03
Data movement
Bulk-export history to cloud object storage (TPT export, Netezza external tables or DistCp from HDFS) and load it with COPY INTO. Changed data keeps flowing through CDC or nightly deltas until cut-over.
- phase 04
Security and cost model
We rebuild roles from the source's roles and profiles, bring users in through SCIM from the identity provider, and split compute into a warehouse per workload, each with a resource monitor.
- phase 05
Parallel run and reconciliation
Compare row counts, column checksums and key aggregates table by table, and run the reports side by side. Nobody moves until every difference is explained or fixed.
- phase 06
Cut-over and decommission
Switch BI tools, extracts and downstream feeds in waves, keep the source read-only for an agreed period, then archive what audit needs and shut the platform down.
What the client got
What changed for the client once the work was done. Described, not scored: we don't publish client figures.
- result 01
Only data and jobs in use were moved
Query logs ranked every table and script by recent use. Anything nobody had touched in months was retired, not converted.
- result 02
Converted code lives in version control
DDL, SQL and dbt models are in a repository. The error-code and activity-count branching that used to live in BTEQ is now in the orchestrator, where a failure actually gets caught.
- result 03
Every table was reconciled before consumers moved
For every table we compared row counts, column checksums and the main aggregates. Differences from case-insensitive comparison and SET tables got explained or fixed.
- result 04
Compute is split by workload, with limits
We gave each workload its own warehouse with a resource monitor, rebuilt roles from the source and brought users in through SCIM. The design is kept as code.
- result 05
Consumers nobody had listed were switched too
Connection logs turned up Excel ODBC connections, SAS jobs and partner feeds. We moved them in planned waves, each with a fallback point.
Risks we plan for
-
Quiet semantic differences
Teradata compares strings case-insensitively by default and SET tables silently drop duplicate rows. Snowflake does neither. Converted SQL can run cleanly and still return different numbers.
-
Control flow buried in scripts
BTEQ scripts branch on error codes and activity counts. That logic has to move into the orchestrator explicitly, or failures stop being caught.
-
Lift-and-shift cost
Queries tuned for an appliance's always-on hardware can scan far more than they need once you pay per use. We review the most expensive ones before go-live. The first invoice is a bad time to find them.
-
Consumers nobody listed
Excel ODBC connections, SAS jobs and partner file drops often appear only in connection logs. Missing one means a broken process on day one.
Where AI helps, and where it needs guardrails
On the Snowflake side, CoWork and Horizon Catalog's Agent Identity let agents query governed data under their own identity. That only works once roles are clean. SnowConvert AI's AI-assisted fixes are listed for SQL Server, Redshift, BigQuery and PostgreSQL, but not Teradata or Netezza. So we use an LLM to explain old BTEQ scripts and draft test cases, and an engineer reviews each one. Agents get read-only access to query history and never touch production data.
forward deployed engineers →What you're left with
We write these for your setup, and they're working documents. Your team should be able to keep things running from them long after we've gone.
- inventory01
Inventory of tables, scripts and consumers, ranked by actual use, with a move or retire call on each
yours to keep - design02
Converted DDL, SQL and dbt models, all checked into version control
yours to keep - report03
We reconcile every table against the source and explain each difference in a report.
yours to keep - code04
We hand over the Snowflake roles, warehouses and resource monitors as code.
yours to keep - runbook05
Cut-over runbook by consumer wave, with a fallback point
yours to keep
Platforms and services on this move
Learn the background
Terms from this page in plain English. If you want the why, the field note behind it is right there.
- 01SCIM
- A way for a company's directory to push users and groups into another system. Auth0 can then keep those groups as their own objects instead of on the profile.in context →
- 02Identity provider
- Whatever checks your password and MFA and then tells each app you're signed in. Okta and Entra ID both play that role, and admins call it the IdP.in context →
- 03refresh
- Before planning, Terraform asks each API what really exists and updates state to match. That's the refresh, and on big states it's the slow part.in context →
- 04Orchestrator
- Think of it as the robots' dispatcher. It holds the schedules, queues and credentials, and tells each robot what to run next.in context →
Certified engineers, wherever you need them.
Some clients want a project team, some want a couple of engineers on theirs. Either works. Onsite, nearshore or offshore, with handovers across time zones that keep things moving.
Onsite
Engineers at your sites in the US
Nearshore
Overlapping hours with US teams
Offshore
Overnight progress on your backlog
All time zones
Coverage handed over across time zones
01Should we go to Snowflake or Databricks?
It depends on the workload. A SQL-heavy warehouse with lots of BI users maps closely to Snowflake. If the Hadoop side is mostly Spark, streaming or ML, Databricks may fit better, and some companies run both. We look at the query mix before we recommend either.
02How long does a warehouse migration take?
Mostly on the objects and scripts still in use, how much data there is, and how many consumers downstream need to switch. We scope it after the inventory. The quickest way to shorten it is to retire unused objects first.
03Does SnowConvert AI do the whole conversion?
No tool does. It covers a lot of Teradata code and Netezza and Hive tables and views. Procedures it doesn't support, scheduling logic and semantic differences still need engineers, and every converted object is tested against source data.
04Can we keep our ETL tool?
Often, yes. Informatica, Fivetran and most ETL tools can target Snowflake. Some teams use the move to shift transformations into dbt inside Snowflake, which we plan as a separate decision.
05How is a Hadoop to Snowflake migration different?
The SQL is the smaller part. SnowConvert AI covers Hive and Spark SQL tables and views, but Spark job code and Oozie workflows sit outside it, so engineers rewrite them as dbt models or Snowpark code run by an orchestrator. Files leave HDFS through DistCp to cloud storage and load with COPY INTO. Read the Hive and Impala audit logs first, because old clusters hold a lot of data nobody queries.
Often on the same roadmap
On-premises Active Directory to cloud-first Microsoft Entra ID
We shrink on-premises Active Directory one application at a time, moving groups, users and app authentication to Microsoft Entra ID as each app is ready.
read the case studyAD FS federation to Entra ID authentication
Converting federated domains from AD FS to Entra ID cloud authentication, moving relying parties app by app and retiring the AD FS farm.
read the case study
Apache Airflow 2 to Airflow 3
Upgrading self-managed Apache Airflow 2, now past end of life, to Airflow 3, and fixing removed features, auth and scheduling changes before DAGs move.
read the case study3fb6360c - 01Book a 30-minute working sessionsenior engineer
- 02Talk to an engineer about your setupreply within 24 hours
- 03See how similar projects wentcase studies
Planning this move?
A 30-minute working session with a senior engineer. No slides.
- The platform and the version you're on
- Where you need to get to, and by when
- What's custom: rules, plugins, integrations
- Any audit finding or deadline driving it