Data Lineage: Migration Playbook

Dashboard Trust

Data lineage is most useful during a migration when it answers three practical questions: what depends on this data, what will break if we change it, and how will we prove the new path is correct? A lineage map does not need to be perfect to be useful. It needs to be accurate enough to protect the dashboards, metrics, downstream jobs, and decisions that matter.

The migration problem lineage solves

Most migration failures are not caused by moving bytes from one system to another. They are caused by hidden dependencies.

A table looks unused until a finance dashboard goes blank. A column looks safe to rename until a sales metric changes. A transformation looks duplicated until someone explains that one version handles refunds and the other does not.

Data lineage gives the migration team a working map of those dependencies. It shows the path from source data to transformed models to semantic definitions to dashboards, alerts, reverse ETL jobs, notebooks, and operational extracts.

For migration work, lineage should be treated as an operating tool, not documentation theater. The goal is not a beautiful graph. The goal is fewer surprises at cutover.

What counts as data lineage in a migration

In a migration, lineage has two useful directions.

• Upstream lineage: where a dataset, model, metric, or dashboard gets its inputs.
• Downstream lineage: what consumes a dataset, model, metric, or field.

You need both. Upstream lineage helps you rebuild and validate the new system. Downstream lineage helps you understand blast radius before changing anything.

Lineage can exist at several levels of detail:

• System level: source application to warehouse to BI tool.
• Dataset level: source table to staging model to mart table.
• Field level: source column to transformed column to dashboard measure.
• Metric level: business definition to SQL logic to dashboard tiles.
• Job level: scheduled task to output dataset to downstream task.

Field-level lineage is valuable, but it is not always required on day one. For most migration planning, start with system-level and dataset-level lineage, then deepen the map around high-risk metrics and executive dashboards.

Lineage question	Why it matters during migration	Typical evidence
Where did this number come from?	Helps rebuild the upstream logic correctly.	Source tables, transformation code, model dependencies, metric definitions
Who uses this dataset?	Shows blast radius before changing or retiring it.	BI usage, query logs, downstream jobs, stakeholder interviews
What changes if this field is renamed?	Prevents broken reports, extracts, and jobs.	Column references, dashboard fields, semantic layer mappings
Can we retire this asset?	Reduces migration scope without guessing.	Usage history, owner approval, replacement mapping
How do we validate the new path?	Turns migration success into evidence, not opinion.	Row counts, reconciliation queries, freshness checks, metric comparisons

Set the lineage scope before mapping everything

A common mistake is trying to map the entire data estate before making progress. That usually creates a long inventory exercise and delays the migration without reducing much risk.

Start with a narrow scope tied to the migration objective. Examples:

• Migrating a BI tool: prioritize dashboards, semantic definitions, published datasets, user permissions, and query dependencies.
• Migrating a warehouse: prioritize ingestion jobs, transformation models, orchestration schedules, downstream exports, and warehouse-specific SQL.
• Refactoring a metrics layer: prioritize metric definitions, filters, joins, grain, ownership, and dashboards that consume each metric.
• Replacing pipelines: prioritize source extraction logic, incremental loading rules, backfills, failure handling, and consumers of each output table.

The scope should identify which assets must be migrated, which can be retired, which need validation, and which require stakeholder sign-off.

Build a migration inventory that people can trust

The inventory is the practical foundation of lineage. It should list the assets affected by the migration and connect them to owners, consumers, and risk.

Do not rely on one source. Query logs, BI usage data, orchestration metadata, transformation code, warehouse object references, and stakeholder interviews all reveal different parts of the map.

A useful inventory includes:

• Asset name: table, model, dashboard, metric, job, report, extract, or notebook.
• Asset type: source, staging, intermediate model, mart, metric, dashboard, export, or operational process.
• Owner: the person or team that can approve changes.
• Upstream inputs: the data this asset depends on.
• Downstream consumers: dashboards, jobs, teams, or decisions that depend on it.
• Usage signal: query frequency, dashboard views, scheduled delivery, or stakeholder confirmation.
• Business criticality: executive reporting, finance close, customer operations, growth analysis, compliance support, or ad hoc use.
• Migration action: migrate, rebuild, validate, retire, merge, or defer.

The inventory will not be perfect at first. Mark confidence explicitly. A low-confidence dependency is not a failure; it is a prompt for investigation before cutover.

Map lineage in layers, not all at once

The fastest path to useful lineage is layered mapping. Each layer answers a different migration question.

1. System map: Which tools exchange data?
2. Pipeline map: Which jobs create which datasets?
3. Model map: Which transformations depend on which tables?
4. Metric map: Which business definitions depend on which models and fields?
5. Dashboard map: Which dashboards, alerts, and reports depend on those metrics and datasets?
6. Operational map: Which reverse ETL syncs, customer exports, spreadsheets, or notebooks consume the data?

This layering keeps the migration team from getting lost in detail too early. First identify the major roads. Then inspect the intersections where business risk is highest.

Classify dependency risk before changing systems

Lineage is only useful if it changes the migration plan. After mapping dependencies, classify each asset by risk and action.

High-risk assets usually share at least one trait: many downstream consumers, unclear ownership, complex business logic, warehouse-specific SQL, high executive visibility, financial impact, or fragile freshness expectations.

Low-risk assets are not automatically safe to ignore. They may be unused, but they may also be invisible because usage tracking is incomplete. Treat low usage as a signal, not proof.

For each important asset, assign one migration action:

• Lift: move the asset with minimal logic changes.
• Rebuild: recreate the logic in a new pattern or tool.
• Merge: consolidate duplicates into one governed version.
• Retire: remove unused or obsolete assets after confirmation.
• Defer: leave the asset outside the first cutover because it is not needed yet or is too risky to move now.

Risk level	Signs	Recommended migration handling
High	Executive dashboard, finance metric, customer-facing export, many downstream dependencies, unclear logic	Assign owner, map field or metric lineage, run parallel validation, require sign-off
Medium	Team dashboard, recurring analysis, moderate usage, known owner, some transformation complexity	Validate key outputs, document differences, migrate in planned cutover window
Low	Low usage, clear replacement, no critical consumers, simple logic	Confirm retirement or migrate after critical path
Unknown	No owner, incomplete usage data, dynamic SQL, notebook logic, manual spreadsheet dependency	Investigate before cutover or isolate from first release

Use lineage to plan cutover order

Cutover order should follow dependency order. Move and validate upstream foundations before downstream dashboards. If a dashboard is migrated before its metric logic is stable, users will test presentation instead of correctness.

A practical cutover sequence is:

1. Freeze or version the business-critical definitions that must not drift during migration.
2. Confirm source extraction and ingestion behavior in the new path.
3. Rebuild staging and core transformation layers.
4. Backfill the historical range needed for comparison.
5. Validate row counts, freshness, primary keys, accepted values, and metric outputs.
6. Migrate dashboards and downstream jobs only after their underlying datasets pass checks.
7. Run old and new paths in parallel for critical metrics when the cost is justified.
8. Switch users to the new path with a rollback plan and named owners.

Lineage turns this from a tool-by-tool checklist into a dependency-aware plan.

Validate the lineage path, not just the final dashboard

A dashboard can match for the wrong reason. A migrated report may show the same revenue total while hiding differences in refunds, time zones, filters, or customer attribution.

Validation should happen at multiple points in the lineage path:

• Source extraction: did the same business events arrive?
• Staging: did parsing, casting, deduplication, and incremental logic behave as expected?
• Core models: did joins preserve grain and avoid duplication?
• Metric layer: did definitions, filters, windows, and exclusions remain consistent?
• Dashboard layer: did visualization filters, date controls, and calculated fields migrate correctly?

The right validation depends on the asset. A finance metric may need exact reconciliation. A product usage trend may tolerate small known differences if the cause is documented. A customer-facing operational export may need record-level comparison.

Layer	Good checks	What the checks catch
Source ingestion	Record counts, freshness, duplicate detection, primary key coverage	Missing loads, delayed data, accidental duplication
Transformation models	Join grain tests, accepted values, null checks, reconciliation to old models	Logic drift, row multiplication, type changes
Metrics	Definition comparison, filter review, time window checks, segmented totals	Metric drift, date handling differences, changed exclusions
Dashboards	Tile-by-tile comparison, filter parity, calculated field review, refresh schedule check	BI-layer differences, broken visuals, stale reports
Operational consumers	Sample file comparison, downstream job success, stakeholder acceptance	Broken exports, reverse ETL issues, workflow failures

Protect dashboard trust during the migration

Dashboard trust is damaged when users discover issues before the migration team does. Data lineage helps you prevent that by showing which dashboards sit downstream of risky changes.

Before cutover, create a trust checklist for high-visibility dashboards:

• Who owns the dashboard?
• Which business decision does it support?
• Which tables, models, metrics, and fields power it?
• Which filters, calculated fields, and date logic are embedded in the BI tool?
• What are the expected freshness and refresh schedules?
• What historical period should match?
• What differences are acceptable, and who can approve them?
• What message will users see if a dashboard is not ready at cutover?

This work is not glamorous, but it prevents the common migration pattern where the data team declares success and business users spend the next month finding broken numbers.

Common lineage failure modes in migrations

Lineage work fails when it becomes either too shallow or too theoretical.

• Only mapping tables: table lineage misses dashboard calculations, spreadsheet extracts, reverse ETL jobs, and metric definitions.
• Ignoring ownership: a dependency without an owner cannot be validated or retired safely.
• Trusting automated lineage blindly: parsers and metadata tools help, but they may miss dynamic SQL, manual exports, notebook logic, or BI-only calculations.
• Skipping usage context: a dashboard viewed once a quarter may still matter if it supports board reporting or finance close.
• Changing definitions during migration: a migration is already a change event. Redefining metrics at the same time makes validation harder unless it is explicitly planned.
• Not recording decisions: if the team cannot explain why an asset was retired, rebuilt, or excluded, the same debate returns during cutover.

The practical fix is to pair automated discovery with human confirmation for critical assets.

Keep lineage useful after the migration

Lineage loses value when it is treated as a one-time migration artifact. The map should become part of how the data system is operated.

After cutover, keep a lightweight operating model:

• Require owners for important datasets, metrics, and dashboards.
• Update lineage when new production models or dashboards are created.
• Review downstream impact before changing or deleting fields.
• Track retired assets so users understand where old reports went.
• Attach validation evidence to critical migrated metrics.
• Use incidents and dashboard bugs to improve the lineage map.

The long-term goal is not perfect documentation. The goal is change safety. When someone asks, “What happens if we change this?”, the team should be able to answer without starting from zero.

Data lineage migration checklist

Use this checklist to make the playbook actionable.

1. Define the migration scope and the systems included.
2. List critical business processes, dashboards, metrics, and operational exports.
3. Collect metadata from the warehouse, BI tool, orchestrator, transformation project, and query logs.
4. Create an inventory with owners, upstream inputs, downstream consumers, usage, and confidence level.
5. Map lineage in layers: systems, pipelines, models, metrics, dashboards, and operational consumers.
6. Classify assets by business criticality and migration action.
7. Identify high-risk dependencies and unresolved ownership gaps.
8. Plan cutover order based on dependency order.
9. Backfill the historical period needed for meaningful comparison.
10. Validate at source, model, metric, and dashboard levels.
11. Get stakeholder sign-off for critical differences or definition changes.
12. Communicate cutover timing, known limitations, and rollback ownership.
13. Retire old assets only after usage and owner confirmation.
14. Keep lineage current after migration through ownership and change review.

Key takeaways

• Data lineage is a migration risk tool, not just a documentation artifact.
• Start with scoped, layered lineage instead of trying to map the entire data estate at full detail.
• Use lineage to decide what to migrate, rebuild, merge, retire, or defer.
• Validate the full lineage path from source to dashboard, especially for trusted metrics.
• Automated lineage helps, but critical dependencies still need ownership, context, and human confirmation.
• After migration, keep lineage connected to change review so dashboard trust does not decay.

Next step

Pick one high-value dashboard or metric in the migration scope. Trace its lineage from source system to final presentation, list every owner and transformation, then define the exact validation checks required before cutover.