# Validate a Data Migration by Comparing Source and Target Files

> Prove a migration or format conversion kept every row and value: export source and target, compare by key, and read missing, extra and changed records.

Page: https://datautility.in/blog/validate-data-migration-compare-source-and-target
This is the Markdown copy of the page. The tool itself runs in the browser: your files are processed on your own device and are not uploaded.

Published 2026-11-19 · Analyze & protect · validate data migration, data migration testing, compare source and target data, data reconciliation csv parquet, verify csv to parquet conversion, migration row count check, data migration checklist, compare database export files

After a migration, a conversion or a pipeline rewrite, the question is always the same: did everything arrive, unchanged? Row counts are a start, but equal counts can hide missing rows and changed values. This guide gives a short validation routine using file exports from both sides and a comparison by key, using free tools in your browser.

Open the tool: [Data Comparison](https://datautility.in/data-comparison)

## Why row counts are not enough

Two tables can both have 1,000,000 rows, with 500 rows missing and 500 duplicated, or with every amount rounded. Count is the first check, not the last.

## The routine

### 1. Export both sides

Export the source table and the target table to files with the same columns. CSV, Parquet, JSON, Avro and Excel all work, and the two sides do not need the same format.

### 2. Profile each side

Open each file in the [data profiler](https://datautility.in/data-profiler). Compare row counts, null counts per column and duplicate keys. A difference here already points to the problem.

### 3. Compare by key

Load both in [Data Comparison](https://datautility.in/data-comparison) and choose the primary key. Read the result in this order:

- **Only in A (removed):** rows lost in the migration.

- **Only in B (added):** rows that should not be there, such as duplicates or test rows.

- **Changed:** rows present on both sides with different values. The tool shows the columns that differ and the old and new value.

- **Matched %:** a quick measure for the report.

### 4. Remove false differences

- A tolerance for rounded decimals.

- Ignore spaces and case if the new system trims or upper-cases text.

- Dates and numbers are compared by meaning, so `2026-01-05` equals a midnight timestamp and `10` equals `10.0`.

### 5. Check aggregates with SQL

For totals, use [SQL on your files](https://datautility.in/sql-on-your-files): sum amounts by month on both sides and compare. Equal rows and equal totals together are strong evidence.

### 6. Keep the evidence

Download the differences file. If the result is identical, keep the screenshot or the summary with the migration ticket.

## Format conversions

For a CSV to Parquet conversion, the specific check is in [compare CSV with Parquet](https://datautility.in/compare-csv-with-parquet). See also [how to convert CSV to Parquet](https://datautility.in/blog/how-to-convert-csv-to-parquet).

## Limits

This validates the data in the files you export. It does not check constraints, indexes, permissions or performance in the target system. Very large tables are better compared in the database itself, or in samples by key range. Compare two snapshots; it is not a live check.

## How to use the Data Comparison: step by step

1. Export source and target.
2. Profile both sides.
3. Compare by primary key.
4. Check totals with SQL and keep the evidence.

## When to use the Data Comparison

- Migration sign-off
- Verifying a format conversion
- Finding lost and duplicated rows
- Reconciling two systems

## Example

Input:

```
before.csv + after.csv, key = id
```

Output:

```
Added 3 · Removed 2 · Changed 5
```

## Frequently asked questions

### How do I validate a data migration?
Export source and target, profile both, then compare by primary key to list rows missing, extra and changed. Check totals with SQL as well, and keep the result as evidence.

### How do I verify a CSV to Parquet conversion?
Load the CSV as A and the Parquet file as B, choose the id column and compare. An identical result means every row and value survived.

### Are matching row counts enough?
No. Equal counts can hide missing and duplicated rows or changed values. Compare by key.

### Can it compare very large tables?
Files are limited by your browser memory. For very large tables, compare samples by key range, or compare inside the database.

Tip: Compare a sample from every key range as well as the whole file. Problems often hide at the edges: the first and last records, or one day of data.

## Related tool pages

- [Compare Two CSV Files](https://datautility.in/compare-two-csv-files): Compare two CSV or Excel files by a key column and see added, removed and changed rows. Free, runs in your browser.
- [Compare Two Excel Files](https://datautility.in/compare-excel-files): Compare two Excel (.xlsx) files by a key column and see every added, removed and changed row and cell. Free, and your files stay in your browser.
- [Compare Two Parquet Files](https://datautility.in/compare-parquet-files): Compare two Parquet files by a key column and see every added, removed and changed row and cell. Free, runs in your browser, nothing is uploaded.
- [Compare Two JSON Files](https://datautility.in/compare-json-files): Compare two JSON files by a key field and see which records were added, removed or changed, down to the nested field. Free, in your browser.
- [Compare Two Avro Files](https://datautility.in/compare-avro-files): Compare two Avro files by a key field and list the added, removed and changed records. Reads the embedded schema, runs in your browser. Free.
- [Compare CSV with Parquet](https://datautility.in/compare-csv-with-parquet): Check that a Parquet file matches the CSV it was made from: find rows and values that differ after a conversion or migration. Free, in your browser.

## Also try

- [Compare files in different formats](https://datautility.in/blog/compare-two-files-different-formats-csv-json-parquet-excel-avro)
- [Compare financial reports](https://datautility.in/blog/compare-financial-reports-reconciliation-files)
- [Compare a CSV and a JSON file](https://datautility.in/blog/compare-csv-and-json-files-by-key)

## More guides

- [Data Profiler: Check Data Quality Before You Use It](https://datautility.in/blog/data-profiler-check-data-quality)
- [Compare Two CSV or Excel Files: Find Added, Removed and Changed Rows](https://datautility.in/blog/compare-two-csv-files-find-differences)
- [Data Masking: Hide Emails, Phones and Names Before Sharing](https://datautility.in/blog/data-masking-hide-emails-phones-pii)
