Compare CSV with Parquet

Check that a Parquet file matches the CSV it was made from: find rows and values that differ after a conversion or migration. Free, in your browser.

Quick answer: Compare two datasets and see what was added, removed or changed.

Related: Data Comparison (CSV and Excel Diff) · Compare Two CSV Files · Compare Two Excel Files · Compare Two Parquet Files · Parquet Guides and Free Parquet Tools

How to use the Compare CSV with Parquet

Compare two datasets and see what was added, removed or changed.

Data Comparison shows exactly what changed between two versions of a file. Pick a key column, or use the first column or the first two columns, and it lists rows that were added, removed or changed, down to the cell. Use it to check an export against last month, to review a data migration, or to find which values a script changed. CSV, Excel, JSON, Parquet and Avro are supported, and the two files can be different formats.

  1. Load dataset A and B
  2. Pick or type the key column
  3. Review and download the differences

Tips

Check a conversion or a migration

After converting a CSV to Parquet, or moving a table to a new system, you want proof that nothing was lost. Load the original CSV as A and the converted Parquet file as B, choose the id column, and compare. An identical result means every row and every value survived. Otherwise you see the rows that are missing on either side and the cells that differ.

What counts as a real difference

A CSV stores everything as text and Parquet stores types, so a naive comparison reports thousands of false differences. Here, numbers are compared by value (10 equals 10.0), true equals the text true, a blank equals a null, and a date written as text equals the same date stored as a timestamp. A value changes only if it really differs.

Set a number tolerance if the conversion rounded decimals.

Example

Input: export.csv + export.parquet, key = id

Output: Matched 99,998 · Missing 2 · Different 0

Browse by topic

Who uses Data Utility

Frequently asked questions · Data format benchmark

Frequently asked questions

How do I check a CSV to Parquet conversion?

Load the CSV as A and the Parquet file as B, choose the id column and compare. If the conversion kept every row and value, the result is identical. Otherwise you see which rows are missing and which cells differ.

Will it report false differences between CSV and Parquet?

It avoids the common ones. Numbers match by value, so 10 and 10.0 are equal, true equals the text true, and a date written as text matches the same date stored as a timestamp.

Can I compare other format pairs?

Yes. CSV, Excel, JSON, Parquet and Avro can be compared in any combination.

Which file formats can I compare?

CSV, TSV, Excel (.xlsx), JSON, Parquet and Avro, in any combination. For example, compare a Parquet file with the CSV it was made from. Nested JSON and Avro fields are flattened into columns such as address.city.

Will 10 in a CSV match 10 in a Parquet file?

Yes. Values are compared by meaning, not by how they are written: 10 equals 10.0, true equals "true", and a date equals the same date in any format. A value only counts as changed if it really differs.

Can Data Comparison do change data capture (CDC)?

It does the snapshot version of CDC for files. Compare the earlier and the latest export by key: added rows are inserts, changed rows are updates and removed rows are deletes. It does not read database logs or run in real time.

How does Data Comparison find differences?

It matches rows by a key column, or by the first two columns, and shows which rows were added, removed or changed, down to the cell. It works with CSV and Excel files.

Related pages

Topic guides

Guides