CSV vs Parquet for Market Data (OHLCV): What to Use

Price bars and trades are stored as files long before they are analysed. Here is how CSV and Parquet compare for OHLCV data.

Open the CSV to Parquet converter

For quant developers and quant researchers, OHLCV data has a fixed shape: a timestamp, an instrument identifier, open, high, low, close and volume. It is also repetitive. The same symbol appears on thousands of rows, timestamps follow a pattern and prices move in small steps. That is the kind of data Parquet compresses well.

Our own size test used 100,000 rows of order data, not market data, so treat it as an example of the pattern and not as a market benchmark. The CSV was 6.87 MB. The Parquet file made with the Data Utility converter was 1.84 MB, about 73 percent smaller. A gzipped CSV was 1.41 MB. Your OHLCV files will give different numbers, so measure your own.

Size is only part of it. A CSV stores everything as text, so every reader has to guess the types again. A timestamp can be read as a string, an identifier such as 000123 can lose its zeros, and a price can be read as text because of one stray character. Parquet stores a type for every column, so a timestamp stays a timestamp and volume stays an integer.

Parquet is also stored by column. A tool such as DuckDB, pandas or Spark can read only the close and volume columns without scanning the rest. For research on large histories this is often a bigger gain than the file size.

Keep CSV when a person will open the file in Excel, when a vendor only accepts CSV, or when the file is small. Use Parquet for large histories, repeated analysis and pipelines. A practical rule is to receive CSV, convert it once to Parquet, and keep both. Convert with the CSV to Parquet tool, then compare the two files on your own data. This article is about file formats and is not investment advice.

Open the CSV to Parquet converter

Related tool pages

  • JSON, CSV, Parquet and Excel Converter: Convert between JSON, CSV, Parquet and Excel online for free, one file or a whole folder. It runs in your browser, so nothing is uploaded.
  • JSON to CSV Converter: Convert JSON to CSV in seconds. Drop a file or paste data and download the result. Private: it runs in your browser and nothing is uploaded. Large files too.
  • CSV to JSON Converter: Convert CSV to JSON in seconds. Drop a file or paste data and download the result. Private: it runs in your browser and nothing is uploaded. Large files too.
  • JSON to Parquet Converter: Convert JSON to Parquet in seconds. Drop a file or paste data and download the result. Private: it runs in your browser and nothing is uploaded.
  • Parquet to CSV Converter: Convert Parquet to CSV in seconds. Drop a file or paste data and download the result. Private: it runs in your browser and nothing is uploaded. Large files too.
  • Parquet to JSON Converter: Convert Parquet to JSON in seconds. Drop a file and download the result. Private: it runs in your browser and nothing is uploaded. Large files too.

How to use the CSV to Parquet converter: step by step

  1. Open the CSV to Parquet converter.
  2. Drop in a sample of your OHLCV CSV.
  3. Convert, then compare the two file sizes.
  4. Check that timestamps and prices kept the right types.

When to use the CSV to Parquet converter

  • Decide how to store a history of price bars
  • Convert vendor CSV files to Parquet once
  • Estimate the storage saved on a data lake
  • Explain the trade-off to a colleague

Example

Input: id,name,age 1,Ada,36 2,Alan,41

Output: A Parquet file with the same columns (id, name, age) and typed values.

Frequently asked questions

How do I convert CSV to Parquet?

Drop your CSV file (or paste the data), check the options, click Convert and download the Parquet result.

How big can a file be?

It depends on your device. A few hundred MB usually works.

Which conversions does the Format Converter support?

Any direction between JSON, CSV, Parquet and Excel (.xlsx), for one file or a whole folder. Nested JSON is flattened into columns such as address.city. For Excel, the first sheet is read; old .xls files are not supported.

Why did my number or date change after converting?

Types are detected from the values. Turn off "Detect types" to keep everything as text, which also keeps leading zeros in IDs and phone numbers.

Tip: Test on a month of your own data first. Compression depends on how repetitive the file is.