What’s The Deal With Apache Parquet?

Apache Parquet is the recommended file format used in every modern data platform, and for good reason. But what are those reasons? And would it really matter if you stuck with CSV?

The short answer is "YES". The slightly longer answer is "Yes, because columns".

The full answer is below, so keep on reading to understand Parquet and why it's become the go-to file format for analytics.

How Columnar Storage Changes the Economics of a Query

CSV (Comma-Separated Values) and most traditional formats store data row by row, with every record written out in full before the next one begins. For a table with 200 columns and 500 million rows, a query that only needs three columns still has to read all 200 columns' worth of data from disk to get them. The irrelevant 197 columns come along for the ride.

Parquet flips this. It stores data column by column: all values for transaction_date together, all values for revenue together, all values for customer_id together. A query that needs those three columns reads only those three columns from storage. The other 197 never leave the disk.

This is known as column pruning, and its impact compounds quickly on wide tables. Combined with predicate pushdown (applying filters before data is read into memory rather than after) the result is that analytical engines read a fraction of the data they would compared with row-based formats. For wide schemas where queries touch only a handful of columns, the I/O reduction is substantial.

The internal structure of a Parquet file reinforces this. Files are divided into row groups (horizontal slices of the data), and within each row group, data is stored by column chunk. Each column chunk carries its own statistics: minimum value, maximum value, null count. When running a query, a query engine reads these statistics first, and if the row group provably can't contain the values being filtered for, it skips the entire group without reading any actual data.

Why Parquet Compresses So Well

Compression ratios (how much smaller a compressed file is compared to the original) are the second area where columnar storage excels. Storing by column means storing the same datatypes together, which allows compression algorithms to be more effective. For example, a column of ISO country codes will contain a small set of repeating strings, while a column of dates will follow allowed patterns.

Dictionary encoding handles the low-cardinality cases automatically: rather than storing "United Kingdom" a million times, Parquet stores a small dictionary and a million integer references to it. Run-length encoding handles repeated sequences and delta encoding handles monotonically increasing integers like timestamps. The cumulative effect is significant: Parquet files are commonly 5–10x smaller than equivalent CSV files. At lakehouse scale, these compaction improvements made a noticeable impact on your infrastructure costs.

There are multiple compression codecs supported at the column level, each with their own strengths: Snappy for speed, Zstd for ratio, and GZIP as a widely-compatible middle ground. You can also utilise different codecs for different columns within the same file, so you can truly optimise your Parquet files around how you use them.

Parquet as the Foundation of the Open Lakehouse

While Apache Iceberg is compatible with multiple file formats, like Avro and ORC, Parquet has become the default file format for Iceberg tables (and other table formats such as Delta Lake), and thus foundational to the modern open lakehouse stack. When Dremio, Spark, Trino, or DuckDB query an Iceberg table, they're reading Parquet files on cloud object storage. The Iceberg layer adds metadata, transaction support, and schema evolution, but Parquet is the physical storage format those engines scan.

Parquet's performance characteristics translate directly into lakehouse query performance. Dremio's Reflections, pre-computed materialised views stored in Parquet on object storage, inherit the same column pruning, predicate pushdown, and compression benefits. Parquet is supported across virtually every analytics engine, language, and cloud platform in production today, which is what makes it the default rather than just a recommendation.

If you want to query Parquet files on your own data, a free Dremio Cloud environment at dremio.com/get-started connects to object storage from day one and supports Parquet natively, whether your files are standalone or managed as Apache Iceberg tables.