Apache Iceberg v4: An Efficiency Rewrite of the Table Format

Iceberg 1.11.0, released in May 2026, contains the full implementation of the v3 spec(ification), including features like deletion vectors, the VARIANT type, and row lineage. But what comes next?

V4, obviously, as four comes after 3. However, it's still early days on the v4 spec, so we are a ways off from the next version release. What the project community is focused on now is designing and agreeing on proposals for what gets added next. And if you read enough of these proposals a pattern emerges. Almost every one of them, whether ratified, in design, or still argued over on the dev mailing list, exists to remove waste somewhere in the format.

Metadata that gets rewritten when it did not need to change. Statistics that get deserialised when only a fraction were needed. Entire rows rewritten to update one column. The work on v4 is a coordinated effort to make Iceberg do less unnecessary work per operation, and that throughline is the clearest way to understand where the format is headed this year.

Making Commits Cost What They Should

The biggest source of waste in Iceberg today is commit amplification. Every commit, regardless of size, writes a new metadata JSON, a new manifest list, and at least one new manifest. If you add a single small file during a streaming job committing every few seconds, you still pay for the full cascade of file rewrites. Even though you are making very small data changes, this translates to a metadata overhead of multiple file writes and resulting compaction jobs to keep all these files tight and performant.

The proposal at the centre of v4, single-file commits, attacks this directly. It introduces a Root Manifest that replaces the manifest list and can inline small changes directly into itself, so a small commit writes one file instead of four. Background maintenance is still a thing, where here it rebalances those inlined entries down into leaf manifests, restoring the familiar layered structure that keeps planning fast on large tables. The tree's shape follows the workload: a streaming table keeps recent writes near the root for cheap commits, while a batch table settles into the classic deep structure.

Two smaller proposals are chasing this same goal: that commit cost should scale with the size of the change, not the size of the table.

• Snapshot offloading moves the table's full snapshot history out of the metadata JSON, so the file rewritten on every commit does not balloon as history accumulates.
• Delta-encoded schemas store schema changes as diffs rather than appending the full schema copy on every evolution.

Data Writes Are Also Getting an Update

The proposal generating the most raw message volume on the list this year is efficient column updates, and it brings write efficiency logic to its most demanding case: AI.

AI workloads frequently involve feature tables and embedding tables that can run hundreds of columns wide. However, with the current Iceberg spec, refreshing a handful of columns means rewriting entire rows, which means rewriting every untouched column alongside the ones that actually changed. The proposed fix writes only the changed columns to new files and stitches them together with the base file at read time.

Making Metadata Cheaper to Read

Commit efficiency is only half the picture. The other half is what happens when an engine has to read metadata back to plan a query, and this is where Content Stats, ratified into the v4 spec in May 2026, comes into play.

Today, per-file column statistics live in generic maps serialised as binary blobs inside row-oriented Avro manifests. Planning a query on a wide table means deserialising all of it, even when the query only touches a few columns. Content Stats replaces the maps with a typed, structured, columnar representation, so statistics become projectable field by field, just like data columns are. With this, an engine can read bounds for just the columns it needs and skips the rest. The same structure also makes room for statistics the old maps could never hold cleanly, like bounds on nested variant fields or bounding boxes for geometry.

A compact bitmap format, refining the structures behind deletion vectors (added in v3), was voted into the repository the following month on the same principle: when reading at scale, smaller, purpose-built encodings beat generic ones.

Relative paths, the other ratified v4 feature, targets operational inefficiencies. Every file reference in Iceberg today is an absolute URI with the bucket and region baked in, so relocating a table for disaster recovery or region migration means rewriting every path in the metadata tree. V4 stores paths relative to a table root instead, so moving the table means updating the catalog's root pointer and nothing else. On the surface this is a relatively basic change, but it's a massive quality of life improvement that quietly removes a whole class of operational pain.

What Iceberg Users Should Do Now

Dremio runs on v3 today, which already delivers a version of this same efficiency principle. Deletion vectors avoid rewriting entire files to mark a handful of rows deleted. The VARIANT type avoids the parsing cost of treating semi-structured data as opaque JSON strings. Row lineage gives rows persistent identity without recomputing it downstream.

Several v4 proposals are direct extensions of these v3 foundations, so adopting v3 now is how teams build the operational familiarity v4 will assume as a starting point.

To see how Dremio's v3 capabilities work in practice, a free environment at dremio.com/get-started connects to your object storage from day one.