This post covers the Apache Iceberg 0.13.0 release from February 2022 and is kept as a record of that release. For the current state of the project, see the Apache Iceberg release history.
On Wednesday, February 9th, the Apache Iceberg community released 0.13.0 with lots of new features and improvements. Also released was a new website and docs design.
2022 has already been so far a great year for Apache Iceberg seeing great coverage such as new levels of support from platforms like Dremio, AWS and Snowflake, and being chosen as the “Open Source Project of the Week” by Software Development Times for the first week of February 2022.
Let’s take a deeper look at many of the new features that come with Apache Iceberg 0.13.0. (Release Notes)
Catalog caching is a technique that can help speed up table reads by allowing engines to not have to read the table's metadata on every read.
Sometimes when multiple readers and writers are using Apache Iceberg tables from different engines the result would be that the cache would not refresh so a manual refresh was required to see the new data. With the new cache expiration feature a time interval can be set for the cache to expire, forcing an automatic refresh to resolve this issue. This can be configured using the setting cache.expiration-interval-ms which will be ignored if cache-enabled is set to false. Read more on this feature here.
When using Iceberg on object stores like S3 and committing to tables from multiple engines concurrently, you previously couldn’t use the Hadoop catalog safely. This is because the check Iceberg relied on to ensure writing via two concurrent separate jobs or engines isn’t atomic, so two concurrent commits could result in data loss.
Iceberg 0.13.0 fixes this issue by leveraging a lock table in services like DynamoDB so all catalogs can use locks for safe concurrency. Read more on this feature here.
From a HiveCatalog you could drop a table but there was no way to add an existing table to the catalog. With this update you can now add existing tables to your catalog by passing the location of the newest metadata file of the external table. This is especially helpful for interacting with Hive external tables in Spark. Read more on this feature here.
In Iceberg’s v2 format, delete files are used to track records that have been deleted. Previously, this wasn’t supported when the table’s underlying file format was ORC. Now, both position and equality deletes are supported for tables backed by ORC files. Read more here.
Along with the core updates detailed previously several vendor integrations were added in version 0.13.0 including.
Several updates to the support available for many of the most popular data processing tools.
RewriteDataFiles now supports table-based optimization, merge-on-read delete. [#2829]
Apache Iceberg in 2022 is adding the features data engineers want and need which is clear given the attention and momentum it has recently received in just a little over a month into the year.
If not currently using Apache Iceberg for your data lake, give it a test run by creating some Iceberg tables using AWS Glue.
For a full treatment of the table format, download Apache Iceberg: The Definitive Guide by Tomer Shiran, Jason Hughes, and Alex Merced, free from Dremio.
To work through Iceberg hands on, explore the free courses at Dremio University.
Deploy agentic analytics directly on Apache Iceberg data with no pipelines and no added overhead.
By unifying data from diverse sources, simplifying data operations, and providing powerful tools for data management, Dremio stands out as a comprehensive solution for modern data needs. Whether you are a data engineer, business analyst, or data scientist, harnessing the combined power of Dremio and Apache Iceberg will undoubtedly be a valuable asset in your data management toolkit.
Editor’s note, September 2026. This post was published in September 2023 and some product details have changed since. Nessie is still available and self-deployable under the Apache-2.0 licence, and it remains the clearest implementation of catalog-level branching. It is not an Apache Software Foundation project, and its development has slowed considerably. For how the current […]
This blog helps you learn about table-driven access policies in Dremio Cloud and Dremio Software v24.1+.
