Dremio Corp., the developer of a data lakehouse platform based on the Apache Iceberg table format, today is throwing its support behind the Polaris data catalog that cloud data warehousing firm Snowflake Inc. released to open source in June.
The move is an important third-party endorsement for Snowflake, battling rival Databricks Inc. to become the platform of choice for artificial intelligence development on top of the open-source Apache Iceberg file system.
Dremio said it would also support Databricks’ rival Unity Catalog managed service, but its endorsement was less full-throated than for Snowflake. Over time, Dremio said it intends to adopt Polaris as its primary catalog and fold in advanced features that won’t be available to Unity Catalog users.
Dremio also said its data catalog for Iceberg is now available for on-premises, cloud and hybrid cloud deployment. Previously, the software had only been provided as a managed service. Dremio claims to be the only lakehouse provider to support all three deployment options. A data lakehouse is a data storage architecture that combines the best features of flexible data lakes and high-performance data warehouses.
Data catalogs, which are organized inventories of data assets within an organization that help users discover, understand and access data efficiently, have been the source of a pitched battle between Snowflake and Databricks. Both claim to be more committed to supporting the popular open-source Iceberg format than the other and both say their catalogs will eventually be freely available as open source.
Databricks partially released its Unity Catalog to open-source in June and says it intends to fully donate the platform to the community over time. Polaris is fully open-sourced under an Apache 2.0 license. Both companies offer managed versions of the open projects.
Read the full story, via SiliconANGLE.
