The Dremio Blog

A crucial component of an Iceberg lakehouse is the catalog, which tracks your tables, making them discoverable by various tools like Dremio, Snowflake, Apache Spark, and more. Recently, a new community-driven open-source catalog named Polaris has emerged at the forefront of open-source Iceberg catalog discussions.

GET A FREE COPY OF "Apache Iceberg: The Definitive Guide" ENROLL IN THE "Apache Iceberg Crash Course" The data lakehouse concept has emerged as a revolutionary solution, blending the best of data lakes and data warehouses. As organizations strive to harness the full potential of their data, choosing the right data lakehouse solution becomes crucial. […]

Connecting the Dots and Data Sources for Analysts Data source connections significantly impact the efficiency of your analytics workflows. Whether you're performing complex statistical analyses, building predictive models, or creating dashboards, your connection type influences your speed to insight. There are four main connection types: JDBC, ODBC, REST, and Arrow Flight. Understanding each helps you […]

While data lakes democratized data access, they also introduced challenges that hindered their usability compared to traditional systems. The advent of table formats like Apache Iceberg and catalogs like Nessie and Polaris has bridged this gap, enabling the data lakehouse architecture to combine the best of both worlds.

Part 1: The Challenge for Organizations Organizations must enable data users to leverage and gain insights from their data seamlessly. The goal is to drive business value through comprehensive data analysis, regardless of where the data resides: on-premises, in the cloud, or hybrid cloud environments. While there is a significant push towards cloud adoption, many […]

Over the years, cybersecurity capabilities have evolved from single-point solutions to comprehensive cyber data platforms utilizing advanced analytic-based technologies. With the exponential growth in the volume, variety, and complexity of cyber-relevant data, cybersecurity professionals must leverage cutting-edge data platform technologies to address their needs effectively and economically. In today’s digital age, virtually all data holds […]

Dremio's built-in lakehouse catalog, powered by Project Nessie, makes data engineering workflows easy by enabling a Git-like experience on Iceberg tables and views. Companies can use branches to make and validate changes to data without disrupting production workloads, instantly merge changes into their production branch when ready, and easily roll back from mistakes if needed. […]

Two prominent solutions have emerged in the on-prem, cloud, and hybrid-cloud lakehouse space: Dremio and Starburst Data. Both platforms offer unique features and benefits. On the surface, the platforms look fairly similar, with federated query capability, object store connectivity, SQL on Hadoop functionality, Iceberg support, and support for hybrid cloud environments. A deeper dive reveals […]

A data flow design for modern data analytics. The medallion architecture empowers data analysts to access trusted data, collaborate with colleagues, and uncover invaluable insights quickly and efficiently. Analysts can unlock the full potential of their organization's data and drive informed decision-making by understanding the distinct layers of the data lakehouse and its role in […]

The demand for flexible and fast data-driven decision-making is critical for modern business strategy. Semantic layers are designed to bridge the gap between complex data structures and business-friendly terminology, enabling self-service analytics. However, traditional approaches often struggle to meet performance and flexibility demands for today’s business insights. This is where a data lakehouse-powered semantic layer […]

Data Mesh, Data Lakehouse, Data Fabric, Data Virtualization—there are many buzzwords describing ways to build your data platform. Regardless of the terminology, everyone seeks the same core features in their data platform: Many of these "Data X" concepts address different aspects of these goals. However, when you integrate solutions that cover all these needs, you […]

Data Mesh, Data Lakehouse, Data Fabric, Data Virtualization—there are many buzzwords describing ways to build your data platform. Regardless of the terminology, everyone seeks the same core features in their data platform: Many of these "Data X" concepts address different aspects of these goals. However, when you integrate solutions that cover all these needs, you […]

Data sharing is becoming increasingly important in the data world. Not all the data we need can be generated in-house, and our data can also be a valuable asset for generating revenue or building strategic partnerships. Leveraging tools that enable data sharing can significantly enhance the value of your data. In this blog, we aim […]

We are thrilled to announce the Public Preview of AutoIngest Pipes, a new way to load data into Iceberg tables designed to simplify the development and management of your data loading pipelines. In today's data driven world, the need for efficient, reliable, and scalable data ingestion has never been more critical. Auto Ingest Pipes is […]

The Unified Apache Iceberg Lakehouse, powered by Dremio, offers a compelling solution for unified analytics. By connecting to a wide range of data sources and minimizing data movement, you can achieve faster, more efficient analytics, improve AI model training, and enhance data enrichment processes. Dremio's advanced processing capabilities and performance features make it a standout choice for any organization looking to unify and accelerate their data analytics platform.