The Dremio Blog

This month, we’re excited to announce automated table cleanup!

We are thrilled to announce the release of an enhanced vectorized Parquet Reader in Dremio software version 24.3 and Dremio Cloud. This Dremio-exclusive reader improves query performance up to 75% for Parquet datasets encoded with the Parquet V2 encodings.  Apache Parquet-MR Writer version PARQUET_2_0, which is widely adopted by engines that write Parquet data, supports […]

The concept of data marts has long been a pivotal strategy for organizations seeking to provide specialized access to critical data for their business units. Traditionally, data marts were seen as satellite databases, each carved out from the central data warehouse and designed to serve specific departments or teams. These data marts played a vital […]

Unveiling Dremio Cloud on Azure, a Fast, Scalable and Secure Lakehouse Platform Introduction Today we introduce Dremio Cloud on Azure, newly landing on Azure in November 2023, in public preview. Dremio Cloud is a powerful lakehouse platform providing streamlined self-service, unparalleled SQL performance, centralized data governance and seamless lakehouse management. In this blog post, we […]

The Dremio Unified Analytics Platform brings your users closer to the data with lakehouse flexibility, scalability and performance at a fraction of the cost.  We're excited to show you some new things we've been working on to help you analyze your data faster and easier than ever, no matter where it's stored. We’ve launched AI-powered […]

Introduction Data types in Dremio fall into two categories: primitive types such as INT and VARCHAR that hold single values, and semi-structured types like LIST, STRUCT, and MAP that hold complex values. Arrays are lists of arbitrary size of any single type, indexed by non-negative integers, and are useful for holding sparse data. Note: LIST […]

The Databricks platform is widely used for extract, transform, and load (ETL), machine learning, and data science. When using Databricks, it's essential to save your data in a format compatible with the Databricks File System (DBFS). This ensures that either the Databricks Spark or Databricks Photon engines can access it. Delta Lake is designed for […]

This blog helps you learn about table-driven access policies in Dremio Cloud and Dremio Software v24.1+.

Introduction User-defined functions (UDFs) are callable routines that make it easier for you to write and reuse SQL logic across queries. In addition, UDFs let you extend the capabilities of Dremio SQL, provide a layer of abstraction to simplify query construction, and encapsulate business logic. UDFs can also serve as row and column policies for […]

We're always looking for ways to better handle and save money on our data. That's why the "data lakehouse" is becoming so popular. It offers a mix of the flexibility of data lakes and the ease of use and performance of data warehouses. The goal? Make data handling easier and cheaper. So, how do we […]

In the age of data-centric applications, storing, accessing, and managing data can significantly influence an organization's ability to derive value from a data lakehouse. At the heart of this conversation are data lakehouse table formats, which are metadata layers that allow tools to interact with data lake storage like a traditional database. But why do […]

Using Airbyte, you can easily ingest data from countless possible data sources into your S3-based data lake, and then directly into Apache Iceberg format. The Apache Iceberg tables are easily readable directly from your data lake using the Dremio data lakehouse platform, which allows for self-service data curation and delivery of that data for ad hoc analytics, BI dashboards, analytics applications, and more.

In the last decade, a lot of innovation in the data space took place to address the need for more storage, more compute, hybrid environments, data management automation, self-service analytics, and lower TCO to operate business at scale.

Introduction Data types in Dremio fall into two categories: primitive types such as INT and VARCHAR that hold single values, and semi-structured types like LIST, STRUCT, and MAP that hold complex values. Arrays are lists of arbitrary size of any single type, indexed by non-negative integers, and are useful for holding sparse data. Note: LIST […]

Dremio Reflection Recommender We are delighted to announce the release of the new Dremio Reflection Recommender. The Reflection Recommender eliminates guesswork by facilitating the creation of Reflections that will accelerate the input query workload. This capability is available now in Dremio Cloud and 24.2 release of Dremio software. Accelerate BI workloads with Reflections in seconds […]