The Dremio Blog

Companies are racing to operationalize agentic AI, yet the final process of getting from data to decision is extremely difficult,  requiring data integration, tuning, and governance management. With Dremio’s latest release, we remove these blockers by putting natural‑language intelligence, explainability, and self‑optimizing performance directly into the lakehouse experience. You get clarity and control without copies, […]

Dremio has introduced several capabilities that inteliigently improve query performance across the data lakehouse. With minimal to no action from users, Dremio will reduce query latency, handle data maintenance tasks, and eliminate redundant compute jobs. This article is a summary of three of these performance management features. Read on to learn how reflections accelerate popular […]

The Intelligent Lakehouse for the Agentic AI Era Data teams today face a familiar challenge: how to unlock value from ever-growing, scattered data without the delays and cost of traditional ETL pipelines. Together, HPE Alletra Storage MP X10000 and Dremio’s Intelligent Lakehouse Platform solve this problem—combining HPE’s flash-optimized performance with Dremio’s open, unified query and […]

Dremio is always striving to abstract away the physical concerns of data, whether the storage location, partitioning schema, or file size optimisation. Thanks to features such as Data Federation, Iceberg Clustering, and Autonomous Performance functionalities, Dremio users get highly-performant access to their data no matter where it lives.  One of the components that delivers this […]

The New Economics of Data Cloud data warehouses like Amazon Redshift were built for a world that no longer exists. In that earlier era, organizations focused primarily on structured business intelligence, static dashboards, and predictable workloads. Data was tightly controlled, compute resources were fixed, and dynamic scalability for rapidly changing workloads was not a concern. […]

Caching dramatically reduces latency and computational costs by storing frequently accessed data closer to where it's needed. Instead of repeated expensive operations - such as fetching from object storage, planning complex queries, or executing SQL - the data you need is provided in fast, local memory. To deliver on this, Dremio implements different layers of […]

The Shift from Warehouses to the Agentic Lakehouse Amazon Redshift has long been a dependable data warehouse for analytics, but the analytics landscape has evolved. Organizations are no longer just running dashboards—they’re powering agentic AI systems that reason, act, and make autonomous decisions based on live business data. These workloads demand real-time responses, high concurrency, […]

This only scratches the surface of Dremio's content. Explore Dremio University and the Dremio Blog to find much more great Dremio Content. Also get involved in the Dremio and OSS community at developer.dremio.com.

Overview Dremio provides out-of-the-box methods of handling complex data types in, for example JSON and parquet datasets. Common characteristics are embedded “columns within columns” and “rows within columns”.  In this blog, we will demonstrate how Dremio can discover and handle these types of data. The examples have been tested on the following Dremio versions: Preparation […]

Agentic AI is an artificial intelligence system that is designed to operate autonomously. With minimal human supervision it can be expected to make decisions and perform tasks with specifically trained agents. This is thanks in large part to Large Language Models (LLMs) which provide agentic AI with enhanced reasoning and the ability to understand context. […]

As education becomes increasingly data-driven, the stakes for protecting sensitive information have never been higher. Regulations like FERPA, COPPA, GDPR, and state-level privacy laws demand rigorous compliance, while rising cyber threats highlight the urgent need for robust security and governance. At the same time, educators and edtech companies cannot afford to sacrifice innovation, students expect personalized learning, administrators need real-time insights, and institutions are exploring AI-driven opportunities to improve outcomes.

Every year, the Gartner® Hype Cycle™ for Data Management helps us understand which technologies are generating buzz and which are delivering real business impact. In the 2024 report, one placement caught my attention: the data lakehouse has shifted from the Peak of Inflated Expectations into the Trough of Disillusionment. At first glance, this might sound […]

By standardizing the interaction between hosts, clients, and servers, MCP unlocks true modularity. You can swap models without breaking workflows, mix and match servers for analytics, email, or storage, and grow your AI capabilities incrementally. The Dremio + SendGrid example shows how easily analytics and action can come together, transforming what used to be manual, multi-step processes into fully automated workflows.

Introduction Dremio can automatically take advantage of partitioning on parquet data sets (or derivatives such as Iceberg or Delta Lake). By understanding the dataset’s partitioning, Dremio can perform partition pruning, the process of excluding irrelevant partitions of data during the query optimisation phase, to boost query performance. (See Data Partition Pruning). Partition bucketing provides a […]

What makes Polaris especially exciting is the trajectory it’s on. Today, it is a powerful, open catalog for Iceberg tables. Tomorrow, it could serve as the central control plane for managing a full range of lakehouse assets, unifying governance, access, and interoperability across an increasingly complex data ecosystem.