The Dremio Blog

A truly modern data lakehouse is defined by more than just its use of open formats. Its value is measured by its intelligence and completeness. It is a platform where AI can take action, where openness extends to other compute engines, where the semantic layer is a structured and programmable asset, and where performance management is autonomous. These integrated capabilities are shifting the baseline for what organizations should expect from a data platform. The focus is no longer just on storing and querying data, but on creating a self-managing, intelligent, and truly unified ecosystem. As you evaluate your data strategy, ask yourself a forward-looking question: As AI becomes a more active participant in our data ecosystems, how will you leverage it to not just analyze your business, but to help run it?

Dremio's AI Functions are more than just a new feature; they are a bridge over the long-standing divide between unstructured data and SQL-based analytics. By embedding LLM capabilities directly into the query engine, Dremio provides a complete workflow to unlock your most inaccessible data. You can now analyze data where it lives, choose the best AI model for the job, manage workloads with enterprise-grade controls, and transform raw files into governed, high-performance data products. The era of inaccessible information is over.

The data lakehouse is evolving beyond just being a repository for data. With Dremio, it's becoming an autonomous, open, and intelligent platform that actively works to simplify your architecture, accelerate your queries, and expand the very definition of what data can be analyzed.

We're excited to announce the introduction of Service Users in Dremio—a new authentication approach designed specifically for the agentic AI era, supporting machine-to-machine (M2M) applications and automated systems. Service users provide a more secure and streamlined way to integrate AI agents,  applications, scripts, and CI/CD pipelines with your Dremio environment. What Are Service Users? Service […]

We’re excited to share, in partnership with Microsoft, that Dremio is the first agentic lakehouse platform to fully support the open source Apache Arrow Database Connectivity (ADBC) driver for Power BI, bringing next-generation performance to your analytics. Whether you’re working with Dremio Cloud or Dremio Software, this enhancement is available across Power BI Desktop, Power […]

In this blog you will learn how to connect each tool to a REST catalog like Dremio Catalog, using bearer tokens and vended credentials to keep your pipelines secure and portable.

Dremio Next Gen Cloud represents a major leap forward in making the data lakehouse experience seamless, powerful, and accessible. Whether you're just beginning your lakehouse journey or modernizing a complex data environment, Dremio gives you the tools to work faster and smarter—with native Apache Iceberg support, AI-powered features, and a fully integrated catalog. From federated queries across diverse sources to autonomous performance tuning, Dremio abstracts away the operational headaches so you can focus on delivering insights. And with built-in AI capabilities, you're not just managing data—you’re unlocking its full potential.

We’re excited to announce Dremio Cloud, The Agentic Lakehouse—the lakehouse built for agents and managed by agents. This milestone marks a major leap forward in Dremio’s evolution, reimagining the modern lakehouse for the agentic era, where intelligent systems collaborate with humans to deliver insights, automate operations, and continuously optimize performance. As organizations accelerate their AI […]

Many data engineers and data analysts spend much of their day in Visual Studio (VS) Code, writing SQL, testing queries, and working with data. Constantly switching between tools disrupts productivity and the user work flow. The VS Code extension for Dremio brings the power of the agentic lakehouse directly into your development environment, enabling you […]

Organizations cannot implement AI quickly when data is fragmented across systems, ungoverned, and lacks the business context AI needs to deliver accurate results. Teams juggle schema knowledge, joins, query tuning, visualization tools, and governance checks before they can answer even a simple business question. With Dremio's Agentic Lakehouse—the only data platform built for agents and […]

The rapid growth of the use of AI throughout the modern data stack has transformed how organizations extract insights from their data. With our latest release, we're excited to announce the general availability of AI Functions — a capability that brings the power of Large Language Models (LLMs) directly into SQL execution, making Dremio’s Agentic […]

Discover how Dremio’s Next Generation Cloud and enterprise MCP Server simplify data exploration with AI-driven queries, governance, and natural-language SQL.

Performance management for Apache Iceberg tables isn’t just about cleaning up small files, it’s about ensuring your data layout evolves in step with your ingestion patterns and query workloads. Dremio’s OPTIMIZE command provides the precision engineers need: merging, splitting, and reclustering data into efficient layouts while keeping metadata lean. With its flexible parameters, you can tailor compaction jobs to strike the right balance between optimization depth, runtime, and cost. At the same time, Dremio’s auto-optimization features mean you don’t always have to run these jobs manually. By letting Dremio continuously monitor and optimize Iceberg tables in the background, your most critical datasets stay query-ready without the overhead of constant maintenance.

The real secret to minimizing Iceberg table maintenance isn’t running more optimization jobs, it’s writing smarter data from the very beginning. By combining batch and streaming ingestion best practices, designing thoughtful partitioning and clustering strategies, tuning table properties, and monitoring file health, you can dramatically reduce the frequency and cost of downstream operations like OPTIMIZE.

Apache Iceberg’s snapshot model is a game-changer for time travel, auditing, and recovery, but it comes with a responsibility: old data must be managed carefully. Without proactive cleanup, tables can accumulate unnecessary files, driving up storage costs, slowing queries, and even creating compliance risks. Dremio’s VACUUM TABLE command provides the control data engineers and architects need to: Expire outdated snapshots, keeping only the versions that align with retention policies. Permanently remove deleted data to meet GDPR and CCPA requirements. Clean up orphan files to ensure storage remains lean and predictable.