The Dremio Blog

By following the steps in this guide, you now have a fully functional Iceberg and Polaris environment running locally. You have seen how to spin up the services, initialize the catalog, configure Spark, and work with Iceberg tables. Most importantly, you have set up a pattern that closely mirrors what modern data platforms are doing in production today.

Seamless result cache for Iceberg was enabled for all Dremio Cloud organizations in May 2025.  Since then, our telemetry has told us between 10% to 50% of a single project’s queries have been accelerated by result cache.  That’s a huge cost saving on executors.  Looking forward, Dremio is doing research on how to bring its reflection matching query re-write capabilities to the result cache.  For example, once a user generates a result cache entry, it should be possible to trim, filter, sort and roll up from this result cache.  Limiting the search space and efficient matching through hashes will be key features to make matching on result cache possible.  Stay tuned for more!

Back in April of this year Dremio debuted its own MCP server, giving the LLM of your choice intelligent access to Dremio’s powerful lakehouse platform. With the Dremio MCP Server the LLM knows how to interact with Dremio; facilitating authentication, executing requests against the Dremio environment, and returning results to the LLM. The intention is […]

The Polaris benchmarking framework provides a robust mechanism to validate performance, scalability, and reliability of Polaris deployments. By simulating real-world workloads, it enables administrators to identify bottlenecks, verify configurations, and ensure compliance with service-level objectives (SLOs). The framework’s flexibility allows for the creation of arbitrarily complex datasets, making it an essential tool for both development and production environments.

Polaris is a next-generation metadata catalog, born from real-world needs, designed for interoperability, and open-sourced from day one. It’s built for the lakehouse era, and it’s rapidly gaining momentum as the new standard for how data is managed in open, multi-engine environments.

With Dremio, you’re not locked into a specific vendor’s ecosystem. You’re not waiting on data engineering teams to build yet another pipeline. You’re not struggling with inconsistent definitions across departments. Instead, you’re empowering your teams to move fast, explore freely, and build confidently, on a platform that was designed for interoperability from day one.

With traditional setups like Claude Desktop, that bridge is tightly coupled to a specific LLM. But with the Universal MCP Chat Client, you can swap out the brain, GPT, Claude, Gemini, Cohere, you name it, and still connect to the same tool ecosystem.

At Dremio, we have spent the last few years developing not only query execution improvements but also game-changing autonomous data optimization capabilities. Dremio is by far and away the fastest lakehouse.  The capabilities deliver 20x faster query performance compared to other platforms, without requiring any manual actions.

Companies today are under constant pressure to deliver faster insights, support advanced analytics, and enable AI-driven innovation. Many organizations chose  Amazon Redshift as their cloud data warehouse. However, as data volumes grow and workloads change, Redshift’s legacy warehouse architecture is not meeting their needs—driving many organizations to consider alternatives. Dremio’s intelligent lakehouse platform: a modern, […]

This guide will equip you with the expertise to easily build an AI-ready data product using Dremio and dbt.

By using properties, Puffin files, and REST catalog APIs wisely, you can build richer, more introspective data systems. Whether you're developing an internal data quality pipeline or a multi-tenant ML feature store, Iceberg offers clean integration points that let metadata travel with the data.

The traditional approach of caching table metadata and periodically refreshing has various drawbacks and limitations. With seamless metadata refresh, Dremio now provides users with an effortless experience to query the most up-to-date versions of their Iceberg tables without wrecking the performance of their queries. So now a user querying a shared table in Dremio Enterprise Catalog powered by Apache Polaris for example can see updates from an external Spark job immediately with no delay, and they never even have to think about it.

Confluent’s TableFlow and Apache Iceberg unlock a powerful synergy: the ability to stream data from Kafka into open, queryable tables with zero manual pipelines. With Dremio, you can instantly access and analyze this real-time data without having to move or copy it—accelerating insights, reducing ETL complexity, and embracing the power of open lakehouse architecture.

Incremental materializations allow you to build your data table piece by piece as new data comes in. By restricting your build operations to just this required data, you will not only greatly reduce the runtime of your data transformations, but also improve query performance and reduce compute costs.

That’s where Sampling comes in. And what if you want to give the user — or the LLM — reusable, structured prompt templates for common workflows? That’s where Prompts come in. In this final post of the series, we’ll explore: How sampling allows servers to request completions from LLMs How prompts enable reusable, guided AI interactions Best practices for both features Real-world use cases that combine everything we’ve covered so far