The Dremio Blog

We’ve explored how Large Language Models (LLMs) work, and how we can improve their performance with fine-tuning, prompt engineering, and retrieval-augmented generation (RAG). These enhancements are powerful — but they’re still fundamentally stateless and reactive.

Clustering is a data layout strategy that organizes rows based on the values of one or more columns, without physically splitting the dataset into separate partitions. Instead of creating distinct directory structures, like traditional partitioning does, clustering sorts and groups related rows together within the existing storage layout.

this post, we’ll walk through the three most popular and practical ways to boost the performance of Large Language Models (LLMs): Fine-tuning Prompt engineering Retrieval-Augmented Generation (RAG) Each approach has its strengths, trade-offs, and ideal use cases. By the end, you’ll know when to use each — and how they work under the hood.

Enterprises today face increasing pressure to extract insights from data quickly while controlling spend. Yet, as data volumes explode across cloud and on-prem environments, traditional architectures often fall short—resulting in higher costs, rigid pipelines, and slower decision-making. The Dremio Intelligent Lakehouse Platform addresses these challenges by delivering faster insights and significant total cost of ownership […]

In this post, we’ll peel back the curtain on the inner workings of LLMs. We’ll explore the fundamental concepts that make these models tick: embeddings, vector spaces, and context windows. You’ll walk away with a clearer understanding of how LLMs “understand” language — and what their limits are.

Artificial Intelligence (AI) has become essential for modern enterprises, driving innovation across industries by transforming data into actionable insights. However, AI's success depends heavily on having consistent, high-quality data readily available for experimentation and model development. It is estimated that data scientists spend 80+% of their time on data acquisition and preparation, compared to model […]

This post kicks off our 10-part series exploring how AI evolved into LLMs, how to enhance their capabilities, and how the Model Context Protocol (MCP) is shaping the future of intelligent, modular agents.

Why SQL Must Evolve in the Era of Agentic Apps and Data-Aware AI SQL has long been the universal language of data. But with the rise of Generative AI and agentic applications, a major shift is underway. We're entering an era where natural language is the interface, and agents are the client. There are two […]

By leveraging the dbt-dremio adaptor, Analytics Engineers can seamlessly sync model descriptions and tags from dbt projects with Dremio to generate wikis and labels for Business Users and Data Analysts.

Integrating row column access control with materializations enables Dremio Reflections to deliver high-performance query execution without compromising on security or flexibility, making it an ideal solution for scalable, secure data access in the lakehouse architecture. Furthermore, by enabling pre-compute materializations to be re-usable across users and roles, significant cost savings can be achieved through more efficient engine resource utilization.

At Dremio, we implemented Autonomous Reflections in our own internal Data Lakehouse. We are happy to report that Autonomous Reflections exceeded our expectations. In just days, we saw significant improvements

Credential vending support in Dremio opens up a more secure and convenient way to query external Iceberg catalogs. By obtaining temporary, table-scoped credentials on the fly, Dremio minimizes long-lived secrets and ensures access is tightly controlled by the catalog’s policies.

Today, we're excited to announce the general availability of Dremio's latest release, delivering accelerated AI and analytics through intelligent automation. Marking the next generation of Dremio, this release represents a significant milestone in our mission to eliminate technical complexity and resource waste through autonomous capabilities, empowering teams to innovate rather than maintain. In today's economic […]

Is Query Performance Slowing Down Your AI and Analytics Initiatives? Slow analytics and AI workloads frustrate users and delay critical insights, draining productivity. If waiting for queries to load feels like the norm, you're not alone. But what if query performance could be accelerated—automatically, without requiring any specialized expertise or manual intervention? Enter Autonomous Reflections. […]

Companies of all sizes now use lakehouse architectures to power their analytics and AI workloads. Lakehouses give companies a single, trusted source of data for analytics and AI tools to access, and eliminate the need for data duplication and vendor lock-in. The catalog, or metastore, is an integral part of the lakehouse that enables tools […]