The Version 25 release underscores Dremio's commitment to providing an SQL query engine that excels in performance and sets new benchmarks in stability and durability.
Dremio's bold assertion that it offers market-leading stability and durability isn't just a claim—it's a commitment backed by substantial advancements in technology. The platform is designed to handle highly complex workloads with unmatched stability, ensuring that no query fails, regardless of the workload's complexity or size.
Explore this interactive demo and see how Dremio's Intelligent Lakehouse enables Agentic AI
At the core of Dremio's new release is improved query durability. This feature guarantees consistent and reliable performance across large and complex workloads, living up to the vision that no query will fail. Such robustness is crucial for businesses that depend on timely and accurate data analytics for their decision-making processes.
Dremio version 25 introduces several key features aimed at bolstering the engine's durability and performance, particularly for substantial and intricate workloads:
One of the standout enhancements is the improved memory management capabilities. This advancement eliminates the risk of out-of-memory issues, a common challenge in data analytics, ensuring uninterrupted query execution. Dremio's memory management system meticulously monitors the dynamic memory usage of high-risk operators, allocating memory as necessary to prevent failures.
When a complex query or process is about to exhaust available memory, Dremio smartly diverts the process to disk, reducing memory consumption and guaranteeing continuity. This optimizes memory usage and enhances performance, scalability, and stability.
Introducing spillable hash joins marks a significant upgrade in Dremio's join processes. When memory capacity is surpassed, the system temporarily shifts data to disk, allowing the join operation to continue without hiccups. This approach significantly boosts scalability, enabling Dremio to process datasets larger than the available memory, thus facilitating seamless queries on vast data volumes.
The new memory arbiter feature further fortifies Dremio's defense against out-of-memory errors. It empowers executors to closely monitor and allocate memory in real time, enhancing the system's responsiveness and reliability during memory-intensive operations.
Dremio's version 25 is not just an incremental update; it's a transformative release that redefines stability and durability in SQL analytics. By introducing sophisticated memory management, spillable hash joins, and a proactive memory arbiter, Dremio ensures businesses can rely on its engine for their most critical and complex data workloads. These enhancements reinforce Dremio's position as a leader in the data analytics space, offering an engine that is powerful but also resilient and dependable, catering to the evolving needs of modern enterprises.
Deploy agentic analytics directly on Apache Iceberg data with no pipelines and no added overhead.
By unifying data from diverse sources, simplifying data operations, and providing powerful tools for data management, Dremio stands out as a comprehensive solution for modern data needs. Whether you are a data engineer, business analyst, or data scientist, harnessing the combined power of Dremio and Apache Iceberg will undoubtedly be a valuable asset in your data management toolkit.
This blog helps you learn about table-driven access policies in Dremio Cloud and Dremio Software v24.1+.
Dremio Arctic bring new features to Dremio Cloud, including Apache Iceberg table optimization and Data as Code.
