Managing and Scaling Executors on Dremio K8s has Never Been Easier

Simplifying Kubernetes Management for the Modern Data Team

Enterprise customers leveraging Dremio on Kubernetes (K8s) have long valued the ability to scale engines according to their performance and cost requirements. However, until now, this capability required specialized Kubernetes expertise and manual configuration processes that added complexity to your data infrastructure management.

Today, we're excited to announce Enhanced K8s Engines—a significant advancement that brings cloud-like simplicity to your on-premises or cloud-based Kubernetes deployments. This innovation represents the first major step in our commitment to deliver a true SaaS-like experience across all Dremio deployment options.

Key Capabilities in Our First Release

UI and REST API-Driven Engine Management

Create, configure, and control your engines directly through Dremio's intuitive user interface or programmatically via our comprehensive REST API. This eliminates the need to interact with Kubernetes configuration files, significantly reducing the technical barrier to engine management.

Zero-Scale Efficiency with Automatic Activation

Engines can now scale to zero when not in use, automatically reactivating when queries arrive. This intelligent scaling capability helps you:

• Optimize costs by eliminating idle compute resources
• Meet demand instantly without administrative intervention
• Simplify operations by removing the need for manual Kubernetes scaling commands

Coming Soon: Advanced Scaling and Governance

Development is already well underway for our second iteration of Enhanced K8s Engines, which will introduce:

Independent Replica Scaling

Similar to our SaaS offering, each engine will support multiple child replicas that run queries independently, enabling:

• Increased query throughput without compromising isolation
• Workload-specific scaling to accommodate diverse business requirements
• Enhanced stability through proper resource allocation

Fine-Grained RBAC Controls

Administrators will soon be able to delegate engine management responsibilities within controlled parameters:

• Empower business units to create and scale engines autonomously
• Establish governance guardrails to maintain cost control
• Reduce administrative bottlenecks while preserving oversight

The Path Forward: True Autonomous Performance

By automating engine management and scaling, we're helping your team shift focus from infrastructure maintenance to data innovation.

Combined with our new Autonomous Reflections and Iceberg Clustering capabilities, Enhanced K8s Engines represent another step toward our vision of a self-optimizing, intelligent data lakehouse platform.

Experience the Difference

To learn more about these exciting capabilities and our broader product vision, register for our Spring 2025 Product Release Virtual Event on April 29th.

Ready to transform your Kubernetes experience? Try Dremio for free today or contact our team to schedule a personalized demo.