Introducing Service Users: Secure Machine-to-Machine Authentication for Dremio

We're excited to announce the introduction of Service Users in Dremio—a new authentication approach designed specifically for the agentic AI era, supporting machine-to-machine (M2M) applications and automated systems. Service users provide a more secure and streamlined way to integrate AI agents,  applications, scripts, and CI/CD pipelines with your Dremio environment.

What Are Service Users?

Service users are non-human accounts built for programmatic access to Dremio. Unlike regular user accounts that are designed for interactive use through the Dremio Console, service users are purpose-built for automated systems that need to authenticate and access Dremio resources programmatically.

Key characteristics of service users:

• API-first design: Built specifically for REST API, Arrow Flight, Arrow Flight JDBC/ODBC, and other programmatic interfaces
• No Console access: Cannot log into the Dremio web interface, reducing attack surface
• OAuth-based authentication: Uses industry-standard OAuth 2.0 client credentials flow
• External integration ready: Supports authentication via Microsoft Entra ID and other external OAuth providers

Why Service Users Are More Secure

Traditional approaches to M2M authentication often involve using regular user accounts with personal access tokens (PATs) or shared credentials. Service users address several security concerns with these approaches:

OAuth Client Credentials Flow

Service users authenticate using OAuth 2.0 client credentials flow, which provides several security advantages:

• Short-lived access tokens: Tokens expire automatically, reducing the window of exposure if compromised
• Centralized token management: Built-in token refresh and expiration handling
• Industry standard: Uses widely adopted OAuth 2.0 protocols with established security patterns
• Audit trails: Better visibility into when and how tokens are used

Reduced Attack Surface

By design, service users:

• Cannot access the UI: No risk of compromised service accounts being used for interactive sessions
• Limited scope: Can be granted only the specific roles and permissions needed for their function
• Purpose-built: Clearly identifiable as non-human accounts in logs and audit trails

Comprehensive Access Control Integration

Service users seamlessly integrate with Dremio's complete access control framework:

Role-Based Access Control (RBAC): Service users can be assigned roles and belong to groups just like regular users. Since each service user is typically dedicated to a single application or process, implementing the principle of least privilege becomes straightforward—grant only the minimum permissions needed for that specific use case.

Fine-Grained Access Control: Service users benefit from Dremio's advanced data governance capabilities, including:

• Row-level filtering: Automatically filter datasets based on the service user's assigned policies
• Column masking: Hide or obfuscate sensitive columns from service users that don't need access
• Dynamic data policies: Apply context-aware access controls based on the requesting service user

External Service Principal Integration

For organizations using Microsoft Entra ID or other identity providers, service users can authenticate using external service principals, enabling:

• Centralized identity management: Leverage existing identity infrastructure
• Enterprise security policies: Apply conditional access and security policies consistently
• Single sign-on integration: Use your organization's existing OAuth configuration

Perfect for Modern Application Architectures

Service users are ideal for common integration scenarios:

CI/CD Pipelines: Automate data pipeline deployments and testing without storing long-lived credentials in your build systems.

Application Integration: Connect applications to Dremio using secure, short-lived tokens that refresh automatically. Each service user can be granted precisely the data access permissions required by that specific application.

Data Integration Tools: Use service users with ETL tools, data orchestrators, and custom applications. Apply row-level security and column masking policies to ensure each integration only accesses the data it needs.

Monitoring and Analytics: Integrate observability tools and custom dashboards that query Dremio metrics and metadata. Service users can be restricted to read-only access with filtered views of operational data.

Enterprise Catalog Integration: Build applications that leverage Dremio's Enterprise Catalog based on Apache Polaris, a fully open-source Iceberg catalog. Service users can utilize full Iceberg support for dataset interoperability with other query engines and catalogs. Through Dremio's connectors to other Iceberg REST Catalogs such as Snowflake Open Catalog or Unity Catalog, service users enable the development of truly enterprise-wide M2M applications that span multiple data platforms securely.

Getting Started

Creating and using service users is straightforward:

1. Create a service user in the Dremio Console under Settings > User Management > Service Users
2. Configure OAuth credentials or external service principal authentication
3. Assign appropriate roles and groups based on the principle of least privilege
4. Configure fine-grained access policies (row filtering, column masking) as needed
5. Integrate with your applications using the generated client credentials

The OAuth client credentials flow handles token management automatically, making integration simple while maintaining security best practices across all connection types—whether you're using REST APIs for metadata operations, Arrow Flight for high-performance data queries, or Arrow Flight JDBC/ODBC drivers for application connectivity.

Enhanced Security Posture

Service users represent a significant step forward in Dremio's security capabilities. By providing purpose-built accounts for M2M authentication that integrate seamlessly with Dremio's complete access control framework, organizations can:

• Eliminate shared credentials and long-lived tokens in automated systems
• Implement true least privilege access with dedicated service users for each application
• Apply enterprise data governance including row filtering and column masking to M2M integrations across multiple catalog systems
• Enable cross-platform interoperability through Apache Polaris and Iceberg REST Catalog connections
• Build truly enterprise-wide applications that securely span Dremio and external data platforms
• Improve audit capabilities with clear identification of service user activity
• Reduce blast radius of compromised credentials through scoped permissions and UI restrictions
• Align with security frameworks by using OAuth standards and external identity integration

Availability

Service users are available now in Dremio.  Whether you're running Dremio on-premises or using our Next Generation Dremio Cloud platform, you'll be able to start securing your M2M integrations by creating your first service user in the Dremio Console.

Ready to enhance your Dremio security posture? Check out our Service User documentation to get started, or explore our OAuth authentication examples for common integration patterns.