Simplifying Data Discovery with the Dremio Connector for Alation

Organizations need tools that seamlessly integrate, manage, and discover data across different platforms. That’s why we’re excited to introduce the Dremio Software OCF Connector, developed by Alation. This new connector makes it easy to catalog your Dremio assets within Alation, helping teams efficiently discover, govern, and collaborate around their data. 

What is the Dremio Connector for Alation? 

The Dremio Software OCF Connector allows Alation to automatically extract metadata from Dremio Software versions 24.3.0 and later. Once installed, the connector catalogs Dremio objects such as schemas, tables, columns, and views, enabling users to search for and access these assets directly within the Alation interface. Currently, this connector supports Dremio Software, but Dremio Cloud support is on the roadmap. 

Key Features of the Dremio Connector 

The Dremio Connector enhances the power of both platforms with the following core features: 

● Automated Metadata Extraction (MDE): The connector automatically extracts metadata from Dremio, ensuring your Alation catalog remains up-to-date.

● Search & Discovery: Quickly search for Dremio assets such as tables, schemas, and views within Alation, making it easier to find and reuse data. 

● Catalog Curation: Organize and enhance your catalog pages, improving data visibility for end users. 

● Data Quality Flags: Propagate data quality flags from Dremio to Alation, helping users trust the data they’re working with. 

● Popularity Indicators: See which data assets are being used the most across the organization, helping prioritize important datasets. 

Who Should Install the Connector? 

The Dremio Connector installation requires collaboration between Alation Administrators and Dremio Administrators: 

● Alation Administrator: 

○ Installs the connector. 

○ Configures Dremio as a data source within Alation’s Catalog. 

● Dremio Administrator: 

○ Creates a service account with necessary permissions.

○ Provides Dremio server and port information, along with a username/password or Personal Access Token for authentication. 

This collaboration ensures that the connector is properly installed and your Dremio assets are correctly cataloged in Alation. 

Core Capabilities of the Dremio Connector 

Here’s a breakdown of what the Dremio Connector enables: 

● Automated Metadata Extraction: Metadata from Dremio is automatically extracted and ingested into Alation. 

● Search & Discover: Quickly search for schemas, tables, views, and other Dremio assets in the Alation interface. 

● Catalog Curation: Organize and manage your data assets for enhanced discoverability. 

● Data Quality Flags: Tag data assets with quality flags, ensuring users can trust the data they use. 

● Sampling & Profiling: Preview Dremio content before deep analysis with sampling, and soon, customizable profiling features. 

Upcoming Features 

While the current capabilities are powerful, several advanced features are planned for future releases, including: 

● Query-Based Metadata Extraction: Custom queries will allow for more flexible and specific metadata extraction. 

● Lineage Tracking: A visual lineage feature will help teams understand data flows and dependencies. 

● Custom Sampling and Profiling: Soon, users will be able to sample and profile data based on custom queries, offering deeper insights into data quality. 

These upcoming features will further enhance how teams interact with and govern their data across platforms. 

Enhancing Collaboration with Alation 

The Dremio Connector also integrates with Alation’s collaborative tools, making it easier for teams to work together on data projects: 

● Threaded Conversations: Start discussions around specific data assets, and collaborate within Alation or through Slack and Teams integrations.

● Trust Flags: Add data quality endorsements or deprecation notices to your catalog entries, ensuring that users can trust the data they’re using. 

● ALLIE AI: The AI-powered assistant helps rename columns with more business-friendly terms, making the data catalog more accessible to non-technical users. 

How to Get Started 

To install the Dremio Software OCF Connector, follow these steps: 

1. Download the Connector: The connector is available as a zip file, which can be uploaded to Alation's Manage Connectors section. 

2. Install the Connector: Alation Administrators should install and configure Dremio as a data source within the Alation Catalog. 

3. Set Up Permissions: Dremio Administrators need to create a service account with the necessary permissions and provide authentication details. 

4. Catalog Your Dremio Assets: Once configured, the connector will automatically catalog your Dremio assets in Alation for easy discovery and governance. 

What’s Next for the Dremio Connector? 

Dremio and Alation are continuously enhancing the Dremio Connector. In addition to the features already mentioned, the roadmap includes support for Dremio Cloud, query-based metadata extraction, and data lineage tracking, among other advanced capabilities. These improvements will help organizations better govern and leverage their data at scale. 

Conclusion 

The Dremio Connector for Alation is a game-changer for organizations looking to streamline data discovery, governance, and collaboration. By easily cataloging your Dremio assets in Alation, you can improve your data’s visibility, trustworthiness, and accessibility across your organization. As new features roll out, the connector will continue to enhance your ability to manage and derive value from your data. 

To get started, check out our installation guide and explore how the Dremio Connector can simplify your data cataloging process. Stay tuned for future updates as we continue to build out new features and expand the connector’s capabilities.