Get Started Free
No time limit - totally free - just the way you like it.Sign Up Now
With the launch of Dremio 4.0.0, I’m delighted to share the Dremio Snowflake Connector that is built with the ARP framework. In this article, let’s see what Dremio is and how the new connector can help you achieve faster time to insight for your organization.
Faster time to insight enables organizations to move rapidly to action and accelerate the speed with which they can bring new product and service capabilities to market.
To enable faster time to insight, organizations across the world have spent considerable money on data collection, storage and analytics. For the most part, data collection and storage has been addressed in the past decade. Hadoop and S3 compatible object storage became the de-facto for storing big data on-prem and S3, GCS, ADLS became the de-facto for cloud storage.
There is also a big list of vendors who provide data visualization like Tableau, Looker, Qlik, Business Objects, SiSense etc.
However, the question of how to provide a semantic layer across the enterprise that can bridge the BI tools with the massive data to achieve the faster time to insight has not been addressed well yet. To address this gap, we need a platform that can not only do the following but also do it well without any dependencies on anything:
Data Virtualization, federation, OLAP on Hadoop solutions falls short in one or many of the above categories.
“Dremio is a data platform that enables businesses to be data-driven by providing faster time to insight with no vendor lock-in”
Dremio addresses the above problems as follows.
Dremio’s contributions to the OSS community is also a big factor on how it is driving the ecosystem towards faster time to insight:
Apache Arrow — The de-facto in memory representation of columnar data that is used by 50+ projects now.
Arrow Flight — Arrow Flight provides a high-performance wire protocol for large-volume data transfer for analytics
Gandiva — Gandiva is a new execution kernel for Arrow that is based on LLVM
Dremio’s Advanced Relational Pushdown (ARP) framework allows data consumers and developers to create custom relational connectors for those cases where data is stored in uncommon locations. Using the ARP framework not only allows you to create better connectors with improved push-down abilities but it also provides a way to develop connectors more efficiently and easily.
The ARP framework lets you build a connector very easily for any data source with a JDBC driver. Even the push downs are simply mappings on YAML which makes it very easy for anyone to create a connector.
With ARP, I started working on building a connector for Snowflake about a month ago. Today,
I’m happy to announce a major release of this connector with the following features:
You can find more information about the connector here: https://github.com/narendrans/dremio-snowflake
Visit Dremio Hub to learn how can you leverage this and other connectors in your organizations.