Today we’re excited to announce the release of the JDBC driver for Apache Arrow Flight SQL, an open source driver that specializes in moving large amounts of data very quickly. This driver benefits Dremio users in terms of performance, efficiency, and time to insight. It allows you to use any JDBC client and connect to your Dremio datasets. You can also connect to any data source that supports Arrow Flight SQL or directly through Python. This new driver works with Dremio Version 21.0+.
Currently, most data analysts and data scientists must use one proprietary driver per database when connecting to their data. This requires them to download and keep each driver up to date, which is cumbersome, especially for less technical users.
The Arrow Flight SQL JDBC driver streamlines the connection experience by creating a universal driver and bringing the performance of Arrow Flight to every JDBC client. As more Arrow Flight SQL endpoints are added to databases, JDBC clients can connect to those databases. Flight SQL is a standardized protocol, so the same driver can talk to any system that supports this protocol. The Apache Arrow Flight protocol saves time and network resources by not having to serialize and then deserialize the data. Using this modern protocol allows you to move large amounts of data over the network quickly, which reduces time to insight and enables users to ask more questions and work with larger datasets.
Explore this interactive demo and see how Dremio's Intelligent Lakehouse enables Agentic AI
Dremio has a long history of working with and contributing to the Apache Arrow community. In that spirit, this driver is now available to the Apache Arrow community. We will collaborate with the broader community to extend the driver to meet the growing community and industry needs.
If you’re using Dremio Cloud, the new driver works with Dremio Cloud with no additional effort. If you’re using Dremio Software, simply upgrade to version 21, which includes the Arrow Flight SQL endpoint that enables you to use the new JDBC driver. To get started with the new driver, download and install it on your client.
We are working with various JDBC-based BI tools to get the new Arrow Flight SQL JDBC driver natively embedded so you won't have to install or manage the driver — it will automatically be part of your choice of tools.
You can leverage the new Arrow Flight JDBC driver and start using it with your Dremio cluster today. For example, here’s an overview of how you would use the new driver with the DBeaver client.
1. First, you need to create a new driver by following these instructions for the DBeaver client in Dremio documentation.
2. Once your driver is set up, you will see it in the Driver Manager.
3. Next, you will create a new connection from the JDBC URL. Then you will see the option of the new driver.
4. And success, you are connected to your data in Dremio.
5. You can now query Dremio from an existing client tool, but under the covers, leverage the performance benefits of Arrow Flight. Give it a try!
Use the following resources to learn more about Apache Arrow Flight and the new JDBC driver:
Deploy agentic analytics directly on Apache Iceberg data with no pipelines and no added overhead.
By unifying data from diverse sources, simplifying data operations, and providing powerful tools for data management, Dremio stands out as a comprehensive solution for modern data needs. Whether you are a data engineer, business analyst, or data scientist, harnessing the combined power of Dremio and Apache Iceberg will undoubtedly be a valuable asset in your data management toolkit.
This blog helps you learn about table-driven access policies in Dremio Cloud and Dremio Software v24.1+.
Dremio Arctic bring new features to Dremio Cloud, including Apache Iceberg table optimization and Data as Code.
