Eliminate Data Transfer Bottlenecks with Apache Arrow Flight

  • Wes McKinneyCreator of the Python pandas project, Co-creator of Apache Arrow, CTO and Co-founder of Voltron Data
  • Matt TopolPrincipal Software Architect, FactSet Research Systems, Inc.
  • Kyle PorterData Architect, Dremio

Session Abstract

Data scientists and analysts need fast access to millions of records. However, the throughput of traditional data transfer APIs such as pyodbc is insufficient to meet their needs, so they extract data into files on their local machines. This results in significant security and governance issues as well as stale data copies.Join us as we explore how Apache Arrow Flight solves this problem by providing a new and modern standard for transporting large data between networked applications. We’ll even run a live bake-off to demonstrate how Arrow Flight enables more than 10x faster transfer rates for highly parallel systems compared to pyodbc.

Ready to Get Started? Here Are Some Resources to Help

Analyst Report

DZone Data Pipelines Trend Report

DZone has released their Data Pipelines Trend Report. It is a survey of software developers, architects, platform engineers, and IT professionals to understand the challenges and potential solutions around ingesting, processing, and leveraging data.

read more


How Enel Group Built a Data Mesh Architecture with Dremio and AgileLab

In this webinar, learn how Enel Group worked with Agile Lab to implement Dremio as a data mesh solution for providing broad access to a unified view of their data, and how they use that architecture to enable a multitude of use cases.

read more


The Digital Bank: Building a data-driven customer experience on the open lakehouse

Learn how direct access to analytics on Amazon S3 with Dremio can help banking and finance firms achieve a consistent and data-driven customer experience.

read more

Get Started Free

No time limit - totally free - just the way you like it.

Sign Up Now

Watch Demo

Not ready to get started today? See the platform in action.

Check Out Demo