Eliminate Data Transfer Bottlenecks with Apache Arrow Flight

  • Wes McKinneyCreator of the Python pandas project, Co-creator of Apache Arrow, CTO and Co-founder of Voltron Data
  • Matt TopolStaff Software Engineer
  • Kyle PorterData Architect, Dremio

Session Abstract

Data scientists and analysts need fast access to millions of records. However, the throughput of traditional data transfer APIs such as pyodbc is insufficient to meet their needs, so they extract data into files on their local machines. This results in significant security and governance issues as well as stale data copies.Join us as we explore how Apache Arrow Flight solves this problem by providing a new and modern standard for transporting large data between networked applications. We’ll even run a live bake-off to demonstrate how Arrow Flight enables more than 10x faster transfer rates for highly parallel systems compared to pyodbc.

Ready to Get Started? Here Are Some Resources to Help

Case Study

Case Study

Dremio Supports Moonfare’s High-Performance Culture with a High-Performance Lakehouse

Moonfare replaced a PostgreSQL-based data warehouse on Amazon Web Services (AWS) with a Dremio data lakehouse to offer data engineers, analysts and business users a high performance platform for business intelligence and predictive analytics empowering them to make better data-driven decisions.

read more

Case Study

Case Study: DB Cargo Gives Users the Green Light to All Data with Dremio

Deutsche Bahn Group (DB) is one of the world's leading mobility and logistics companies. The DB Cargo business unit manages DB's rail freight business.

read more
Case Study

Case Study

Case Study: Amazon Accelerates Supply Chain Decision Making with Dremio

Amazon's Supply Chain Finance Analytics team developed a new analytics architecture with Dremio to simplify ETL processes, accelerate queries, and provide analytics on a unified view of the data.

read more

Get Started Free

No time limit - totally free - just the way you like it.

Sign Up Now

See Dremio in Action

Not ready to get started today? See the platform in action.

Watch Demo

Talk to an Expert

Not sure where to start? Get your questions answered fast.

Contact Us