High-Performance Big Data Analytics Processing Using Hardware Acceleration

In order to address the challenge of increasingly expensive and time-consuming big data analytics pipelines, hardware accelerators such as GPUs and FPGAs are increasingly being used to reduce the overhead associated with data processing, and improve the utilization as well as the cost and power efficiency of compute infrastructure. These systems are being integrated in various cloud services such as Amazon and Nimbix, and have become a prime feature of the Microsoft Azure offering.This talk will give an introduction to FPGAs and discuss their advantages and challenges in the context of big data analytics. We’ll also discuss Fletcher, an open source platform to integrate FPGA accelerators with big data analytics frameworks efficiently. Based on Apache Arrow, Fletcher is intended to tackle the challenges of long development times and poor cross-platform support, and FPGA components are easily integrated into Arrow pipelines. We will present several high-throughput applications where FPGA accelerators are integrated into big data analytics pipelines. This includes regular expression matching achieving up to 60x acceleration, Parquet decompression and Arrow conversion at 3x acceleration allowing real-time Parquet data ingest, and ultra-low latency JSON to Arrow conversion.Finally, we will demonstrate FPGA integration into Dremio, allowing for the transparent acceleration of SQL queries on high-performance accelerators.

Topics Covered

Apache Arrow
In-Memory Formats


Zaid Al-Ars

Zaid Al-Ars

Zaid Al-Ars is an associate professor at Delft University of Technology, where he leads the Accelerated Big Data Systems group, focusing on developing computing infrastructures for efficient processing of big data analytics applications. Zaid is also co-founder of a couple of big data companies specializing in high-performance analytics solutions and AI, and serves on the advisory board of a number of high-tech startups.

Ready to Get Started? Here Are Some Resources to Help

Case Study

Case Study

Dremio Supports Moonfare’s High-Performance Culture with a High-Performance Lakehouse

Moonfare replaced a PostgreSQL-based data warehouse on Amazon Web Services (AWS) with a Dremio data lakehouse to offer data engineers, analysts and business users a high performance platform for business intelligence and predictive analytics empowering them to make better data-driven decisions.

read more

Case Study

Case Study: DB Cargo Gives Users the Green Light to All Data with Dremio

Deutsche Bahn Group (DB) is one of the world's leading mobility and logistics companies. The DB Cargo business unit manages DB's rail freight business.

read more
Case Study

Case Study

Case Study: Amazon Accelerates Supply Chain Decision Making with Dremio

Amazon's Supply Chain Finance Analytics team developed a new analytics architecture with Dremio to simplify ETL processes, accelerate queries, and provide analytics on a unified view of the data.

read more

Get Started Free

No time limit - totally free - just the way you like it.

Sign Up Now

See Dremio in Action

Not ready to get started today? See the platform in action.

Watch Demo

Talk to an Expert

Not sure where to start? Get your questions answered fast.

Contact Us