Powering a Data Mesh with Dynamic GraphQL Schema Generation

Exposing thousands of datapoints through a generic API can be difficult. To avoid constantly needing new and different API endpoints to expose new data, FactSet developed an infrastructure for onboarding new datapoints into a GraphQL schema with simple configurable metadata. By leveraging Dremio, Apache Arrow, and GraphQL, we dramatically reduce the cost and time to market of onboarding new content while bolstering performance.

This presentation will look at how we use metadata to dynamically generate a GraphQL schema, how GraphQL queries are translated to run-time queries against Dremio, and how we do this without sacrificing performance or flexibility.

Download PDF

Speakers

Matt Topol

Hailing from the faraway land of Brentwood, NY, and currently residing in the rolling hills of Connecticut, Matt Topol has always been passionate about software. After graduating from Brooklyn Polytechnic (now NYU-Poly), he joined FactSet Research Systems, Inc. in 2009 to develop financial software. In the time since, Matt has worked in infrastructure and application development, has lead development teams, and has architected large-scale distributed systems for processing analytics on financial data. Matt is a committer on the Apache Arrow repository, frequently enhancing the Golang library and helping to grow the Arrow Community. Recently, Matt wrote the first and only book on Apache Arrow, “In-Memory Analytics with Apache Arrow,” and joined Voltron Data in order to work on the Apache Arrow libraries full-time and grow the Arrow Golang community.

In his spare time, Matt likes to bash his head against a keyboard, develop/run delightfully demented games of fantasy for his victims–er–friends, and share his knowledge with anyone interested who’ll listen to his rants.

Stephen Perkins

Ready to Get Started? Here Are Some Resources to Help

Webinars

Smart Data – Smart Factory with Octotronic and Dremio

Guides

What Is a Data Lakehouse?

The data lakehouse is a new architecture that combines the best parts of data lakes and data warehouses. Learn more about the data lakehouse and its key advantages.

Whitepaper

Simplifying Data Mesh for Self-Service Analytics on an Open Data Lakehouse

The adoption of data mesh as a decentralized data management approach has become popular in recent years, helping teams overcome challenges associated with centralized data architecture.