Real – Time Hybrid Cloud Data Streaming

Migrating to the cloud presents many challenges around data movement. Wayfair rapidly grew from a $1B business past $10B. Supporting such growth presented challenges in our data streaming and analytics space. In this session, we will share our approach to solving challenges around real-time analytics and streaming our production data into the cloud on the scale of tens of gigabytes to terabytes of data per day.

We’ve built a highly scalable cloud-native data streaming platform that helps us in our gradual transition to the cloud. Apache Beam, together with Google Cloud Platform services, enables massive-scale real-time reporting for our internal customers and business as a whole.

Speakers

Philip Portnoy

Philip Portnoy has 15 years of experience in the data platforms engineering space.

Today he leads a technical team within Wayfair, focused on designing systems and solutions around online data storage. Philip’s team is building tools and automation to provide seamless cloud experience to the engineering and data science community in Wayfair.

The main focus for Philip’s team at the moment is building Wayfair’s DBaaS (database-as-a-service) platform on top of Google’s offerings in the cloud: CloudSQL, Spanner, Compute, and Storage.

Ready to Get Started? Here Are Some Resources to Help

Webinars

Smart Data – Smart Factory with Octotronic and Dremio

Guides

What Is a Data Lakehouse?

The data lakehouse is a new architecture that combines the best parts of data lakes and data warehouses. Learn more about the data lakehouse and its key advantages.

Whitepaper

Simplifying Data Mesh for Self-Service Analytics on an Open Data Lakehouse

The adoption of data mesh as a decentralized data management approach has become popular in recent years, helping teams overcome challenges associated with centralized data architecture.