Real – Time Hybrid Cloud Data Streaming
Migrating to the cloud presents many challenges around data movement. Wayfair rapidly grew from a $1B business past $10B. Supporting such growth presented challenges in our data streaming and analytics space. In this session, we will share our approach to solving challenges around real-time analytics and streaming our production data into the cloud on the scale of tens of gigabytes to terabytes of data per day.
We’ve built a highly scalable cloud-native data streaming platform that helps us in our gradual transition to the cloud. Apache Beam, together with Google Cloud Platform services, enables massive-scale real-time reporting for our internal customers and business as a whole.
Philip Portnoy has 15 years of experience in the data platforms engineering space.
Today he leads a technical team within Wayfair, focused on designing systems and solutions around online data storage. Philip’s team is building tools and automation to provide seamless cloud experience to the engineering and data science community in Wayfair.
The main focus for Philip’s team at the moment is building Wayfair’s DBaaS (database-as-a-service) platform on top of Google’s offerings in the cloud: CloudSQL, Spanner, Compute, and Storage.