Webinars
July 29, 2020
Functional Data Engineering – A Set of Best Practices
Batch data processing (also known as ETL) is time-consuming, brittle and often unrewarding. Not only that, it’s hard to operate, evolve and troubleshoot.In this talk, we’ll discuss the functional programming paradigm and explore how applying it to data engineering can bring a lot of clarity to the process. It helps solve some of the inherent problems of ETL, leads to more manageable and maintainable workloads and helps to implement reproducible and scalable practices. It empowers data teams to tackle larger problems and push the boundaries of what’s possible.
Topics Covered
Data Lake Storage
Speakers
Maxime Beauchemin
Maxime Beauchemin recently joined Lyft as a Software Engineer after some time at Airbnb as a data engineer developing tools to help streamline and automate data engineering processes. He is also the creator and lead committer on Apache Airflow and Apache Superset. He mastered his data warehousing fundamentals at Ubisoft and was an early adopter of Hadoop/Pig while at Yahoo in 2007. More recently, while at Facebook, he developed analytics-as-a-service frameworks around engagement and growth metrics computation, anomaly detection, and cohort analysis. He’s a father of three, and in his free time he’s a digital artist.
