CI/CD on the Lakehouse: Making Data Changes and Repair Safe and Easy

In Software development developers use Continuous Integration and Continuous Deployment as a technique to automate the integration of new code safely and quickly. A challenge in the data world is often integrating new data ingested from batches and streams with the same safety and speed. As the data world adopts more and more of these software best practices in a great trend we like to call “Data as Code” how can we begin automating our data integration in ways that are also safe and fast.

In this talk we’ll discuss:
– How to isolate your data ingestion
– How to Audit your ingested Data
– How to integrate your new data
– How to automate these steps

Session ID: BO103