Webinars

Rethinking Ingestion: CI/CD for Data Lakes

At first glance, ingesting data into a data lake may seem like a one-step process — you simply add files to an object store. What else is there to do?!It turns out that there is more you can do, and blindly writing new data introduces a host of potential problems. For example, how do you know the data you write is accurate and conforms to schema? The truth is, once you’ve written it to the lake, in a sense, it’s already too late.What we propose and will cover in this talk, is a new strategy for data lake ingestion. One where new data can be added in isolation, then tested and validated, before “going live” in a production table. Finally, we’ll show how git-for-data tools like lakeFS and Nessie enable this ingestion paradigm in a seamless way.

Ready to Get Started? Here Are Some Resources to Help

Whitepaper Thumb

Whitepaper

Simplifying Data Mesh for Self-Service Analytics on an Open Data Lakehouse

read more
Whitepaper Thumb

Whitepaper

Dremio Upgrade Testing Framework

read more
Whitepaper Thumb

Whitepaper

Operating Dremio Cloud Runbook

read more
get started

Get Started Free

No time limit - totally free - just the way you like it.

Sign Up Now
demo on demand

See Dremio in Action

Not ready to get started today? See the platform in action.

Watch Demo
talk expert

Talk to an Expert

Not sure where to start? Get your questions answered fast.

Contact Us

Ready to Get Started?

Bring your users closer to the data with organization-wide self-service analytics and lakehouse flexibility, scalability, and performance at a fraction of the cost. Run Dremio anywhere with self-managed software or Dremio Cloud.