Build Data Lake Pipelines at Scale – Using only SQL

Building data pipelines for cloud data lakes is fraught with complexity as organizations aspire to analyze every data type, especially semi-structured event data. Pipelines have become painful and tedious for data engineers to develop and maintain in the face of accelerating scale and frequent change cycles.

This talk will cover:

• The pipeline operations work that burdens data engineering including orchestration, data lake table management and infrastructure management.
• Upsolver’s declarative approach, where you define pipelines using only SQL transformations on raw data. All of the mundane engineering work is automated.
• High-scale pipeline examples across several industries and use cases.