Data Lake Engines

A data lake engine is an application or service which queries and/or processes the vast sets of data stored in data lake storage. Data lake processing engines like Apache Spark are often used for batch data transformation jobs and machine learning. Data lake query engines such as Dremio and Presto are used to analyze structured and semi-structured data in place for business intelligence (BI) and data science.
...

How Z-Ordering in Apache Iceberg Helps Improve Performance

This tutorial introduces the Z-order clustering algorithm in Apache Iceberg and explains how it adds value to the file optimization strategy.
Read more
...

Apache Iceberg 101 – Your Guide to Learning Apache Iceberg Concepts and Practices

This article provides an introductory course on the concepts and practices of Apache Iceberg tables for running scalable data lakehouses.
Read more
...

Getting Started with Apache Iceberg in Databricks

Getting started with Apache Iceberg in Databricks is straightforward. This article walks through the setup and usage step by step.
Read more
...
Data Lake Engines Apache Spark Subsurface LIVE Sessions

Tracking & Triggering Pattern with Spark Stateful Streaming

Using Apache Spark Stateful Streaming to create services that minimize processing time while keeping everything under defined SLAs.
Read more