Alex Merced

Apache Iceberg 1.12.0 expands v3 engine support, REST catalog operations, streaming reliability, and v4 groundwork. Learn the breaking changes and use a practical upgrade plan.

Apache Polaris 1.8.0 adds semantic-model privileges, safer commit handling, pagination controls, CLI improvements, and important database upgrade steps.

Learn how Iceberg v3 stores geometry and geography, handles CRS metadata, writes Parquet values, and preserves spatial statistics across engines.

See how the Iceberg View Spec stores SQL representations, schemas, dependencies, versions, and dialects so catalogs can share views across engines.

Understand Iceberg table encryption, envelope encryption, key metadata, KMS integration, Parquet modular encryption, and operational tradeoffs.

Learn how Iceberg REST server-side scan planning works, including capabilities, asynchronous plans, polling, cancellation, batching, and tradeoffs.

Explore Iceberg REST multi-table commit semantics, atomicity, conflict handling, client support, and the role of Apache Polaris in coordinated commits.

Use MERGE INTO with Iceberg for idempotent upserts and SCD Type 2 history, with SQL patterns, deduplication, late data, and file maintenance.

Model Iceberg storage cost across bytes, object requests, snapshots, metadata, compaction, and lifecycle rules without breaking table history.

Build Iceberg observability with ScanReport, CommitReport, REST metrics, metadata-table queries, SLOs, alerts, and a practical health dashboard.

Learn how Polaris realms partition identities and metadata for multi-tenant deployments, what they isolate, and which controls still belong outside Polaris.

Configure Apache Polaris with an external OIDC provider, map claims to principals and roles, rotate keys, and troubleshoot Keycloak or Okta authentication.

Understand Polaris generic tables for Delta Lake and other assets, including registration, metadata, access boundaries, beta limitations, and use cases.

Define Polaris policies for data compaction, metadata compaction, orphan cleanup, and snapshot expiry, with inheritance and safe rollout guidance.

Connect Trino to Apache Polaris through the Iceberg REST catalog with OAuth, credential vending, writable tables, security checks, and troubleshooting.

Use DuckDB and iceberg-rust with Apache Polaris through the Iceberg REST catalog, including OAuth, credentials, read and write support, and caveats.

Configure an Iceberg Kafka Connect sink with Apache Polaris, map topics to tables, manage OAuth and credentials, and operate commits and maintenance.

Plan and execute a migration from AWS Glue Data Catalog or Amazon S3 Tables to Apache Polaris with inventory, dual validation, cutover, and rollback.

Compare current Iceberg v3 read and write support across Spark, Flink, Trino, Dremio, AWS Glue, PyIceberg, DuckDB, and iceberg-rust.

Use a repeatable exit cost test to measure lakehouse vendor lock-in across data, metadata, identity, SQL, operations, egress, and migration time.

Deploy Apache Polaris with Helm and PostgreSQL, then configure secrets, probes, scaling, ingress, migrations, backups, upgrades, and observability.

Configure Polaris event listeners and audit logging, choose event fields, protect sensitive data, export records, and build security and operations alerts.

Apache Iceberg 1.12 and Iceberg Rust 0.11 bring geospatial types, Variant improvements, encryption, V4 groundwork, and breaking changes teams should test now.

These are not competing choices. One describes what your platform is made of, the other describes how you organise tables inside it. The costly decision is the copy count.

PyIceberg reads and writes Iceberg from Python with no JVM. Here is how to point it at Apache Polaris, and the metadata work that makes it worth keeping installed.

Flink writes Iceberg tables through a catalog. Here is how to wire it to Apache Polaris, and why the checkpoint interval is the most consequential setting in the whole pipeline.

Orphan files are invisible to snapshot expiration and can quietly quadruple your storage. Here is how cleanup finds them, and how to run it without deleting a live table.

Snapshot expiration is what makes a delete in Apache Iceberg reduce your storage bill. Here is what it removes, what it refuses to remove, and how to run it without losing rollback.

A manifest file is an Avro file listing data files that belong to an Apache Iceberg table, along with per-column statistics for each one. Manifests are why Iceberg can plan a query without listing directories: the engine reads partition summaries in the manifest list to skip whole manifests, then reads column bounds inside the survivors […]

Apache Polaris implements the Iceberg REST Catalog specification, a set of HTTP endpoints that any Iceberg client can speak. The specification covers namespace and table management, an atomic commit protocol built on assertions rather than locks, server-side scan planning, and three different ways for a client to obtain storage access. Because the contract is the […]