Alex Merced

Compare documented Iceberg MERGE, UPDATE, DELETE, overwrite, and branch-write support across Spark, Flink, Trino, and Dremio, with version caveats.

Build restartable Iceberg incremental pipelines with snapshot ranges, append scans, changelog views, durable checkpoints, branch awareness, and gap recovery.

Evolve nested Iceberg structs, lists, and maps safely using field IDs, nullability, type-promotion rules, defaults, and cross-engine compatibility tests.

Evolve Iceberg sort orders safely, measure pruning quality, choose sort or Z-order rewrites, and decide when existing data justifies a controlled rewrite.

Choose Iceberg write distribution mode none, hash, or range, then tune Spark adaptive execution and target file sizes for balanced, efficient writes.

Upgrade Apache Polaris with compatibility checks, persistence schema migrations, rolling deployment gates, token tests, and a practical rollback plan.

Plan a Nessie-to-Polaris Iceberg catalog migration that reuses existing object storage, validates table metadata, preserves rollback, and limits downtime.

Size Apache Polaris with workload-based load tests, PostgreSQL pool limits, Kubernetes requests and limits, replica scaling, and measurable capacity gates.

Design Apache Polaris service principals for machine workloads, rotate client secrets safely, assign least-privilege roles, and audit dormant identities.

Connect Spark 4 to Apache Polaris with the Iceberg REST catalog, OAuth credentials, catalog scopes, and storage credential vending, then verify writes.

Diagnose Apache Polaris authentication failures by separating token issuance, validation, principal mapping, role grants, and credential-vending errors.

Instrument Apache Polaris with Prometheus metrics, OpenTelemetry traces, Jaeger, request IDs, dashboards, alerts, and practical service-level objectives.

Iceberg keeps per-file bounds automatically but table-level NDV statistics only when you compute them. Why that decides join order, and how to keep them fresh.

Recovering Polaris means recovering a metastore and object storage separately. What to back up, the order to restore in, and the failure mode nobody plans for.

What causes an Iceberg commit conflict, the retry properties that govern it, the two isolation levels, and how to design conflicts out of a busy table.

Parquet and ORC are columnar and built for reading. Avro is row-oriented and built for writing. How to choose, and why Iceberg quietly uses all three.

Parquet is how columnar data sits on disk and Arrow is how it sits in memory. Every difference between them follows from that one split.

Iceberg is a table format and Parquet is a file format. They are different layers of one stack, not alternatives, and an Iceberg table contains Parquet files.

Apache Iceberg 1.12.0 expands v3 engine support, REST catalog operations, streaming reliability, and v4 groundwork. Learn the breaking changes and use a practical upgrade plan.

Apache Polaris 1.8.0 adds semantic-model privileges, safer commit handling, pagination controls, CLI improvements, and important database upgrade steps.

Learn how Iceberg v3 stores geometry and geography, handles CRS metadata, writes Parquet values, and preserves spatial statistics across engines.

See how the Iceberg View Spec stores SQL representations, schemas, dependencies, versions, and dialects so catalogs can share views across engines.

Understand Iceberg table encryption, envelope encryption, key metadata, KMS integration, Parquet modular encryption, and operational tradeoffs.

Learn how Iceberg REST server-side scan planning works, including capabilities, asynchronous plans, polling, cancellation, batching, and tradeoffs.

Explore Iceberg REST multi-table commit semantics, atomicity, conflict handling, client support, and the role of Apache Polaris in coordinated commits.

Use MERGE INTO with Iceberg for idempotent upserts and SCD Type 2 history, with SQL patterns, deduplication, late data, and file maintenance.

Model Iceberg storage cost across bytes, object requests, snapshots, metadata, compaction, and lifecycle rules without breaking table history.

Build Iceberg observability with ScanReport, CommitReport, REST metrics, metadata-table queries, SLOs, alerts, and a practical health dashboard.

Learn how Polaris realms partition identities and metadata for multi-tenant deployments, what they isolate, and which controls still belong outside Polaris.

Configure Apache Polaris with an external OIDC provider, map claims to principals and roles, rotate keys, and troubleshoot Keycloak or Okta authentication.