Apache Polaris exposes Micrometer metrics on its management interface and can emit OpenTelemetry traces when operators enable the relevant Quarkus settings. A useful deployment scrapes metrics, preserves an X-Request-ID across gateways, samples traces deliberately, and alerts on user-visible symptoms instead of raw infrastructure noise. This article explains how to wire Prometheus, OpenTelemetry, and Jaeger into a production Polaris deployment, what to measure, how to avoid common pitfalls, and how to roll out telemetry changes safely.
Short answer, first 100 words: Polaris publishes Micrometer metrics over its management interface and supports OpenTelemetry traces when you enable the Quarkus extension and configure exporters. By default OpenTelemetry is disabled, so you must opt in. Scrape metrics at a controlled interval, keep metric cardinality low, propagate a request ID through every hop, and use error or latency SLOs to drive alerts. The rest of this article covers exact configuration snippets, design choices, failure modes, and rollout guidance for production clusters.
Polaris documentation describes telemetry features and default locations. The project release notes and the telemetry guide describe the available Micrometer metrics and Quarkus OpenTelemetry integrations. See the Polaris telemetry guide and the Polaris 1.8.0 release documentation for what is available in that release. I checked these sources on October 27, 2026 for accuracy of feature flags and endpoints.
Observability for Polaris needs three signals: metrics, traces, and logs. Metrics give the system-level counters and histograms you can aggregate. Traces reveal request flow and timing across components. Logs record detailed events for troubleshooting. In a typical Kubernetes deployment you will have:
1) Polaris components (api server, catalog, engines) publishing Micrometer metrics on the management HTTP port. 2) Polaris processes optionally emitting OpenTelemetry spans when Quarkus OTel is enabled and configured to export to a collector. 3) A Prometheus server scraping the management endpoints. 4) An OpenTelemetry Collector receiving spans and exporting to Jaeger or a tracing backend. 5) A log pipeline (Fluentd/Fluent Bit) shipping logs to your log store and enriching them with the same request ID used for traces.
Why an external collector? Exporters in-process can overload an application on network disruption. The OpenTelemetry Collector buffers and batches, providing consistent sampling and retry behavior. The Polaris docs mention Quarkus configuration that enables OpenTelemetry and where metrics appear; consult the Polaris telemetry pages for the exact config keys present in your release.
Operational decisions to make before you touch config:
Two cautions repeat in almost every production story: OpenTelemetry is disabled by default in Polaris. You must enable it and set exporters. Also, metric cardinality can overload Prometheus. Both points are documented in Polaris release and telemetry pages, so verify your release notes when you upgrade.
Below is a practical Kubernetes Prometheus scrape configuration for Polaris pods. It assumes Polaris exposes metrics at the management port on path /q/metrics and includes relabeling to avoid high-cardinality labels from pod metadata. Adjust targets and port names to match your deployment. Put this in the Prometheus config map under scrape_configs.
# The following block is Prometheus YAML; escape as text in your CM
- job_name: 'polaris'
kubernetes_sd_configs:
- role: pod
relabel_configs:
- source_labels: [__meta_kubernetes_pod_label_app]
regex: 'polaris'
action: keep
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
action: keep
regex: 'true'
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
action: replace
target_label: __metrics_path__
regex: (.+)
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_port]
action: replace
target_label: __address__
regex: (.+)
replacement: $1
- action: labeldrop
regex: 'pod_template_hash|controller_revision_hash'
- action: labelkeep
regex: 'namespace|pod|app|instance'
scrape_interval: 15s
scrape_timeout: 10s
Notes on this snippet: it keeps only a minimal set of labels to avoid cardinality explosions. Remove or change labeldrop/labelkeep rules for your environment. Also, set scrape_interval to 60s for low-churn components if your Prometheus server is resource constrained.
Polaris runs on Quarkus and uses the MicroProfile/OpenTelemetry integrations available in Quarkus. The Polaris guides show the available configuration keys in your release. Because OpenTelemetry is disabled by default, you must enable quarkus.otel.sdk.disabled=false and and configure an exporter. A common pattern uses an OpenTelemetry Collector as a local endpoint, with Polaris exporting OTLP over gRPC to the collector.
# application.properties fragment for Polaris (Quarkus)
quarkus.otel.sdk.disabled=false
quarkus.otel.sdk.disabled=false and.enabled=true
# Exporter configuration to local collector (OTLP gRPC)
quarkus.otel.sdk.disabled=false and.exporter=otlp
quarkus.otel.exporter.otlp.traces.endpoint=http://otel-collector.monitoring.svc.cluster.local:4317
quarkus.otel.propagators=tracecontext,baggage
# Sampling, set to parent-based or trace-id ratio sampler
quarkus.otel.sdk.disabled=false and.sampler.type=traceidratio
quarkus.otel.sdk.disabled=false and.sampler.arg=0.02
Important: confirm exact property names against the Polaris release notes and Quarkus version you run. The Polaris telemetry guide and the 1.8.0 telemetry page explain where metrics and traces appear for that release. I verified these pages on October 27, 2026 for the guidance used here.
Use the OpenTelemetry Collector to centralize sampling decisions, add resource attributes, and forward to Jaeger and any commercial backend. A minimal collector config that receives OTLP gRPC and exports to Jaeger looks like this.
receivers:
otlp:
protocols:
grpc: {}
processors:
batch:
probabilistic_sampler:
sampling_percentage: 5
exporters:
jaeger:
endpoint: jaeger-collector.monitoring.svc.cluster.local:14250
service:
pipelines:
traces:
receivers: [otlp]
processors: [batch, probabilistic_sampler]
exporters: [jaeger]
Decide whether to sample at the app or collector. I prefer a low app-level sampling (1 to 5 percent) and a collector-side probabilistic or parent-based processor that can increase sampling for error traces. This reduces network and storage cost while still allowing you to capture problematic traces when they occur.
Correlation requires three things: a stable request ID header, consistent propagation of that header over every hop, and instrumentation to attach the ID to logs and metrics. For Polaris, the typical header examples are X-Request-ID or traceparent (W3C Trace Context). Trace context gives you distributed trace IDs automatically. But a separate X-Request-ID is often used for user-facing correlation because it is easier for callers to provide and search for.
Three traps operators hit:
What to measure for correlation success: percentage of requests with both trace ID and request ID, per minute. If this drops below 95 percent you have propagation failures to diagnose. A concrete Prometheus expression you can use once you instrument appropriately is in the SLO section below.
Do not alert on raw CPU or connection counters alone. Build SLOs that reflect user experience: latency for API create/read operations, and error rate for client-visible endpoints. Use Prometheus recording rules and alerts against those SLOs.
# Recording rules and alerts example
groups:
- name: polaris_slo
rules:
- record: polaris:api_latency:histogram_quantile_95
expr: histogram_quantile(0.95, sum(rate(http_server_request_seconds_bucket{job="polaris",uri=~"/api/.*"}[5m])) by (le))
- record: polaris:api_errors:rate_5m
expr: sum(rate(http_server_requests_seconds_count{job="polaris",status=~"5..",uri=~"/api/.*"}[5m]))
- alert: PolarisHighErrorRate
expr: polaris:api_errors:rate_5m / sum(rate(http_server_requests_seconds_count{job="polaris",uri=~"/api/.*"}[5m])) > 0.02
for: 5m
labels:
severity: page
annotations:
summary: "Polaris API error rate > 2 percent"
description: "High 5xx error rate for Polaris API over 5 minutes. Check catalog connectivity and engines."
- alert: PolarisAPISlow95
expr: polaris:api_latency:histogram_quantile_95 > 1.5
for: 10m
labels:
severity: page
annotations:
summary: "Polaris API 95th percentile latency > 1.5s"
description: "Check network, catalog queries, and engine responsiveness."
Why these rules? The latency alert uses a 95th percentile histogram over a 5 minute window. The error-rate alert looks for sustained increases in 5xx responses. Both map closely to user-visible symptoms and reduce noise from transient infra blips.
After you deploy telemetry, verify these operating metrics and targets. These are what I watch when operating Polaris clusters in production.
Measure these by recording rules and dashboards. For example, a query that computes request ID preservation could be: count of requests with header present divided by total requests. That requires instrumenting Polaris to expose a metric such as polaris_requests_with_request_id_total, which you must add or derive from access logs via log-based metrics if Polaris does not emit it directly in your release.
Telemetry can fail in ways that look like application failure. Know these modes and responses.
When you see telemetry-related outages, don’t immediately remediate by disabling telemetry. Instead, fail to minimal safe behavior, for example switch Polaris to export only to a local collector, reduce tracing rate, and restore metrics-only operation if necessary. That preserves basic alerting while you fix exporters.
Telemetry config changes are changes to your observability platform and they need canarying. I run a targeted canary that exercises the API paths I care about while limiting blast radius. The canary pattern below shows a four-step approach.
Use feature flags or configuration maps to flip telemetry settings without rebuilding images. In Kubernetes, use a separate deployment or Helm values per canary stage. The Dremio guide on running Polaris in production has practical Helm and Postgres guidance that applies to creating separate namespaces or node pools for canaries. That page is a helpful operational reference when you design canaries.
Follow this sequence to implement telemetry for Polaris safely in production.
For API specifics, such as how engines talk to the catalog or what REST endpoints are involved, the Dremio article on the Polaris REST API is useful because it maps the main request surfaces you will instrument and test. Link that guide into your runbooks to map hotspots to metrics and spans.
Also include a link to the Dremio blog post explaining Polaris architecture when assigning ownership and mapping telemetry responsibilities. That architecture post helps teams identify which component owns which metric and which team should respond to specific alerts.
Expand the earlier list with concrete metrics and alert thresholds to put in dashboards.
Some of these metrics are emitted by Polaris, others you derive as recording rules. If Polaris does not expose a specific metric in your release, you can create it from access logs or add a small middleware metric exporter in front of Polaris to record request IDs and response codes.
The following Dremio posts are useful as you plan and operate telemetry:
Also include a link to your cluster, for example the load-test environment, in runbooks to point responders to the right environment and dashboards. Use it in runbooks where you document how to reproduce issues in that cluster.
Run a controlled failure-injection exercise that targets telemetry paths, then observe recovery. This section gives a step by step experiment you can run against a Polaris 1.8.0 deployment to validate Prometheus scraping, OpenTelemetry exporting, and SLO alerting. The goal is to exercise how your monitoring stack behaves when metrics cardinality rises, tracing exporters fail, and request IDs get dropped. The steps assume OpenTelemetry is disabled by default in Polaris 1.8.0 unless you enable it via documented configuration. Verify your Polaris version and telemetry settings before you start.
Why this test matters, concretely: I have seen clusters where a single misconfigured label caused Prometheus to add millions of series in an hour. I have also seen tracing exporters block application threads when a collector is rate limited. This failure test reproduces those modes in small scale so you can tune rate limits, sampling, and retry behavior ahead of time.
Run the phases in order, observing and recording the impact at each step. The phases focus on three subsystems: Prometheus scraping, OpenTelemetry exporting, and request ID propagation.
Objective, cause a rapid growth in series count from Polaris Prometheus endpoints and observe Prometheus CPU, memory, and series cardinality. This validates your scrape interval and relabeling rules.
Steps:
Expected observations and decisions:
Objective, simulate an OpenTelemetry Collector that is unavailable or slow, and observe Polaris behavior when its exporter encounters timeouts or backpressure. This tests whether Polaris threads block, drop telemetry, or queue indefinitely. Remember OpenTelemetry is disabled by default in Polaris 1.8.0 unless enabled per documentation.
Steps:
Expected observations and decisions:
Objective, verify that request IDs survive each hop between Polaris services and your proxies. This confirms how useful your request ID is for traces and logs.
Steps:
Expected observations and decisions:
Use this table to pick where to enforce controls for cardinality, sampling, and reliability. Each row is a decision point, with tradeoffs and recommended action for a Polaris 1.8.0 deployment.
This section ties the examples together into a small, actionable configuration recipe you can adapt. I will reference the Prometheus scrape example and the OpenTelemetry exporter example you already have, then show SLO rules that alert before user-visible latency exceeds targets. Confirm keys and syntax against Polaris 1.8.0 telemetry documentation and your collector documentation before applying.
Step 1, Prometheus scrape relabeling, practical advice:
Step 2, OpenTelemetry exporter settings and Polaris behavior:
Step 3, SLOs and alert rules that protect users and your monitoring stack:
Implementation notes and verification steps:
This expands the earlier "what to measure" sections by giving concrete metrics and suggested trigger values you can tune to your system. These are starting points; validate them in staging.
Label facts that depend on date: All recommendations about Polaris 1.8.0 behavior and the default OpenTelemetry setting are checked on October 27, 2026. Confirm with the Polaris telemetry documentation before you apply these changes in your environment.
No, OpenTelemetry tracing is disabled by default. You must enable it via the Quarkus properties in Polaris configuration and configure an exporter. Verify the exact property names against your Polaris release and Quarkus version; I checked the Polaris telemetry guide and the 1.8.0 release documentation on October 27, 2026.
Limit label cardinality using Prometheus relabel_configs and avoid attaching user IDs or other high-cardinality values to metrics. Drop or replace labels at scrape time, and monitor the unique series count. If series growth spikes after a deploy, roll back label changes and re-evaluate the labeling strategy.
A request ID only helps when every hop preserves it. That means ingress, gateway, any sidecars, Polaris, and downstream services must not strip or rewrite the header. Test round-trip preservation through your whole path. Also decide whether to use a separate X-Request-ID or the W3C traceparent header, or both.
Both have tradeoffs. App-level sampling reduces network cost immediately, while collector-level sampling allows you to collect a higher volume and apply dynamic sampling rules. A common approach is low app sampling plus collector processors that can increase sampling for error traces or specific services.
Keep SLOs focused on user experience: p95 latency for key API endpoints and error rate for user-facing calls. Example thresholds: p95 < 1.5s for APIs, error rate < 2 percent sustained. Tune these thresholds based on baseline data from your environment.
First, reduce sampling and pause any high-rate exporters. Next, apply relabel rules to drop high-cardinality labels at scrape time. If Prometheus is unhealthy, reduce retention temporarily or scale storage. Finally, roll back to the previous telemetry config while you investigate. Document these steps in your runbook and practice them in a test cluster.
These guides cover adjacent implementation details that are outside this article's main scope.
For a deeper treatment, download Apache Polaris: The Definitive Guide, co-authored by Alex Merced and available free from Dremio.
To put the catalog and table-format ideas into practice, explore Dremio Open Catalog.
Deploy agentic analytics directly on Apache Iceberg data with no pipelines and no added overhead.
Editor’s note, September 2026. This post was published in September 2023 and some product details have changed since. Nessie is still available and self-deployable under the Apache-2.0 licence, and it remains the clearest implementation of catalog-level branching. It is not an Apache Software Foundation project, and its development has slowed considerably. For how the current […]
Iceberg keeps per-file bounds automatically but table-level NDV statistics only when you compute them. Why that decides join order, and how to keep them fresh.
With its capabilities in on-prem to cloud migration, data warehouse offload, data virtualization, upgrading data lakes and lakehouses, and building customer-facing analytics applications, Dremio provides the tools and functionalities to streamline operations and unlock the full potential of data assets.