Apache Polaris includes a MongoDB-backed NoSQL metastore option, but the current documentation treats it as experimental. Teams evaluating it must configure and bootstrap the metastore with matching server and admin-tool versions, schedule daily NoSQL maintenance, test backup and restore, and compare results with the PostgreSQL path Polaris recommends for production.
The MongoDB metastore option is experimental. Evaluate it as an operations problem that includes architecture, bootstrap ordering, daily maintenance, backup, restore, and failure recovery. The behavior described here was checked against Polaris 1.8.0 and the in-development MongoDB documentation on November 9, 2026, so verify version-specific configuration before deployment.
Short version up front: treat MongoDB persistence as experimental, run the Polaris admin-tool that matches your server version when you bootstrap, automate a daily maintenance job for MongoDB compaction and repair, and have a tested fallback plan to PostgreSQL if you need a production-ready metastore. The Polaris docs and admin-tool pages are the primary references for exact feature flags and steps. See the Polaris metastore pages for 1.8.0 and the in-development MongoDB metastore doc for version-specific details.
Polaris stores metadata about resources, policies, catalogs, and access control in a pluggable metastore. The MongoDB implementation maps Polaris entities to BSON documents in a set of collections. The metastore implementation is documented as experimental in the Polaris in-dev metastore page, and 1.8.0 documents PostgreSQL as the recommended production metastore. Read both the in-development MongoDB metastore documentation and the 1.8.0 metastore overview to understand the supported capabilities and limits.
High level components:
Design consequences:
Polaris servers call a MongoDB replica set through the configured metastore connection. Document replica-set topology, backup boundaries, connection settings, and the oplog window expected under peak write load. If sharding is introduced, include routers and config servers in the failure and restore model.
Before committing to MongoDB persistence, answer these questions. They determine whether the experimental path is realistic for your environment.
I will show a minimal, explicit configuration for Polaris 1.8.0 servers using MongoDB. Confirm key names in your release, and verify that your admin-tool and server versions match before bootstrapping. The Polaris 1.8.0 metastore and admin-tool pages list the exact flags and environment variables the release recognizes.
Polaris server application.conf snippet. Replace hosts, credentials, and database name. This is an example; check your Polaris release for exact property names.
<!-- language: text -->
polaris.persistence.type = "nosql"
polaris.persistence.nosql.provider = "mongodb"
polaris.persistence.nosql.mongodb.connection-string = "mongodb://polaris_user:password@mongo1:27017,mongo2:27017/?replicaSet=rs0&readPreference=primaryPreferred"
polaris.persistence.nosql.mongodb.database = "polaris_metastore"
polaris.persistence.nosql.mongodb.write-concern = "majority"
Notes on the snippet:
Use a user with a minimal set of privileges required to create collections, create indexes, and perform reads and writes. For example, create a role that permits readWrite on the polaris_metastore database plus clusterMonitor or the least privilege combination your MongoDB version supports. Keep network traffic internal, and prefer TLS between Polaris and MongoDB in production.
The admin-tool creates the initial collections and indexes. Confirm its output during bootstrap. If you later add an index manually, test upgrade sequences and watch for index builds impacting primary performance. For large datasets, consider building indexes using background or rolling index build strategies that your MongoDB version supports.
Polaris ships an admin-tool that performs metastore bootstrap and offline migrations. The admin-tool and the server must be exactly the same Polaris version when you bootstrap. The Polaris 1.8.0 admin-tool page lists the tool usage and flags for that release. Using a mismatched admin-tool can create incompatible data structures.
Example bootstrap sequence, assuming Polaris 1.8.0 artifacts. Substitute paths for your environment. Run the admin-tool on a host that has network access to the MongoDB replica set but is not competing with server instances during schema creation.
<!-- language: bash -->
# Unpack or download the exact admin-tool release that matches your Polaris server
tar xzf polaris-admin-tool-1.8.0.tgz
cd polaris-admin-tool-1.8.0
# Verify admin-tool version matches server
./polaris-admin --version
# Run bootstrap against MongoDB; check logs for index creation and collection count
./polaris-admin bootstrap \
--persistence-type nosql \
--nosql-provider mongodb \
--nosql-connection-string "mongodb://polaris_user:password@mongo1:27017,mongo2:27017/?replicaSet=rs0" \
--nosql-database polaris_metastore
Expected admin-tool output and checks:
Start Polaris servers only after successful bootstrap. If you start a server before bootstrap, it may attempt to auto-create collections using a different schema or partial indexes, which can lead to later incompatibilities. The bootstrap ordering matters, the admin-tool must be run first, then servers can start and join the cluster.
The Admin Tool connects to MongoDB first and performs the documented bootstrap work. Polaris servers start only after bootstrap succeeds. Capture logs from both steps and fail deployment when server startup cannot verify the expected metastore state.
After bootstrap, start Polaris servers in a rolling manner. For a rolling upgrade of the Polaris application alone, you can run different server instances one at a time. However, a persistence-type migration is different from a rolling application upgrade. If you need to switch from MongoDB to PostgreSQL you must plan a data migration and cutoff window; do not assume you can change persistence type in place without downtime or irreversible steps.
When performing a server upgrade that includes metastore schema migrations, always:
Polaris documentation and operational experience indicate that daily NoSQL maintenance is part of normal operation for MongoDB persistence. You must schedule recurrent tasks to keep the MongoDB deployment healthy, reduce fragmentation, and ensure backups capture consistent points in time. MongoDB documentation lists backup best practices and options. The following is a practical daily maintenance cycle to automate and test. Adjust timing for your traffic and data retention needs.
Example scheduled daily script outline. Replace tool names and credentials for your environment. This is a template for a cron job or a Kubernetes CronJob.
<!-- language: bash -->
#!/bin/bash
# daily-mongo-maintenance.sh
# 1) check replica set status
mongosh --host "mongo1:27017" --username polaris_user --password "password" --eval "rs.status()" > /var/log/polaris/mongo-rs-status.log
# 2) check replication lag
mongosh --quiet --eval 'db.getMongo().getDBs()' > /dev/null
# Replace with appropriate monitoring query to measure replication lag
# 3) run compact selectively (WiredTiger)
# Do NOT run compact on entire DB at high traffic times; use maintenance windows
mongosh --host mongo1:27017 --username polaris_user --password "password" polaris_metastore --eval "db.getCollectionNames().forEach(function(c){ if (db[c].stats().wiredTiger && db[c].stats().size>100000000) { print('Compacting',c); db.runCommand({compact:c}); } })"
# 4) trigger snapshot backup (example: LVM snapshot, filesystem snapshot, or cloud provider snapshot)
# Replace with your provider snapshot command
Daily maintenance consequences and cautions:
A daily maintenance timeline should account for health checks, the documented NoSQL maintenance command, backup creation, and scheduled restore testing. Place resource-intensive work outside peak catalog load and keep backups beyond the longest required recovery window.
Backups are only useful if you can restore them in a time and manner that satisfies your recovery objectives. MongoDB guides present several backup strategies, including mongodump/mongorestore, filesystem or volume snapshots, and cloud-provider continuous backups. For a Polaris metastore, do not assume mongodump is sufficient for large datasets without testing the restore under load. The MongoDB backups documentation lists the tradeoffs for different approaches.
Test scenarios you must automate and validate:
Worked restore example, using mongodump/mongorestore. This is suitable for small test datasets; larger production datasets usually require snapshots or continuous backup features.
<!-- language: bash -->
# dump data from production replica primary
mongodump --host mongo1:27017 --username polaris_user --password "password" --db polaris_metastore --out /backups/polaris_metastore-$(date +%F)
# restore to a test cluster
mongorestore --host test-mongo1:27017 --username polaris_user --password "password" --db polaris_metastore /backups/polaris_metastore-2026-11-09/polaris_metastore
# start Polaris against the restored test metastore and run smoke tests
After restore, run automated Polaris functional tests: list catalogs, fetch policies, evaluate a sample access check, and validate user authentication flows. Record timings for each test and compare to pre-failure baselines.
Because the MongoDB metastore in Polaris is documented as experimental, you need a strict checklist before you consider it production-ready for any critical environment. A single unchecked item should keep you on the PostgreSQL path Polaris recommends for production. This is a decision gate, not a checklist you can half-complete.
The production readiness diagram should show a gate with the required pass/fail items: documentation match, bootstrap success, daily maintenance automation, backup and restore demonstrated, and rollback plan validated. If any item fails, stop and reassess.
If any item is missing, keep MongoDB metastore limited to development and extended testing environments. Polaris publishes a guide for running in production using Kubernetes, Helm, and PostgreSQL; consult that page when your risk tolerance requires a supported production path.
Here are concrete failure modes you will encounter and the signals to monitor. I include mitigation steps you can automate or document for operators.
Follow this sequence for a safe evaluation rollout. Each step has a test and a go/no-go decision. Do not skip tests to save time.
Track these metrics continuously to detect degradation early. Set alerts and runbook links for each metric.
Keep a change log of schema changes and admin-tool operations tied to release versions. If you observe a metric regression after an admin-tool migration, you will need to correlate the migration to any metadata format changes.
If you are running Polaris on Kubernetes and plan to use the recommended PostgreSQL path in production, see the guide that explains running Polaris in production with Kubernetes, Helm, and PostgreSQL. That guide shows patterns for reliable upgrades and persistence management that you can adapt to a MongoDB-backed metastore in staging.
For architecture context about how Polaris components interact, including placement of the metastore role, read the Polaris architecture overview. That article helps you think about where to run the admin-tool, how to design network topology, and how clients talk to Polaris servers.
If you are new to Polaris and want a general orientation to the project and its goals, the "What is Apache Polaris" guide gives conceptual background that helps when you evaluate tradeoffs between metastore types and operational effort.
This is a practical sequence of tests you should run. Each step produces artifacts you must store for postmortem and decision-making. Expect to iterate multiple times.
Do not assume parity between MongoDB and PostgreSQL metastore behavior. They differ in query patterns, index semantics, storage, and operational demands. Do not assume you can change persistence type as part of a rolling upgrade. A persistence-type migration is a separate project that requires data export or conversion, and it often needs downtime or coordinated cutover steps.
If you are experimenting, use MongoDB metastore in staging with automated maintenance and frequent restore tests. If you require a stable, supported production metastore today, follow Polaris guidance and use PostgreSQL for production deployments. Keep documentation for your exact Polaris release and admin-tool handy. Facts checked on November 9, 2026.
No, the MongoDB metastore is documented as experimental. For production, Polaris documentation recommends PostgreSQL. Verify your release notes before deciding.
Yes, daily NoSQL maintenance is part of normal operation for a MongoDB-backed Polaris metastore. Schedule health checks, selective compaction, and backup snapshots as described earlier.
No, a persistence-type migration is different from a rolling application upgrade. It requires an explicit migration plan, data export/import, and likely downtime or coordinated cutover steps. Test the migration end to end.
That depends on dataset size and RTO/RPO. For small datasets mongodump/mongorestore can work. For larger datasets, use filesystem or cloud snapshots or a continuous backup solution as described in MongoDB backups guidance. Test restores to verify the approach.
Replication lag, primary disk IOPS and latency, index build activity, storage growth, Polaris API error rates, and backup/restore durations. Track baselines and alert on deviations.
Run the admin-tool from a secure host with network access to the MongoDB replica set. Prefer running it from an operations host or a CI/CD runner with audited access, and ensure the admin-tool version matches your Polaris server. Do not run bootstrap with a mismatched admin-tool version.
Continue with these related implementation guides.
For a deeper treatment, download Apache Polaris: The Definitive Guide, co-authored by Alex Merced and available free from Dremio.
To put the catalog and table-format ideas into practice, explore Dremio Open Catalog.
Deploy agentic analytics directly on Apache Iceberg data with no pipelines and no added overhead.
Editor’s note, September 2026. This post was published in September 2023 and some product details have changed since. Nessie is still available and self-deployable under the Apache-2.0 licence, and it remains the clearest implementation of catalog-level branching. It is not an Apache Software Foundation project, and its development has slowed considerably. For how the current […]
Iceberg keeps per-file bounds automatically but table-level NDV statistics only when you compute them. Why that decides join order, and how to keep them fresh.
With its capabilities in on-prem to cloud migration, data warehouse offload, data virtualization, upgrading data lakes and lakehouses, and building customer-facing analytics applications, Dremio provides the tools and functionalities to streamline operations and unlock the full potential of data assets.
