New platform enables a groundbreaking Git-like experience for data engineers
Santa Clara, Calif – March 2, 2022 – Dremio, the lakehouse company, today announced the general availability of Dremio Cloud, the world’s first free data lakehouse platform and the addition of two new services: Dremio Sonar, a lakehouse engine built for SQL, and Dremio Arctic, a metadata and data management service for Apache Iceberg that provides a unique Git-like experience for the lakehouse. Dremio Sonar is now generally available, and Dremio Arctic is in public preview.
“Dremio Cloud is the world’s first lakehouse platform that was built from the ground up for SQL workloads, including mission-critical BI,” said Billy Bosworth, Chief Executive Officer at Dremio. “In the past, companies had to weigh the pros and cons of data lakes vs. data warehouses. We’ve eliminated this tradeoff by providing a frictionless and infinitely-scalable service that delivers the combined benefits of both.”
Dremio Cloud Is Open and Forever-Free
With an open lakehouse architecture, unlike a cloud data warehouse, data itself takes center stage, no longer hostage to specific vendor formats or systems. Companies can choose the right data processing/query engine for each use case, both today and in the future. There is no vendor lock-in with Dremio because the data is open, by default, persisted in Apache Iceberg tables backed by Apache Parquet files in the company’s Amazon Simple Storage Service (Amazon S3) account. Iceberg is an Apache Software Foundation project with contributors from dozens of companies including Apple, Netflix, Salesforce, and Dremio. Dremio Sonar also supports additional table and file formats, as well as the ability to query data across a variety of data sources beyond Amazon S3, providing users with maximum flexibility.
Dremio is providing a forever-free edition of Dremio Sonar and Dremio Arctic on Dremio Cloud, supporting unlimited production use and infinite scale, with end-to-end security and SOC 2 Type 2 compliance. Dremio Sonar is over 2x faster than Dremio’s existing query engine, thereby requiring less than 50% of the cloud infrastructure previously needed, resulting in substantial cost savings. The Enterprise edition of Dremio Cloud adds advanced security controls, such as custom roles and enterprise identity providers (e.g. Okta), and enterprise support options.
Dremio Sonar Eliminates the Need for Cloud Data Warehouses
Dremio Sonar is a forever-free service that is purpose-built for SQL and delivers all the performance and functionality of a data warehouse on Amazon S3, without lock-in, cost, and excessive data copies. Powered by Apache Arrow, Dremio Sonar’s best-in-class query performance empowers companies to run mission-critical business intelligence (BI), from ad-hoc queries to low-latency dashboards, directly on the lakehouse.
Powered by the open source Apache Iceberg project, Dremio Sonar fully supports SQL Data Manipulation Language (DML) insert, update, delete operations directly on the lakehouse. Because the data is automatically optimized in Amazon S3 as it is being changed, customers no longer need to fall back to a data warehouse when data is rapidly mutating.
“Companies can plug Dremio Sonar into any existing data lake and start querying in minutes, with support for multiple lakehouse metastores, including AWS Glue and Dremio Arctic, as well as the ability to query Amazon S3 directly,” said Tomer Shiran, Founder and Chief Product Officer at Dremio. “In addition, organizations can easily analyze data that resides outside the lakehouse (e.g., in relational databases) and perform live and accelerated joins across multiple sources.”
Dremio Sonar eliminates the need to physically export data into BI extracts/imports, cubes, data marts and derived data warehouse tables, eliminating the need for a cloud data warehouse. Key features include:
- Dremio’s Reflections enable sub-second query response times by automatically and transparently rewriting query plans to utilize different aggregations or layouts of tables and views.
- Dremio Spaces creates a frictionless, personal, and collaborative user experience with an integrated semantic layer. With Dremio Spaces, data teams can deliver a consistent and secure view of the data to data consumers, and analysts can curate new datasets in a self-service manner.
- Database-style updates and deletes (SQL DML support), powered by Apache Iceberg, enable data warehousing workloads directly on the lake.
- Dremio SQL Runner and SQL Profiler are new visual tools that enable SQL-proficient users to run and troubleshoot queries and scripts directly from the Dremio interface.
- Dremio Database Connectors allow for joins across multiple data sources (Oracle, SQL Server, Redshift, etc.) beyond just the data lake.
- Native third-party integrations further simplify the experience with Dremio connectivity now built into many popular client tools, including Power BI, Tableau, dbt, Hex, Preset and many others.
Dremio Arctic Delivers a Git-Like Experience for Data
Dremio Arctic is a forever-free service that delivers a Git-like experience for the data lakehouse, powered by the open source projects Iceberg and Nessie. Dremio Arctic’s Nessie-based metastore simplifies data engineering and analysis by making data workflows as powerful and intuitive as working with source code in the following ways:
- Branches allow users to ingest, transform and experiment in isolation, ensuring consistency across tables in the lakehouse.
- Tags and Time Travel allows data analysts and data scientists to reproduce historic dashboards or machine learning models without maintaining data copies.
- A simple user interface similar to GitHub provides observability and governance for data.
- Table optimizations of Apache Iceberg tables are automatically scheduled and executed in the company’s Amazon Web Services (AWS) account, eliminating the need to manually optimize and maintain the data.
Dremio Arctic is now in Public Preview with support for a wide variety of lakehouse engines, including Spark, Flink, Presto, Trino, and Dremio Sonar.
Quotes
“At Garvis, we’ve been a cloud-native company from day one, so we want to build an open data stack that enables us to scale quickly to support our customers. Dremio Cloud is a great solution for us as it lets us focus on our core expertise, while getting access to the latest Dremio features without any administration or IT operations management on our side. I was very impressed with the quick setup — we were up and running within 30 minutes and had migrated our existing Preset dashboards over to Dremio Cloud soon after. We are confident that this is a solution that scales along with our organization as we grow quickly, in terms of engine scalability, data lineage and governance, and enterprise-grade security,” said Geert-Jan Van Den Bogaerde, Chief Technology Officer at Garvis.
“Amazon S3 offers industry-leading scalability, data availability, security and performance for some of the largest and most innovative companies around the world. With Dremio Cloud, a new cloud-scale service built on Amazon S3, customers can create open architecture data lakes based upon Apache Iceberg for self-service SQL analytics. I am excited at how this service will make it easier for customers to quickly query and analyze their data across multiple sources and at scale,” said Kevin Miller, Vice President and General Manager at AWS S3.
"Tableau is committed to making data and analytics technology more powerful, easier to use and integrated in the flow of business. The new single sign-on-enabled connectivity with Dremio Sonar and Arctic enables analysts to explore and visualize current, historical, and experimental data snapshots on cloud data lakes, without having to create and maintain countless copies of data, which will help surface insights faster,” said Brian Matsubara, Vice President, Global Technology Alliances at Tableau.
“Both Preset and Dremio believe in an open modern data stack and we are excited that our customers can now use Dremio Cloud to create interactive analytics that users trust and love,” said Maxime Beauchemin, CEO and Founder of Preset.
“Hex is built for telling stories with data, and we’re excited to partner with Dremio Cloud’s open Lakehouse platform to enable customers to do this at scale,” said Barry McCardel, Co-Founder and CEO of Hex.
To learn more about Dremio Cloud, please visit https://dremio.com/dremio-cloud-now, and to get started on Dremio Cloud, please visit https://app.dremio.cloud/signup.
About Dremio
Dremio is the lakehouse company. Hundreds of enterprises around the world, including brands like Allianz Global Investors, FactSet, Knauf Insulation, Nutanix and OTP Bank, use Dremio’s SQL engine to deliver mission-critical BI on the lake. As the original creator of Apache Arrow, Dremio is on a mission to reinvent SQL for data lakes and meet customers where they are in their cloud journey. Dremio was founded in 2015 and is headquartered in Santa Clara. To learn more, follow the company on GitHub, LinkedIn, Twitter, and Facebook, or visit www.dremio.com.
Media Contact
Gillian Roberts
(818) 395-2948
