11 minute read · October 15, 2021

Open Data Architecture: Companies Accelerate Queries and Time to Insight by 30x

Preeti Kodikal · Product Marketing Director, Dremio

Data and technology leaders embrace modern approaches to data architecture to better position their companies to be agile, resilient and competitive. Have you read what Dremio founder, Tomer Shiran has to say about the 3 Key Trends Shaping the Future of Data Infrastructure in the world of analytics today? Organizations have realized that an open, modern cloud data architecture is the way forward for success, and helps companies be ready for current and future innovations. This blog post shares how organizations have leveraged Dremio to drastically simplify their architectures and data engineering and reduce their query time and time to insight from days and weeks to seconds! You may be wondering:

Why is Dremio getting so much attention in the market and with investors?
Why are data engineers, data architects, data scientists and data analysts all over the world talking about and using Dremio?
Why are Dremio technologies — for example, Apache Arrow, which was shared with the entire open source community by Dremio — being downloaded 20 million times per month and used by every data scientist in the world?

The answer: Because Dremio is not just building a “better mousetrap,” according to Dremio Chief Revenue Officer Alan Gibson in his blog! Specifically, in the world of data analytics, we are not just taking a “lipstick on the pig” approach to making an age-old data warehouse architecture look “modern” by simply dumping the old technology on the cloud. Instead, Dremio is redefining the data analytics industry! We’re redefining the way you build and use your data analytics architecture — it’s a whole new architecture and paradigm in analytics! Not just data democratization! Not just getting rid of data silos! Yes to ALL of the above, of course! So, why is Dremio different from others? The key differentiators of Dremio from traditional data warehouses and “lipstick on the pig” cloud warehousing architectures include:

Dremio enables you to enjoy high-performing BI queries and dashboards directly on your cloud data lake storage — no copying or moving of data, no multiple data copies all over your architecture, and no more BI extracts or cubes. This is revolutionary. Imagine completely eliminating expensive and unnecessary data copies altogether!
Dremio provides out-of-the-box integrations with Tableau, Power BI and other BI tools that you are already comfortable with — no need to install or learn any new BI tools!
Unlike data warehouse vendors, which require you to load data into their vendor-owned storage as a prerequisite to using their SaaS offering, Dremio Cloud, our fully managed SQL lakehouse platform built on an open architecture, allows you to keep the data in your own cloud account. Dremio provides lightning-fast query performance directly on data lake and lakehouse storage and connects to S3, ADLS, Hadoop or wherever your data resides.
The Dremio Query Engine comprises Columnar Cloud Cache (C3) and data reflections, which work alongside Apache Arrow and Arrow Flight to make queries on your data lake storage very, very fast.
Dremio’s shared semantic layer provides consistent semantics across all users, applications and tools, and empowers self-service data access for data analysts and data scientists while centralizing security and governance.
Dremio is designed from the ground up to deliver industry-leading, enterprise-grade infrastructure security and is purpose-built to provide the industry’s highest levels of security with multiple layers of protection; data is encrypted both at rest and on the wire.
- Dremio’s multi-engine, multi-cluster data plane architecture consists of multiple right-sized engines to support different workloads in the company. For example, a company can have a medium engine for executive dashboards, a large engine for batch jobs and an extra-large engine for data science queries.
- Dremio Cloud introduces the capability for engines to have multiple replicas, where engine replicas are added and removed dynamically based on the workload. This helps companies tackle any level of concurrency while maintaining consistent performance.

Fortunately, Dremio’s modern data architecture makes it possible to query data lakes directly at interactive speed — at the speed of business — avoiding the need for a data warehouse and redundant data copies. Learn how the Dremio SQL lakehouse platform helps you break free from traditional, closed data warehouse vendors and liberates your organizations from proprietary vendor lock-in, empowers your teams with accelerated time to value and enables you to put your data in the hands of more people, faster, with self-service access and high-levels of enterprise-grade security. You can read this datasheet about how Dremio Cloud Service, our frictionless, infinitely scalable, fully managed SaaS solution, and our industry-defining innovations built on an open data architecture help you simplify your data engineering while giving you full control of your data and enterprise-grade security. In the words of a Dremio customer: “We have to constantly size capacity for peak loads and run our engines 24x7. Auto-scaling from Dremio helps us manage this problem.” Several customers worldwide have shortened their time to insight from days or weeks to minutes and seconds with Dremio’s simplified, no-copy open data architecture, high-performance BI dashboards and interactive analytics directly on the cloud data lake storage. Here are a few examples of how Dremio customers have simplified data engineering and reduced their query time on cloud data lake storage up to 30x, from minutes to seconds!

Vitesco, the former powertrain division of Continental, is a leading global developer and manufacturer of modern powertrain technologies for sustainable mobility and is a leader in the electrification of vehicles. Because of the nature of their business, Vitesco needed a flexible data analytics platform because their data model is dynamic, as they continually add new machines or measurements. Read the Vitesco case study to learn about how Dremio enabled self-service data access directly on their data, which not only helped them eliminate the high costs associated with expensive external consultants, but also accelerated their time to insights without waiting for external support for data access. Dremio’s platform, built with open architecture, enabled Vitesco’s analysts and engineers to work with their preferred data analytics tools such as: Excel, KNIME, MATLAB, MicroStrategy, Minitab, Power BI, Python, R, Tableau, etc. In spite of strict regulations around data, Vitesco significantly improved their security and governance with Dremio’s semantic layer. Dremio has not only expanded the data access and visibility of their engineers and analysts, but Vitesco is also more equipped to scale up capacity when needed because of Dremio’s elastic engines. These efficiencies have freed up more resources to focus on innovation and accelerate Vitesco’s path to clean mobility.
NewDay is one of the UK’s leading consumer credit finance companies, with over five million customers and receivables of over £3 billion. With Dremio, NewDay was able to improve productivity for business users and data scientists through self-service access to data on S3 and their query times reduced from minutes or hours to seconds! NewDay also realized significant financial savings through resulting process efficiencies and by removing the need to pay third parties to host their older data warehouse solutions. Read the NewDay case study to learn how they improved productivity through self-service access to data, highly accelerated query speeds and time to insight, achieved process efficiencies, enhanced governance and security and enabled more advanced data analytics capabilities with Dremio!
Henkel, an international chemical and consumer goods company, headquartered in Germany, uses Dremio to not only join silos of data, but also accelerate business insights and query speeds from several minutes to 8 seconds! Read the Henkel case study to see how Henkel saved millions through productivity improvements in their global supply chain with Dremio!
OTP Bank and its subsidiaries offer banking services to more than 16 million enterprises, retail, commercial and government customers in 11 countries and is one of the leading banking groups in central and Eastern Europe. OTP Bank’s Big Data Team Lead, Lotar Schin shares: “We did not want a vendor that would lock us in. We thought the Apache Arrow-based solution would bring big data closer to the business and give us better access to it.” OTP Bank’s primary driver for using Dremio was to get better insights into activity happening across the bank, and they realized several efficiencies and savings with immediate access to data in multiple systems without having to make requests to IT for data access. Read the OTP Bank case study to find out how they not only got faster access to data, but also managed to significantly decrease the complexity of their data architecture, separate storage from reporting, free themselves from vendor lock-in and increase IT productivity and focus on more high-value projects.
AP Intego is a leading digital insurance agency and one of the nation’s fastest-growing insurtech companies whose goal is to simplify the purchase, use and renewal of insurance for small business owners. AP Intego wanted a solution that would give them a 360-degree customer view, enable queries across all their data from multiple data sources, with consistently fast performance at a reasonable cost, and the ability to integrate data operations and engineering to build new applications and functionality. Read the AP Intego case study to understand how they achieved sub-second query performance across multiple data sources and systems. Learn how they not only achieved high service levels and efficiencies in their business through faster insights, but are also exploring new use cases and expanding their business because of the flexibility and openness of the Dremio platform!
NCR Corporation is a leading software- and services-led enterprise provider for POS solutions in the financial, retail, hospitality, telecom and technology industries, and does business in 180 countries. NCR creates software, hardware and services including self-service kiosks, POS terminals, automated teller machines, check processing systems, barcode scanners and business consumables. With Dremio, NCR was able to accelerate queries 10-30x and allow self-service data access to users directly through Dremio’s semantic layer on top of the data lake. As a result, they reduced development time for reports and dashboards from months to days.

Would you like to achieve the same levels of query acceleration, time to insight and savings as the companies above? You can get started with Dremio Cloud today! What is your plan to modernize your data architecture and make your company more agile, resilient and competitive? Connect with me on LinkedIn or email me and share your story.

Article Topics

Open Data Architecture