4 minute read · May 21, 2021

Are you ready for data democratization?

Deepa Sankar · Vice President, Portfolio Marketing, Dremio

Organizations are tapping into data and analytics more than ever now. As they are modernizing their data platforms and adopting cloud, they are looking into various options - building data warehouses, data lakes and data hubs. Recent report from Mckinsey states that managing data is not cheap. A big chunk of money is spent on data sourcing, architecture, governance, and consumption. Fragmented data repositories can eat up to 20 percent of the average IT budget. And building a data warehouse will not solve data democratization problems. In order to maximize the value of data, organizations need to revisit their data architecture. An open, modern cloud data architecture with self-service access to data is critical for success. So what is an open data architecture? Data is no longer available in relational format. They are stored in formats such as JSON, Parquet, CSV today. To integrate this data, organizations need modern object storage such as AWS S3, ADLS or Scality and Dell EMC ECS, making it easier to store in a data lake. The bigger question is how do you access that data with everyday BI and analytics tools such as Tableau and Power BI? A high performance SQL query engine such as Dremio with a business friendly semantic layer and governance will help organizations to maximize the value of data. In a nutshell, an open data architecture will offer the flexibility to use best-of-breed engines and data formats in multi-cloud environments. This will help you to address current and future needs without any vendor lock-in.

Here are some examples of organizations who achieved success with a modern, open data architecture. Henkel - Henkel’s Laundry & Home Care division generates massive datasets in its supply chain for demand planning, forecasting, supply network planning, production scheduling, manufacturing and logistics for 33 production plants, 70 contract manufacturers and 60 warehouses around the world. They built a modern data platform in the cloud with Microsoft ADLS, Databricks, Dremio and Tableau. Henkel reduced its query time 30x, from 3-4 minutes to 8 seconds. The new architecture helped them to break data silos, accelerate business insights and improve productivity. NewDay - NewDay is one of the UK’s leading consumer credit finance companies, with over five million customers and receivables of over £3 billion. As part of digital transformation effort, they decided to modernize their legacy system to support BI, advanced analytics and data science capabilities. They built a cloud-based AWS data platform utilizing a data lake centralized on Amazon S3 and Dremio as their high performance query engine. Now their business users and data scientists have self-service access to data on S3 and the queries return results in seconds versus hours. NCR - NCR Corporation is a leading software- and services-led enterprise provider in the financial, retail, hospitality, telecom and technology industries. NCR generates massive amounts of data every day—sales, device service data, IoT status, transactional data, shipping and supply chain data, and much more. For many years, NCR stored a majority of their datasets in a data warehouse system and they needed a long-term cost-effective, scalable open data architecture to support all types of analytics. Using a modern architecture with Hadoop, Dremio and Tableau, they were able to shorten development time of data pipelines, accelerate queries by 30X and deliver results with complex queries in 30 seconds. AP Intego - AP Intego is a leading digital insurance agency and one of the nation’s fastest-growing insurtech companies whose goal is to simplify the purchase, use and renewal of insurance for small business owners. Their sales and service reps needed a full view of customer information to proactively provide service recommendations. However data was spread across Parquet, CSV, Salesforce, Zendesk and Applied Epic. Using a cloud data lake architecture on AWS S3 with Dremio, business users interactively explore their 10 major data sources on an ad hoc basis using the Dremio web interface, BI platforms and data science tools. Let me know how you plan to modernize your data architecture. Connect with me on LinkedIn and share your story.

Article Topics

Open Data Architecture