The Results
With Dremio, anyone in the company can now access energy data and calculations across tens of thousands of meters in a matter of seconds. Dremio has helped unleash analyst productivity and creativity. With Dremio’s semantic layer, analysts can easily combine data in new and interesting ways to gain business insights without requiring the assistance of data engineers.
More importantly, Dremio is helping build a data foundation that can scale and serve as a platform for future innovation. By helping address a variety of technical and operational challenges related to data management, Dremio is helping position the company to expand into new markets and pursue additional opportunities.
Self-Service Access For Analysts
With Dremio, analyst productivity has exploded. Leap estimates an overall productivity gain of more than 30%. Terabyte-scale queries that previously took minutes now return in seconds - a game-changer for busy analysts that spend their entire day responding to partner inquiries and developing predictive models.
“We had underestimated how much data productivity would take off,” says Marco Rietveld, lead data engineer at Leap. “Analysts are now able to easily derive and share new data views in Dremio’s semantic layer, organize data into folders, and easily answer business-level questions without the need for custom scripts or coding.”
Improved Data Governance And Regulatory Compliance
Within weeks of deploying Dremio for self-service data access, usage grew rapidly, and analysts began developing and sharing derived data views. Dremio gave analysts the tools they need to document and logically organize datasets in shared spaces with appropriate security controls. “It’s hard to anticipate problems that you don’t know about until you run into them,” says Marco Rietveld. “It was nice that Dremio had already thought about these problems and baked data governance functionality into the product.”
The energy sector is highly regulated. Data needs to be 100% correct, and regulators may challenge providers at any time to demonstrate how they arrived at decisions. Compliance requires easy access to prior datasets and calculation reproducibility. Dremio provides the capabilities needed to help ensure compliance, including access controls, auditability, and data lineage tracking.
Reduced Data Engineering Workload
Before deploying Dremio, analysts relied on data engineers to develop custom PySpark scripts to extract datasets in CSV format from raw meter data in S3. Whenever analysts required access to new data, data engineers needed to develop, test and deploy new PySpark pipelines leading to a backlog of requests. By exposing data with Dremio, analysts can directly query Parquet files on S3, PostgreSQL, and other sources. They can also join data from multiple sources using simple SQL queries.
Not only does this improve analyst productivity, but it dramatically reduces data engineering workload. Reducing the data engineering backlog increases business agility and helps Leap respond faster to changing market conditions and competitive pressures.
More Efficient Use Of Cloud Infrastructure
Unlike some query tools that require the continuous deployment of expensive cloud infrastructure, Dremio supports Elastic engines enabling organizations to right-size cloud resources for each distinct workload. Rather than relying on a “one-size fits all” model, Dremio can schedule execution engines to run independently at different times and automatically start and stop engines based on workload requirements at runtime. This allows organizations to isolate workloads and maintain service levels while dramatically reducing cloud infrastructure spending.
Faster, Higher-Quality Decisions For Improved Competitiveness
Dremio frees analysts and data scientists to focus on the business. It provides a way for anyone in the company to access energy data and calculations on tens of thousands of meters within seconds. With easy access to data, analysts can respond faster to customer inquiries and quickly detect and resolve anomalies in data. With Dremio, productivity has been unleashed. Analysts can now spend more of their day analyzing and understanding energy usage patterns and building improved predictive models – activities that are critical to optimizing operations and boosting revenue and profitability.
