The Results
The new system has been live since 2020 and is gaining more and more users. Currently, 600 are registered and 300 users are actively exploring the entire spectrum of DB Cargo data with the tools of their choice. Business units and IT benefit from the advantages, including:
Flexibility
The new system is not only much faster than the old data warehouses, but also more agile. This is an important plus for the IT team, because now data is processed only when needed, while Dremio ensures that it ends up directly in the user applications.
Efficiency
The cloud data lake promises unlimited storage at foreseeable costs and enabled the IT team to separate storage and compute. With Dremio, data no longer needs to be actively collected. Instead, it is practically delivered to the door via direct pipelines and native connectors, thus eliminating the formerly frequent access of the production system. This relieves the load on the system, reduces the risk of failures and saves costs.
Time to Value
Where is my data? The users had to spend 60% of their time on answering this question. When the data was finally found, more often than not a table was missing from the dataset. With Dremio this tedious and frustrating search has ended. Today, subject matter experts have all crucial information at their fingertips and can focus 100% on their analyses.
Speed
To be able to include the latest data in their data warehouses, the IT team had to move older data first. Anyone needing this historic data had to wait a week or more. This delay was particularly annoying for statistics, as all the data that was not available on the reference date became obsolete. Today, all data can be accessed without limitations. This is possible using Reflections, a unique Dremio feature. Reflections store datasets in already optimized formats. They eliminate the need to access source systems and provide immediate results.
Use Cases Operational Excellence in Transportation Planning
In the capacity control application, a shipment is planned from the initial enquiry to the delivery at the destination and the shipment quality is monitored. Every change to the plan, for example whether the contract has been signed or the shipment is already on its way, is logged in JSON.
The total data volume is 10 terabytes per year. And with Dremio's virtual datasets up to 200 gigabytes of JSON data per day can be conveniently analyzed any time in near-real time in tools like Qlik. So the analyst can identify a need for action, for example when a route is frequently blocked during storms due to fallen branches.
Successfully Combating Noise with IoT Sensor Data
As the use of particularly noisy freight wagons was banned in Germany on December 13, 2020, measuring and evaluating sensor data and making the appropriate constructional changes allowed DB Cargo to increase the number of quiet freight wagons to 90 percent. These wagons are 10 decibels quieter, which is a perceived halving of the noise. Dremio played the key role in delivering these 50 terabytes of IoT data.
