The Dremio Blog

As we conclude, remember that the world of data is ever-evolving. The combination of Dremio and dbt isn’t just a solution; it's a continuously advancing pathway to data excellence, unlocking potential and opportunities for businesses ready to embrace the future of data management.

Dremio offers a robust platform for creating data products by simplifying data integration, providing a semantic layer for data curation, and enabling secure data sharing. Whether you're curating data for a single product or managing multiple data products, Dremio's features can streamline the process and enhance collaboration among data professionals, ultimately leading to the successful creation of valuable data products.

Dremio stands as a formidable solution to the pervasive challenge of data silos. Unifying disparate data sources enables organizations to leverage their data assets fully, enhancing decision-making and operational efficiency. As the data landscape evolves, tools like Dremio will be critical in shaping a more integrated and insightful approach to data analytics.

Whether through direct PyArrow library usage or leveraging the dremio-simple-query library for simplified querying and data manipulation, the synergy of these tools opens up new possibilities for data analysis and processing. The ability to convert data streams into different formats ensures compatibility with a wide array of data processing and analytics tools, making this approach highly versatile.

With the release of Dremio Software Enterprise Edition 24.3, we’ve added Kubernetes autoscaling to Dremio. Kubernetes autoscaling for Dremio streamlines resource management with an emphasis on memory and CPU utilization in Dremio workloads.  Now, Dremio on Kubernetes scales automatically, reducing time spent in administration and R&D to forensically size clusters. Understanding Kubernetes Autoscaling Kubernetes autoscaling […]

this is a product release announcement regarding new ingestion capabilities for Apache Iceberg. Customers can now use COPY INTO to get data in parquet format into Iceberg tables.

Dremio's Data and Analytics Predictions for 2024: data lakehouse adoption, the rise of Apache Iceberg, DataOps, Data Mesh and Generative AI

The demand for insightful and high-performance dashboards has never been greater. As organizations accumulate vast amounts of data, the challenge lies in visualizing this data efficiently, especially when dealing with large datasets. In this article, we will delve into the realm of BI dashboards, exploring the hurdles that hinder their performance for sizable datasets. Traditionally, […]

This month, we’re excited to announce automated table cleanup!

Dremio has released a new version of the Dremio vectorized Parquet reader that will improve query performance on Parquet datasets encoded with the Parquet V2 encodings by up to 75%

The concept of data marts has long been a pivotal strategy for organizations seeking to provide specialized access to critical data for their business units. Traditionally, data marts were seen as satellite databases, each carved out from the central data warehouse and designed to serve specific departments or teams. These data marts played a vital […]

Unveiling Dremio Cloud on Azure, a Fast, Scalable and Secure Lakehouse Platform Introduction Today we introduce Dremio Cloud on Azure, newly landing on Azure in November 2023, in public preview. Dremio Cloud is a powerful lakehouse platform providing streamlined self-service, unparalleled SQL performance, centralized data governance and seamless lakehouse management. In this blog post, we […]

The Dremio Unified Analytics Platform brings your users closer to the data with lakehouse flexibility, scalability and performance at a fraction of the cost.  We're excited to show you some new things we've been working on to help you analyze your data faster and easier than ever, no matter where it's stored. We’ve launched AI-powered […]

This blog helps you learn about array functions in Dremio Cloud and Dremio Software v24.3+.

The Databricks platform is widely used for extract, transform, and load (ETL), machine learning, and data science. When using Databricks, it's essential to save your data in a format compatible with the Databricks File System (DBFS). This ensures that either the Databricks Spark or Databricks Photon engines can access it. Delta Lake is designed for […]