Data engineering teams are often defined by the gap between a business question and a verified answer. Requests languish in backlogs while engineering teams struggle with ETL pipelines and the technical debt of siloed warehouses. When reports finally arrive, the data is frequently stale, leading to debating the accuracy of the numbers rather than the […]
Last week I published a blog about some of the risks to consider when implementing Agentic AI in your workflows or organisation. But what about the other side of that coin, the benefits this revolutionary technology can bring about? Read on to learn several of the ways that Agentic AI is transforming how we work […]
Agentic AI offers several compelling benefits that are transforming how we work and solve problems. The key difference from traditional AI is agency, the agent’s ability to plan, use tools, take actions in sequence, and work toward goals rather than just responding to individual prompts. There are many upsides to embracing AI agents in your […]
Large Language Models (LLMs) are capable of understanding and generating language. However, they do not understand or process language in the same way that a person like you or I does. When reading text, humans construct meaning by processing the syntax of each sentence as it unfolds. This involves combining word definitions, using context from […]
Dremio has introduced several capabilities that inteliigently improve query performance across the data lakehouse. With minimal to no action from users, Dremio will reduce query latency, handle data maintenance tasks, and eliminate redundant compute jobs. This article is a summary of three of these performance management features. Read on to learn how reflections accelerate popular […]
Dremio is always striving to abstract away the physical concerns of data, whether the storage location, partitioning schema, or file size optimisation. Thanks to features such as Data Federation, Iceberg Clustering, and Autonomous Performance functionalities, Dremio users get highly-performant access to their data no matter where it lives. One of the components that delivers this […]
Caching dramatically reduces latency and computational costs by storing frequently accessed data closer to where it’s needed. Instead of repeated expensive operations – such as fetching from object storage, planning complex queries, or executing SQL – the data you need is provided in fast, local memory. To deliver on this, Dremio implements different layers of […]
Agentic AI is an artificial intelligence system that is designed to operate autonomously. With minimal human supervision it can be expected to make decisions and perform tasks with specifically trained agents. This is thanks in large part to Large Language Models (LLMs) which provide agentic AI with enhanced reasoning and the ability to understand context. […]
Reflections are a query acceleration functionality unique to Dremio, that work by minimising data processing times and reducing computational workloads. Debuting in the early days of Dremio, Reflections accelerate data lake queries by creating optimised Apache Iceberg data structures from file-based datasets, delivering orders-of-magnitude performance improvements. However, the game-changing aspect of this technology was not […]
A promise of self-service data platforms, such as the Data Lakehouse, is to democratise data. The idea is that they empower business users (BUs), those with little or no technical expertise, to access, prep, and analyse data for themselves. With the right platform and tools your subject matter experts can take work away from your […]
At the time of writing this blog, pulsemcp has over 5,000 MCP servers registered and available to download. These servers provide LLMs the tools to control diverse software and applications – from online shopping and web scrapers to music streaming, cloud infrastructure, and image generation. The list goes on and on. MCP is evidently finding […]
Back in April of this year Dremio debuted its own MCP server, giving the LLM of your choice intelligent access to Dremio’s powerful lakehouse platform. With the Dremio MCP Server the LLM knows how to interact with Dremio; facilitating authentication, executing requests against the Dremio environment, and returning results to the LLM. The intention is […]
Join the Dremio/dbt community by joining the dbt slack community and joining the #db-dremio channel to meet other dremio-dbt users and seek support. The data management strategy of the data lakehouse is to combine the key strengths of data lakes and data warehouses into one, unified platform. From data lakes it utilises low-cost and scalable […]
Join the Dremio/dbt community by joining the dbt slack community and joining the #db-dremio channel to meet other dremio-dbt users and seek support. Documentation is a critical component of any product, from food to furniture to software. Whether you are looking at a t-shirt or a supermarket chicken it will come with something explaining what […]
Join the Dremio/dbt community by joining the dbt slack community and joining the #db-dremio channel to meet other dremio-dbt users and seek support. A major data trend in 2024, and carrying on into this year, was the move to self-service analytics. According to our recent survey of IT and data decision-makers, 80% of companies we […]
Join the Dremio/dbt community by joining the dbt slack community and joining the #db-dremio channel to meet other dremio-dbt users and seek support. The boom in AI witnessed both in the performance of available models and in the appetite of businesses in utilising them has been unprecedented in the last couple of years. There are […]
