Using the Dremio MCP Server with any LLM Model

If you’ve explored the Model Context Protocol (MCP) ecosystem, you’ve probably seen Claude Desktop in action. It’s a slick, intuitive MCP client, but it’s also the only commercial client widely available right now. That’s great if you’re all-in on Claude... but what if you want to tap into the power of other LLMs like OpenAI’s GPT-4, Mistral, or Anthropic’s Claude through the CLI?

That’s precisely the question I set out to answer. And the result is the Universal MCP Chat Client, a LangChain-based, terminal-friendly, model-agnostic chat tool that brings agentic AI workflows right to your command line. Whether you’re a data engineer, ML tinkerer, or just LLM-curious, this tool helps you plug any LangChain-compatible model into a multi-tool MCP environment.

In this tutorial, we’ll walk through how to use this client and hook it up to the Dremio MCP Server, a server designed to give LLMs intelligent access to Dremio’s powerful lakehouse platform. By the end, you’ll be chatting with your data using the LLM of your choice, all from your terminal.

If you want to try this out with a copy of Dremio Community Edition running on your laptop for free, we will guide you on how to set that up in seconds with Docker at the end of this blog.

Let’s get you set up.

What’s MCP Anyway? Think of It Like “Tool Time” for LLMs

Before we dive into the code, let’s take a moment to talk about what makes MCP such a game-changer.

MCP (Model Context Protocol) is a framework that gives large language models structured access to external tools. Think of it like plugging your LLM into a power strip of utilities—whether it’s querying a database, running a weather check, or doing math, MCP helps the model “know what tools it has” and “when to use them.”

Here’s the vibe:

• The LLM is the brain—it interprets your questions.
• The MCP Server is the toolbox—it knows how to perform specific tasks.
• The MCP Client is the bridge—it connects your brain to the tools.

With traditional setups like Claude Desktop, that bridge is tightly coupled to a specific LLM. But with the Universal MCP Chat Client, you can swap out the brain, GPT, Claude, Gemini, Cohere, you name it, and still connect to the same tool ecosystem.

And when one of those tools is the Dremio MCP Server, suddenly your model isn’t just smart—it’s data-smart. That means real-time insights, metadata exploration, and context-aware queries against your Dremio environment.

Next, we’ll prepare your environment so you can try it out for yourself.

Getting Set Up: Your Local AI Toolkit

Alright, let’s roll up our sleeves and get your local environment ready to run the Universal MCP Chat Client with the Dremio MCP Server. This setup gives you a chat-based interface powered by your LLM of choice, right from the terminal.

✅ Prerequisites

Before we start, make sure you’ve got the following installed:

• Python 3.11+
• uv package manager – Required for running the Dremio MCP Server
• Docker – (Optional) For running Dremio OSS locally
• An API key for your preferred LLM (OpenAI, Claude, etc.)

1. Clone the Universal MCP Client

Let’s start by grabbing the client code:

git clone https://github.com/AlexMercedCoder/langchain-mcp-client
cd langchain-mcp-client

2. Install the Dependencies

I recommend using a virtual environment for isolation:

pip install -r requirements.txt

Or manually:

pip install python-dotenv langchain langgraph langchain-mcp-adapters fastmcp openai rich langchain-openai

3. Configure Your Environment

Create a .env file in the project root with your preferred LLM settings. For example, if you're using OpenAI:

OPENAI_API_KEY=your-openai-api-key
LLM_MODEL=openai:gpt-4.1
DETAILED_OUTPUT=false

Want to use Claude, Gemini, Mistral, or another model? The client supports all LangChain-compatible models, just update the LLM_MODEL and API key accordingly (examples available in the clients documentation).

4. Start Thinking About Servers

You’ll be connecting the client to one or more MCP servers. We’ll configure that in a mcp_servers.json file soon, but first, let’s get the Dremio MCP server up and running.

Setting Up the Dremio MCP Server

Now that your client is ready to go, let’s get the other half of the equation running: the Dremio MCP Server. This server acts as a tool provider, giving your LLM access to query metadata, discover data patterns, and interact with Dremio—all through natural language.

1. Clone the Dremio MCP Server

Start by cloning the official Dremio MCP Server repo:

git clone https://github.com/dremio/dremio-mcp
cd dremio-mcp

Note: The server expects Python 3.11+ and uses the uv package manager, so make sure you’ve got both installed.

2. Create the Dremio Server Config

This step connects the MCP server to your Dremio instance via API. You’ll need your Dremio URI and a Personal Access Token (PAT).

If you're using Dremio Cloud, you can grab a PAT from the web UI. If you're running Dremio OSS locally, you'll need to generate one via the REST API.

You can use the dremio-simple-query Python library to get the token programmatically:

from dremio_simple_query.connect import get_token

payload = {
    "userName": "your-username",
    "password": "your-password"
}
token = get_token(uri="http://localhost:9047/apiv2/login", payload=payload)

print("------TOKEN BELOW-------")
print(token)
print("------TOKEN ABOVE-------")

dremio-simple-query is not required by the Dremio MCP Server or MCP Client, so you’ll need to install it separately if you want to use it for token generation:

pip install dremio-simple-query

Once you have your PAT, configure the server:

uv run dremio-mcp-server config create dremioai \
    --uri http://localhost:9047 \
    --pat <PAT_TOKEN>

3. Launch the MCP Server

Run the Dremio MCP Server with:

uv run dremio-mcp-server run

And just like that, the Dremio MCP Server is live and ready to accept connections.

Tip: The default config file lives in ~/.config/dremioai/config.yaml. You can override it with --config-file if needed.

🔗 Connecting the Client to Dremio MCP

Now that the Dremio MCP Server is up and humming, let’s connect the dots so your Universal MCP Chat Client can start sending it prompts.

This is where the mcp client's mcp_servers.json file comes into play. It tells the client which MCP servers to talk to, how to launch them, and what kind of transport to use (in our case, stdio for local communication).

1. Update mcp_servers.json

Inside your cloned langchain-mcp-client directory, open or create the mcp_servers.json file and add the following entry:

{
  "dremio": {
    "transport": "stdio",
    "command": "uv",
    "args": [
      "run",
      "--directory",
      "/absolute/path/to/dremio-mcp",  
      "dremio-mcp-server",
      "run"
    ]
  }
}

Important: Replace /absolute/path/to/dremio-mcp with the actual path where you cloned the Dremio MCP Server.

This tells the chat client to spin up the Dremio MCP Server locally using uv, and communicate with it via standard input/output streams.

2. Launch the Chat Client

From the root of the langchain-mcp-client directory, start the chat loop:

python client.py

You’ll be dropped into a clean terminal-based chat UI. From here, you can start querying your Dremio environment like so:

>>> What tables do I have access to?
>>> Show me insights from the sales.transactions table.
>>> What’s the row count in arctic.orders?

Behind the scenes, LangChain’s ReAct agent analyzes your prompt, chooses the appropriate tool on the Dremio MCP Server, and executes the task—all from the comfort of your terminal.

Under the Hood: How the Magic Happens

So, you’ve got a chat interface, a smart model, and a Dremio backend talking to each other—but how exactly does it all work? Let’s unpack what’s going on behind the scenes.

The LLM + LangChain Agent

When you run client.py, the Universal MCP Chat Client kicks off a LangChain-powered ReAct agent. This agent isn’t just a fancy prompt wrapper—it’s decision-making logic that can:

• Understand your natural language input
• Decide which tool (MCP server) can help
• Invoke that tool with the right parameters
• Return the output to you in the chat

ReAct = Reason + Act. It's how the agent figures out a plan before calling any tool.

The MCP Tool Server (Dremio MCP)

The Dremio MCP Server is a FastMCP server—a standardized way to expose tools to LLMs. Think of it like a command center for all things Dremio:

• List datasets
• Analyze schema patterns
• Run SQL (if enabled)
• Detect data anomalies

Each function on the server is a decorated Python method exposed as a tool. When your agent sends a request, the server handles it and returns a response—no need for the agent to know how Dremio works under the hood.

🪝 The Transport: Stdio

In this setup, the client launches the MCP server as a subprocess and communicates over stdio—plain old standard input and output. This keeps things local, simple, and portable.

You could also use streamable_http if you wanted to run the MCP server as a web service and connect remotely.

🧬 The Flow in Action

1. You type a question:
   "What columns are in the sales.transactions table?"
2. The LangChain agent parses the question.
3. It selects the dremio MCP server from mcp_servers.json.
4. It formats a tool call to the server, like:
   
   { "tool": "get_columns", "args": { "table": "sales.transactions" } }
   
5. The Dremio MCP server runs the logic, hits the Dremio API, and sends back the results.
6. The client displays it in your terminal, beautifully formatted.

Leveling Up: Custom Tools, More Models, and Full Flexibility

Now that your Dremio-powered LLM agent is up and running, let’s talk about where you can take this next. The real power of this setup is how modular and flexible it is, so let’s walk through some cool ways to customize and extend it.

Add Your Own Tools

Want to go beyond Dremio? You can easily plug in additional FastMCP tool servers alongside it. For example, let’s say you want to add a greeting tool:

# File: mcp_servers/my_tools_server.py
from fastmcp import FastMCP

mcp = FastMCP("MyTools")

@mcp.tool
def greet(name: str) -> str:
    return f"Hello, {name}!"

if __name__ == "__main__":
    mcp.run(transport="stdio")

Add it to your mcp_servers.json:

"mytools": {
  "command": "python",
  "args": ["./mcp_servers/my_tools_server.py"],
  "transport": "stdio"
}

Now your agent can greet users and query Dremio. Boom, multi-tool agent unlocked.

Swap In Different LLMs

Want to try Claude 3, Mistral, Gemini, or even open-weight models from Together or Fireworks? Just update the .env file with the appropriate LLM_MODEL and API key:

LLM_MODEL=anthropic:claude-3-sonnet-20240229
ANTHROPIC_API_KEY=your-api-key

No code changes. Just set, save, and go.

Enable Advanced Dremio Features

The Dremio MCP Server config file supports advanced options like:

• Enabling experimental features
• Allowing view creation (allow_dml: true)
• Using modes like FOR_SELF or FOR_PROMETHEUS for deeper introspection

These features open up powerful workflows like performance analysis and metric-based insights, all triggered through natural language.

Wrapping Up: Your AI-Driven Data Command Center Awaits

And there you have it—you’ve just connected a universal, LLM-agnostic chat client to the Dremio MCP Server, turning your terminal into an intelligent, tool-savvy interface for querying data, discovering patterns, and streamlining analytics workflows.

Whether you’re building internal tools, exploring data as an analyst, or just experimenting with agentic LLM workflows, this setup gives you flexibility, control, and a great developer experience.

Want to Try This Without a Dremio Account? Run Dremio Locally with Docker

You don’t need a Dremio Cloud account or on-prem install to get started—Dremio OSS runs beautifully in Docker.

Here’s how to spin it up:

docker run -p 9047:9047 -p 31010:31010 -p 45678:45678 -p 32010:32010 \
  -e DREMIO_JAVA_SERVER_EXTRA_OPTS=-Dpaths.dist=file:///opt/dremio/data/dist \
  --name try-dremio dremio/dremio-oss

This will:

• Start Dremio’s web UI on http://localhost:9047
• Expose Flight and REST ports so the MCP server and Arrow integrations work
• Keep your instance self-contained and disposable

To stop and clean up:

docker stop try-dremio
docker rm try-dremio

This makes it super easy to test out Dremio MCP workflows with your own data—or even just play with the built-in samples.