How To Share A Query Profile

   
  • Dremio

Table of Contents

Table of Contents

Intro

In this tutorial we’ll show you how to share a Dremio query profile. Query profiles store important metadata about queries that you run in Dremio, and can make it easier to help Dremio’s engineers debug any issues you encounter.

Prerequisites

You will require access to a Dremio deployment and have at least one data source connected. Alternatively, you can still follow the tutorial but you won’t be able to try out the steps yourself.

Dremio Jobs

In Dremio most work is managed as jobs, including all queries. Dremio keeps a history of all jobs so you can easily take a closer look at how a job was executed. This is where you’ll find information about who submitted the query, how long it ran, the number of results, the SQL that was submitted, and comprehensive profile information.

Let’s look at an example. First, open one of your datasets. Here we’ll use the SF_Incidents2016.json physical dataset from Working With Your First Dataset:

How to share a query profile - Dremio

To preview this dataset, Dremio executed a preview query against the JSON file on Amazon S3. We can now click on Jobs to see the history of jobs in Dremio:

How to share a query profile - Dremio

Now we see a list of jobs on the left, and a detailed view of the selected job on the right:

How to share a query profile - Dremio

Finding Your Job

In this case the first job is the one we just ran. However, in most cases the job you’re looking for may not be at the top of the list. Dremio makes it easy to find a job by using the filters that let you narrow jobs down by time, job status, and which process or user created the job:

How to share a query profile - Dremio

You can also search based on text included in the job, such as a column or table name you used in a query:

How to share a query profile - Dremio

Downloading Your Profile

Once you’ve identified the job you want to investigate, select it on the left in order to view it on the right. Let’s look at the job we issued on the police incidents data:

How to share a query profile - Dremio

Let’s take a look at how to share this profile with others. At the bottom of the job details you can see a friendly narwhal with a button to “Download Profile.” If you click this button, detailed profile information will be downloaded as a zip file.

You can now share this zip file with a Dremio engineer to help understand any issues you may be encountering. Note that none of your data is stored in the profile, only metadata about your query and underlying datasets.

Ready to Get Started? Here Are Some Resources to Help

Webinars

Automating the Open Data Lakehouse Using an Open Intelligent Metastore

This webinar focuses on how coupling governance with an intelligent metastore can transform a data lake into a data lakehouse that can support the organization’s data workflows and analytics applications.

read more

Analyst Report

DZone Data Pipelines Trend Report

DZone has released their Data Pipelines Trend Report. It is a survey of software developers, architects, platform engineers, and IT professionals to understand the challenges and potential solutions around ingesting, processing, and leveraging data.

read more

Webinars

How Enel Group Built a Data Mesh Architecture with Dremio and AgileLab

In this webinar, learn how Enel Group worked with Agile Lab to implement Dremio as a data mesh solution for providing broad access to a unified view of their data, and how they use that architecture to enable a multitude of use cases.

read more

Get Started Free

No time limit - totally free - just the way you like it.

Sign Up Now

Watch Demo

Not ready to get started today? See the platform in action.

Check Out Demo