Databricks Cost

Databricks Cost breaks down what you're spending on Databricks and the underlying cloud vendor, so you can see where costs are concentrated and act on it.

Filters

The Data Source Filter lets you switch between connected Databricks accounts or projects.

Overview

Overall Summary shows three totals for the selected time period:

• Total Cost — Databricks cost plus vendor cost.

• Databricks Cost — cost from Databricks resources.

• Cloud Provider Cost — cost from vendor resources.

Overall Cost Trend plots Databricks Cost, Cloud Vendor Cost, and Total Spend over the selected time period, so you can spot when costs spiked or dropped and correlate it with a specific date.

Cost by Cluster Type

This section breaks Databricks cost down by cluster type.

Cost Over Time shows daily cost trends for four cluster types: All Purpose, DLT, Job Cluster, and Others.

Cluster Cost by Type is a donut chart summarizing total cost per cluster type, color-matched to the Cost Over Time lines, showing at a glance which type contributes most to spend.

Top 5 Cluster Cost lists the five most expensive clusters in the selected period, labeled by Cluster ID, so you know exactly where to focus optimization efforts.

Cost by Vendor Service

This section breaks vendor cost down by service type: Bandwidth, Storage, Virtual Machines, and Virtual Network.

Cost Over Time shows daily cost trends for each service.

Cloud Vendor Cost is a donut chart summarizing total vendor cost by service type, color-matched to the Cost Over Time lines.

Top 5 Instance Cost lists the five most expensive instances by vendor cost, labeled by instance type.

Azure Databricks Cost Retrieval Methods

ADOC supports two ways to retrieve Databricks and cloud vendor cost data from Azure: the API Method and the Data Export Method.

API method

The API method queries Azure's Cost Management API directly, giving you cost data that matches the Azure Portal in near real time. It's the preferred method where available.

Set up a service principal in Azure, assign it the required roles, and configure ADOC to query the Cost Management API for both Databricks and cloud vendor costs.

Example request to fetch Databricks cost (replace all IDs, secrets, and tokens with your own — the values below are placeholders):

Fetching cloud vendor cost uses the same endpoint without the ServiceName filter, so results aren't restricted to Databricks.

Data export method

The Data Export method schedules a cost data export from Azure to Blob Storage, which ADOC then reads. It's useful for large volumes of data, though it can introduce minor data mismatches.

1. Set up the export for Databricks and cloud vendor costs:

   1. In the Azure Portal, go to the resource group where your Databricks instance is deployed.

   2. Navigate to Cost Management > Cost Analysis and select Download.

   3. In the side pane, select CSV as the format, then select View all exports.

   4. Select Create, choose Daily export for month-to-date costs, and confirm.

   5. Repeat for the Managed Resource Group to export cloud vendor costs.

2. Create a service principal to access the export:

   1. In Microsoft Entra ID > App Registrations, select New Registration and register the application.

   2. Under Certificates & Secrets, create a new client secret and store the value securely — it's shown only once.

   3. Assign the service principal Reader and Storage Blob Data Reader roles on the storage account holding the export files.

3. Configure ADOC to use the exported data:

   1. Navigate to Register and either add a new data source or select Edit Configuration on an existing one.

   2. Set Cost Fetch Method to Data Export method.

   3. Enter the Azure Tenant ID, Subscription ID, Client ID, and Client Secret for the service principal.

   4. Enter the Azure Resource Group and Managed Resource Group for the Databricks workspace.

   5. Save the configuration.

Known limitations

Additional resources

• Creating a Service Principal (Azure Documentation)

• Cost Analysis (Azure Documentation)

• Databricks API Documentation

• Cost Management Forecast REST API

What's next

• Databricks Compute – Cross-reference cost against cluster and job activity.

• Databricks Query Studio – Identify which specific queries are driving cost.