ADLS Fileset Catalog

Explanation (Core Concepts)

What is an ADLS Fileset Catalog?

An ADLS fileset catalog registers an Azure Data Lake Storage Gen2 location in xStore as a governed fileset catalog. After onboarding, you create schemas and filesets that point at paths under the container, and linked compute engines read/write them. This guide covers the ADLS-specific backend and permissions; the shared steps live in Fileset Catalogs.

This guide is written for the Platform Administrator. It assumes a running xStore (xstore-demo-doc-cluster) with a metalake (demometalake).

Info

Security note: All screenshots mask the credentials — the client ID, client secret, and tenant ID are masked. The storage account, container, and path are not secrets and are shown.

Key Concepts

Info

New to xStore catalogs? See Fileset Catalogs — Core Concepts for the shared model and the create-schema / create-fileset steps.

Concepts specific to ADLS:

Location: ADLS Gen2 path, abfss://container@account.dfs.core.windows.net/path.
Authentication (one of): Account Key, SAS token, or Service Principal (Azure AD: Client ID + Client Secret + Tenant ID). This guide uses a service principal.
Storage Account Name: the Azure storage account (without .dfs.core.windows.net).

Tutorial (Getting Started)

Prerequisites

A running xStore cluster and a metalake.
An ADLS Gen2 account + container, and one of: account key, SAS token, or an Entra ID service principal (Client ID, Client Secret, Tenant ID).
Administrative permissions on the xDP platform.

Minimum ADLS permissions

Grant the storage principal an Azure RBAC role on the storage account/container (and, because compute queries run as the per-user identity, the principal those users map to). Reference: Azure — Assign a role for blob data access.

Read-only: Storage Blob Data Reader.
Read-write: Storage Blob Data Contributor.

# Read-only (use "Storage Blob Data Contributor" for read-write) az role assignment create --role "Storage Blob Data Reader" \ --assignee <client-id-or-principal> \ --scope "/subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Storage/storageAccounts/<account>/blobServices/default/containers/<container>"

Steps

Follow the shared Create a Fileset Catalog wizard. Select Fileset → Azure Data Lake as the provider:

At the Backend step, enter the location and service-principal credentials:

Location: abfss://<container>@<account>.dfs.core.windows.net/<path>
Storage Account Name: <account>
Client ID / Client Secret / Tenant ID (masked below).

Backend — ADLS service principal (credentials masked)

Review the summary (credentials masked) and click Create Catalog:

The catalog is created as an adls / fileset catalog with status In use:

ADLS fileset catalog detail (credentials masked)

Then create a schema and filesets as described in Fileset Catalogs.

Reference

Backend fields

Field	Required	Description
Location	Yes	`abfss://container@account.dfs.core.windows.net/path`.
Storage Account Name	Yes	Azure storage account (without `.dfs.core.windows.net`).
Authentication	Yes	Account Key, SAS token, or Service Principal.
Client ID / Client Secret / Tenant ID	Service principal	Azure AD app credentials (stored as secrets; masked in the UI).

Minimum permissions

Mode	Azure role
Read-only	Storage Blob Data Reader
Read-write	Storage Blob Data Contributor

Best Practices

Use a service principal scoped (via RBAC) to the specific container, not the whole subscription.
Rotate the client secret periodically and update the catalog.
Protect credentials — xDP stores them as secrets and masks them in the UI.

What's Next

Create schemas and filesets — see Fileset Catalogs.
Link an xCompute cluster so Spark/Trino can read/write the filesets.

Last updated on

Was this page helpful?