Spark Job Details

The Spark Job Details page shows all runs for a job alongside its full execution configuration. Use it to monitor job health, trigger manual runs, and drill into individual executions.

Navigating to Job Details

From the side navigation, go to Spark Jobs.
Click the name of the job you want to inspect.
The details page opens showing the Job Runs table and the Execution Configuration,Application Configuration,Image Configuration,Driver Configuration,Spark Configuration panel on the right.

Job Runs Table

The Job Runs table lists every past execution with:

Column	Description
Run ID	Unique identifier — click to open run details
Status	`Running`, `Success`, `Failed`, `Stopped`
Run Time	Total elapsed time
Triggered By	`Manual` or `Schedule`
Run As	User identity used for data access
Started	Timestamp of run start

Use the run activity chart above the table for a quick visual summary of run counts by date.

To trigger a manual run: Click the Run (play) icon in the top-right corner, then confirm.

Execution Configuration

The right-hand panel shows the complete configuration used for the job, grouped into sections:

Section	What it shows
Basic Configuration	Job Type, Spark Version, Batch Scheduler, deploy mode
Schedule Configuration	Cron expression and enabled/disabled status
Application Configuration	Main application file, Python version, arguments
Image Configuration	Container image path, pull policy
Driver Configuration	Cores, memory, labels
Executor Configuration	Cores, memory, dynamic allocation settings
Data Access	xStore Catalog / Metalake, or Data Store connections
Spark Configuration	Custom Spark property overrides

Last updated on

Was this page helpful?