Spark Jobs

The Spark Jobs page provides a consolidated view of Spark job activity across the cluster. Use this page to analyze job execution, monitor failures, identify active workloads, and track job performance.

The page also provides filtering and sorting capabilities to help you focus on specific jobs, users, namespaces, or job statuses.

Navigation

1. In the xObserve UI, navigate to Spark from the left navigation pane.

2. Select Spark > Jobs.

3. Select the desired cluster from the cluster selector in the upper-right corner.

The Spark Jobs page opens, displaying all Spark jobs executed within the selected time range.

Time Period

• Use the Time Period selector in the upper-left corner of the page to filter job data for a specific monitoring window.

• You can view job activity for predefined periods such as the last hour, last 24 hours, or custom time ranges.

Search and Filters

Use the search bar to locate specific jobs and narrow down results based on:

• Namespace – Filter jobs by Kubernetes namespace.

• Users – Filter jobs by the user who submitted them.

• Name – Filter jobs by job name.

• Status – Filter jobs by execution status.

  • Click the logs icon to view the job-specific logs. For details, see Monitor Logs.

Job Summary

The Jobs page displays the following metrics for each job group:

Job Information

• Group Name – Name of the Spark job group.

• Job Count – Total number of jobs within the group.

Job Status

• Failed Jobs – Number of jobs that failed during execution.

• Running Jobs – Number of jobs currently in progress.

Job Performance

• Avg Duration/Job – Average execution time per job.

• Total Duration – Total execution time for all jobs within the group.

Additional Actions

• Sort job information using the column headers.

• Refresh the page to retrieve the latest job metrics.

• Export Results: Export record details for offline analysis or reporting.

  • A maximum of 10,000 records can be exported in a single operation. Exporting all available records at once is not supported.

  • To export additional records, refine your search criteria and perform multiple export operations.