Spark Dashboard

The Spark Dashboard provides visibility into Spark workload activity, resource allocation, and job execution trends across the cluster.

Use this dashboard to monitor Spark job performance, track resource consumption, analyze user activity, and identify workload patterns.

Navigation

1. In the xObserve UI, navigate to Spark from the left navigation pane.
2. Select Spark > Dashboard.
3. Select the desired cluster from the cluster selector in the upper-right corner.

The Spark Dashboard opens, displaying cluster activity, application health, resource utilization, and workload performance metrics.

Time Period

• Use the Time Period selector in the upper-left corner of the dashboard to filter metrics and charts for a specific monitoring window.
• You can view Spark activity for predefined periods such as the last hour, last 24 hours, or custom time ranges.

Metrics

Workload Overview

• Users – Number of users running Spark jobs during the selected time period.
• # of Jobs – Total number of Spark jobs executed during the selected time period.

Resource Allocation

• Avg CPU Allocated – Average CPU resources allocated to Spark jobs.
• Avg Mem Allocated – Average memory allocated to Spark jobs.

Job Status

• Succeeded – Number of jobs that completed successfully.
• Running – Number of jobs currently in progress.
• Failed – Number of jobs that failed during execution.
• Finishing – Number of jobs that are in the process of completing.
• Killed – Number of jobs that were terminated before completion.

Charts

Job Performance

• Average Job Time – Displays the average execution time and total execution time of Spark jobs over the selected time period.

User Activity

• Top 20 Users (By Query) – Displays the users generating the highest Spark query activity.

Job Trends

• Job Execution Count – Displays the number of Spark jobs executed over time, helping identify workload trends and activity patterns.