Broker Dashboard

Pulse monitors Pinot’s memory, CPU, garbage collection, and threads to assess JVM health and identify performance issues.

With Pulse, you can:

Track query compilation, execution, reduce operations, and scatter-gather behavior to evaluate query performance.
View burst QPS, server health, and request load to monitor system throughput and reliability.

In the Pulse UI, navigate to the menu option and select Pinot > Broker Dashboard. The metrics and charts appear on the screen for you to analyze.

JVM Metrics Overview

Pulse provides a range of JVM-level charts to help monitor Pinot’s internal health. These include:

Memory Usage: Tracks heap and non-heap memory across initialization, usage, and limits.
CPU and Swap: Shows system and process CPU usage, and available swap space.
Garbage Collection (GC): Displays frequency and duration of GC events for young and old generations.
Thread Activity: Monitors live thread counts, peak usage, and total threads started.

These charts help identify memory pressure, performance degradation, and potential bottlenecks in the Pinot JVM process.

Query Performance Metrics

Pulse charts also provide deep insights into Pinot query behavior across brokers:

Query Compilation: Tracks the count, latency, and max time of compiled queries by table and host.
Query Execution: Shows execution time, execution count, and performance trends by host/table.
Reduce & Scatter-Gather Operations: Monitors data aggregation and distribution patterns per table and host.
Burst QPS: Highlights peak query throughput at both table and host levels.
Unhealthy Servers: Flags unresponsive or degraded Pinot servers.

These metrics help in tuning Pinot’s query pipeline and understanding real-time system behavior.

Pinot Broker Charts

Chart	Description
Memory: Heap Memory	This chart shows the heap memory usage at different stages: Heap_Memory_Init_Value: Initial heap memory requested by the JVM at startup. Heap_Memory_Committed Value: Heap memory reserved by the OS and guaranteed for JVM use. Heap_Memory_Used_Value: Current heap memory consumed by live objects in the application. Heap_Memory_Max_Value: Maximum heap memory allowed for the JVM to use.
Memory: Non-Heap Memory	This chart shows the heap memory usage at different stages: Non_Heap_Memory_Init: Initial non-heap memory requested by the JVM at startup, including metaspace, code cache, and compressed class space. Non_Heap_Memory_Committed: Memory reserved by the OS for JVM non-heap use, such as class metadata and compiled code. Non_Heap_Memory_Used: Current non-heap memory in use, reflecting actual consumption of metaspace, code cache, and class space. Non_Heap_Memory_Max: Maximum non-heap memory allowed for the JVM to use.
Operating System: CPU Load	This chart shows: SystemCpuLoad – Recent CPU usage of the entire system, as reported by the operating system. ProcessCpuLoad – Recent CPU usage of the Pinot JVM process, relative to the total available CPUs.
Operating System: Free Swap Space Size	Shows the amount of free swap space available on the host system (In Bytes), as seen by the JVM process running the JVM Broker.
GC: Collection Count	Shows the number of garbage collection (GC) events over time for both: G1 Young Generation (frequent short-lived collections) G1 Old Generation (infrequent, long-lived collections) Tracks the frequency of GC events to help identify memory pressure and potential performance issues in the JVM.
GC: Collection Time	Shows the time taken (in milliseconds) for each GC event. G1 Young Generation (frequent short-lived collections) G1 Old Generation (infrequent, long-lived collections) These charts help assess JVM memory management efficiency.
Threading: Thread Count	This chart displays key thread metrics for the Pinot JVM process: ThreadCount – Current number of live threads. PeakThreadCount – Highest number of live threads observed since the JVM started (resets on restart). TotalStartedThreadCount – Total threads ever created (including terminated ones); this value keeps increasing until the JVM restarts.
Max Burst QPS: Table level	Shows the maximum burst QPS (Queries per second) per table as defined by the query throttle configuration.
Max Burst QPS: Host level	Shows the maximum burst QPS (Queries per second) per host as defined by the query throttle configuration.
Reduce Operations: Table Level	Shows the number of reduce operations on the Pinot Broker per table.
Reduce Operations: Host Level	Shows the number of reduce operations on the Pinot Broker per host.
Unhealthy Servers	Shows the number of unhealthy Pinot servers.
Reduce Operations: One-Minute Rate	Shows a one-minute rate of reduced operations on the Pinot Broker.
Reduce Operation: Max Time	Shows the maximum reduced operation time observed by Pinot Broker.
Scatter Gather: Host Level	Show the number of scatter-gather operations performed by the Broker per host for queries.
Scatter Gather: Table Level	Show the number of scatter-gather operations performed by the Broker per table for queries.
Scatter Gather: One Minute Rate	Show the One Minute Rate of the number of scatter-gather operations performed by the Broker per table for queries.
Request Compilation: Table Level	Shows the total number of queries compiled by the Broker per table.
Request Compilation: Host Level	Shows the total number of queries compiled by the Broker per host.
Request Compilation: Latency Stats	Shows the 99th and 999th percentile latency of query request compilation for different tables.
Request Compilation: Max Time	Shows the maximum time taken by the broker to compile or parse a query request.
Query Execution: Max Execution Time	Shows the maximum query execution time observed by the Pinot Broker.
Query Execution: Host Level	Shows the count of queries executed per host.
Query Execution: Table Level	Shows the count of queries executed per table.

Last updated on

Was this page helpful?