MapReduce Details

The MapReduce page displays the details of a selected query.

The following panels are displayed in the MapReduce Details page:

Summary Panel

The following table provides description of the details displayed in the summary panel:

Metric	Description
Status	The status of the job can be one of the following: Succeeded, Failed, Killed, Undefined.
User	The name of the user.
Elapsed Time	The total time taken by the job.
Queue	The name of the queue.
Started Time	The time at which the application execution started.
Finished Time	The time at which the application execution ended.
# of Unique Runs of the Application	The number of applications used by the application.
Peak Memory	The highest amount of memory allocated by the application.
Peak VCore	The highest number of Vcore allocated by the application.
Memory Used	Amount of memory used within the application in percentage.
VCore Used	Number of virtual cores used within the application in percentage.
Data Read	The amount of data read by the application.
Data Written	The amount of data written to an output file format.

MapReduce Stats

Summary Panel

The following table provides description of the metrics displayed in the MapReduce Stats summary panel:

Metric	Description
Avg Map Time	The average time taken to complete the mapping phase.
Avg Merge Time	The average time taken to merge the mapper and reducer phase.
Avg Reduce Time	The average time taken to complete the reducer phase.
Avg Shuffle Time	The average time taken to transfer the map output from Mapper to Reducer.

Chart

The chart displays the Mappers & Reducers by elapsed time.

The chart displays the statistics of processing of large data sets on a worker node. You can monitor the statistics of following processes by elapsed time:

Mappers
Reducers

By default, both the Mappers and the Reducers stages are displayed. If you want to filter data, perform the following:

Click Show. The drop-down menu is displayed.
Select Mappers to filter data related to Mapper. Select Reducers to display data related to the Reducer.
(Optional) Select All to remove the applied filter

These statistics can be sorted by Duration and Start Time. To sort, perform the following:

Click SortBy. The drop-down menu is displayed.
Select Duration to sort data by duration. Select Start Time to sort data by the starting time of the process.
(Optional) Select None to remove the applied sort.

Note Scroll along the slider to view the top Mapper and Reducers.

Table

The following table provides description of the metrics displayed in the MapReduce Stats summary table:

Metric	Description
Memory Used	Mappers: The total amount of memory used in the mapping process in percentage. Reducer: The total amount of memory used in the reducing process in percentage. Total: The total amount of memory used in the mapping and reducing process in percentage.
CPU Used	Mappers: The total amount of CPU used in the mapping process in percentage. Reducer: The total amount of CPU used in the reducing process in percentage. Total: The total amount of CPU used in the mapping and reducing process in percentage.
HDFS Read	Mappers: The total amount of HDFS data read in the mapping process in percentage. Reducer: The total amount of HDFS data read in the reducing process in percentage. Total: The total amount of HDFS data read in the mapping and reducing process in percentage.
HDFS Writes	Mappers: The total amount of HDFS data written to an output file format in the mapping process in percentage. Reducer: The total amount of HDFS data written to an output file format in the reducing process in percentage. Total: The total amount of HDFS data written to an output file format in the mapping and reducing process in percentage.

Query

The Query panel displays the SQL query details.

Note To copy SQL, click and to beautify the SQL, click .

VCore & Memory During Application Runtime

The chart displays the VCores and the Memory

Cluster & Queue Usage During Application Runtime

The chart displays the amount of cluster used and the amount of queue used in percentage for the duration of the application.

Last updated on

Was this page helpful?