Hive on Spark Query Details

The Hive on Spark Query Details page contains the following panels.

Summary
Query Trends
Recommendations
Query
YARN Diagnostics
MapReduce Stats
Query Execution Metrics
Query Plan and DAG

The summary panel displays the following information.

Field Name	Description
User	The name of the user that executed the job.
State	The state of the job that can be one of the following: Created, Initialized, Compiled, Running, Finished, Exception, or Unknown.
Duration	The duration of the query execution.
Start Time	The time at which the query execution started.
End Time	The time at which the query execution ended.
Bytes Read	The number of bytes read while executing the query across a given number of stages.
Bytes Written	The number of bytes written to a file format while executing the query across a given number of stages.
Shuffle Bytes Read	The number of shuffle bytes read (in bytes) across a given number of stages.
Shuffle Bytes Written	The number of bytes written in shuffle operations across a given number of stages.
jvmGCTime	Time spent by the JVM in garbage collection while executing a task.

Query Trends

The Query Trends panel displays a chart showing the pattern of jobs running at a particular time, based on the following factors.

Metric	Description
Elapsed Time	The time taken to run the jobs at a particular time.
VCores	The number of VCores consumed to execute the query within a timeframe.
Memory	The amount of memory used to execute the query within a timeframe.

Comparing Runs

Click Compare Runs to compare different runs of the query. Select the runs that you want to compare. You can choose from upto 10 previous runs of the query. The metrics that are different are highlighted and displayed at the very top of the comparison result.

Query

The Query panel displays the query along with the Joins and Table Name used in the query.

Core Usage by Locality

The Core Usage by Locality chart displays the core usage by the following locality types. The chart also displays Core Used and Core Wasted values (in %).

Process Local: The tasks in this locality are run within the same process as the source data.
Node Local: The tasks in this locality are run on the same machine as the source data.
Rack Local: The tasks in this locality are run in the same rack as the source data.
Any: The tasks in this locality are run anywhere else but not on the same node or rack.
No pref: The tasks in this locality have no locality preference.
Idle: The tasks in this locality that are idle.

Query Plan

Query Plan is a logical representation of how MapReduce executes the query, where a query is broken into different stages. The query plan is displayed in JSON format.

Last updated on

Was this page helpful?