User

The name of the user that executed the job.

State

The state of the job that can be one of the following: Created, Initialized, Compiled, Running, Finished, Exception, or Unknown.

Duration

The time taken to run the query.

Delay Time

The YARN scheduler delay time. The difference in time taken for a query to move from Accepted state to Running state.

Start Time

The time at which the query execution started.

End Time

The time at which the query execution ended.

Queue

The name of the Queue in which the Query is stuck. You can

# of Vertices

The number of vertices in the query.

HDFS Data Read

The amount of HDFS data read.

HDFS Data Written

The amount of HDFS data written to an output file format.

Application ID

The ID of the application of the user you are currently viewing.

Elapsed Time

The time taken to run the jobs at a particular time.

VCores

The number of VCores consumed to execute the query within a timeframe.

Memory

The amount of memory used to execute the query within a timeframe.

HDFS Read

The amount of HDFS data read.

HDFS Write

The amount of HDFS data written to an output file format.

Metric

Displays the metric difference of the runs

Config

Displays the configuration difference of the runs.

No limit statement on final projection

No limit clause

SELECT col1 FROM table WHERE col1 = true;

Star projection

Use qualified column names

SELECT * FROM table LIMIT 10;

Possible query error

No where clause detected

SELECT col1 FROM table;

Table partition is unused

Partition not used

SELECT * FROM table;

Possible optimization

Function call in predicate, use a CTE query.

SELECT * FROM

Possible optimization

Aggregate Function call in a Having clause, Lift in to where clause.

SELECT col1 FROM GROUP BY col1 HAVING COUNT(col1) > 1

Possible query error

This 1uery uses multiple tables, but has non joins and no where clause. This may cause a cartesian join.

SELECT t1.col, t2.col FROM t1, t2

Possible order optimization

Orderby can be replaced with Sortby to use multiple reducers. Ordering will be enforced within reducers.

SELECT col1, col2 FROM t1 ORDER BY col1, col2

Change conversion type or decrease threshold to improve performance

Severity: 1

For single stage using fetch operator, either modify task conversion type or threshold to improve performance.

select * from call_center where cc_ call_center_id="AAAAAAAABAAAAAAA" limit 10;

SELECT *, FILTER on partition columns (WHERE and HAVING clauses), LIMIT only.

Move transformations on projection

Severity: 1

select * from (

select st1,i1,sa,sb,a,b from t1

) t1 join t2 on (

st1=st2 and

i1=i2 and

case when sa = 'yes' then a when sb = 'yes' then b end = x and

st2 = 'asd1'

Avoid row_number() functions as its not vectorized

Severity: 1

Non-vectorized operator $operator identified in the $stage

SQL with row_number()

Specifically typecast the columns which are going in for string comparison to be cast as string if not already string datatype

Severity: 1

Non-vectorized operator $operator identified in the $stage

where string_col=double_ col

Cross join identified, revert and refrain from usage

Severity: 0

Cross join is identified, please check the join keys or query conditions

select * from <table1> cross join <table2

Identify M/R non-vectorized

Severity: 1

Complex queries usually have large numbers of joins, often over 10 joins per query.

Severity: 1

More than 10 joins identified in the query

Table Name

The table used in the query.

Database Name

The Database to which the Table belongs to.

Filter expression

The expression used in query filtering.

Expected Size

The total number of rows in the table.

Expected Rows

The number of rows returned on executing the query.

Start Time

The time at which the YARN application started.

End Time

The time at which the YARN application ended.

State

The state of the YARN application. The state can be one of the following: Created, Initialized, Compiled, Running, Finished, Exception, or Unknown.

Message

The diagnostic message in the YARN application.

Message Count

The number of diagnostic messages.

Time

The minute at which the job is executed.

Preempted MB

The amount of processes that need priority to run the job.

Preempted VCores

The number of VCores that need priority to run the job.

Allocated MB

The amount of memory allocated to the query (in Mb).

Avg Memory

The average a mount of memory used.

Avg VCore

The average amount of VCores used.

Running Containers

The number of containers running in the query.

Queue Usage %

The amount of queue usage (in %).

Cluster Usage %

The amount of cluster usage (in %).

State

Displays the state of the cluster

Message

The diagnostic message.

Vertex ID

Displays the Vertex ID. Clicking on the Vertex ID directs you to the Query Plan and DAG

Status

Displays the Tez job status, such as Succeeded, Failed, Submitted, Killed, and Error.

Total Tasks

The total number of tasks in each vertex

Duration

The total time taken to execute the query, including any waiting time for dependent vertices.

Failed Attempts

The number of completed job attempts with an unexpected status value

Killed Tasks

The number of duplicate copies of a task that were attempted and terminated

CPU Time

The CPU time taken to complete the query

GC Time

Time spent in garbage collection while executing a query

Input Records

The number of input records

Output Records

The number of output records

Task

Committed Heap Bytes

The maximum amount of memory (in bytes) that can be used for memory management.

CPU Milliseconds

The CPU time (in ms).

GC Time

Time spent by the JVM in garbage collection while executing a query.

Input Records Processed

The number of input records processed.

Merge Phase Time

The time taken to merge a query phase.

Merged Map Outputs

The number of map outputs that were merged.

Output Bytes

The number of output bytes written to a file format while executing the query at a given time.

Output Records

The number of output records.

Physical Memory Bytes

The amount of physical memory the query uses.

Shuffle Bytes To Disk

The number of shuffle records written to disk.

Shuffle Bytes To Mem

The number of shuffle records written to memory.

Shuffle Bytes

The number of shuffle bytes the query uses.

Shuffle Phase Time

The time taken to shuffle a query phase.

Spilled Records

The number of spilled records.

Virtual Memory Bytes

The amount of virtual memory used by queries.

File System Metrics

HDFS Bytes Written

The number of HDFS bytes written to the query executor.

File Bytes Written

The number of file bytes written to the query executor.

HDFS Bytes Read

The number of HDFS bytes read.

File Bytes Read

The number of file bytes read.

DAG Metrics

Total Launched Tasks

The number of tasks launched in DAG.

Rack Local Tasks

The number of tasks that are local to the rack.

Data Local Tasks

The number of tasks that are local to the data.

of Succeeded Tasks

The number of tasks that completed successfully.

Application Master Cpu Time

The time taken by the CPU in application master.

Application Master GC Time

The time taken by the GC in the application master.

HIVE Metrics

Hive Files Created

The number of HIVE files that were created.