Apply Performance Recommendations

Pulse provides recommendations for each application type to help you optimize job execution.

For jobs of every application type you submit, you can view recommendations to optimize the performance of your jobs. These recommendations are based on jobs submitted by applications in the YARN queue of your Hadoop cluster.

Steps

1. In the Pulse UI, select YARN from the left navigation bar > Application Explorer.
2. On the Application Explorer page, click Recommendations.

The recommendations are categorized for focused analysis. Use these insights to identify inefficiencies in your cluster and apply best practices for improving performance, resource utilization, and job stability.

• Low Mapper Memory Usage

  • Mapper tasks using significantly less memory than allocated.
  • Reduce mapper memory allocation to optimize resource utilization and improve cluster throughput.

• Low Max Memory

  • Applications using significantly less memory than allocated.
  • Reduce maximum memory for these tasks to improve overall resource utilization and increase cluster throughput.

• High Memory

  • Jobs are consuming excessive memory.
  • Optimize job configurations to balance resource usage and prevent bottlenecks.

• Task Runtime Skews

  • Tasks within an application have highly varying runtimes.
  • Identify and optimize long-running outlier tasks to improve overall job performance.

• Small Tasks

  • Tasks process minimal amounts of data, causing inefficient execution due to overhead (e.g., task setup time, I/O operations).
  • Consolidate or optimize these tasks to reduce setup, I/O, and scheduling inefficiencies.