Configure YARN Metrics Agent

This page describes how to configure the YARN Agent to collect metrics and apply optimization for YARN.

The configuration allows you to connect to your cluster, manage authentication, control metric collection frequency, and fine-tune agent behavior.

Cluster Connection Settings

Configure the basic cluster connection and authentication settings.

Field	Description	Example
Cluster Name	Unique identifier for the cluster. Automatically populated from the current cluster.
Resource Manager Endpoints	Comma-separated list of Resource Manager endpoints.	<<IP_Address:port>
Health Manager Endpoint	Endpoint for the Health Manager service.	<<IP_Address:port>

Authentication & Security

Set up Kerberos and SSL configuration for secure communication.

Field	Description	Default
Enable Kerberos	Enables Kerberos authentication.	Disabled
Enable SSL for Resource Manager	Enables SSL for the Resource Manager connection.	Disabled

Monitoring Frequencies

Configure how frequently different metrics are collected and emitted.

Field	Description	Default Value
Containers Monitoring Frequency (ms)	How often to monitor containers (in milliseconds).	100
Containers Metric Emit Frequency (ms)	How often to emit container metrics (in milliseconds).	5000
Nodes Monitoring Frequency (ms)	How often to monitor nodes (in milliseconds).	1000
Nodes Metric Emit Frequency (ms)	How often to emit node metrics (in milliseconds).	5000
Explore PIDs Frequency (ms)	How often to explore process IDs (in milliseconds).	1000
Heartbeat Interval (seconds)	Heartbeat interval between agent updates (in seconds).	30

Advanced Settings

Fine-tune the YARN Agent’s performance and optimization behavior.

Field	Description	Default Value
Use FQDN for Yarn Optimize	Uses Fully Qualified Domain Names for YARN optimization.	Disabled
Enable Overcommit Timeout	Enables timeout handling for overcommit operations when the YARN Optimizer Service is unavailable. In this case, the Agent automatically restores optimization settings to their normal values to maintain cluster stability.	Enabled
Overcommit Timeout (seconds)	Timeout for overcommit operations (in seconds).	60
Node ID Port	Port number used for node identification.	45454
Node Manager Port	Port number for Node Manager connections.	8042

File Paths & Configuration

Specify file system paths and configuration file locations used by the YARN Agent.

Field	Description	Example
Yarn Site XML Path	Path to the yarn-site.xml configuration file.	/etc/hadoop/conf/yarn-site.xml
Yarn Binary Path	Path to the YARN binary executable.	/usr/bin/yarn

Agent Control Settings

Configure agent behavior, host enablement, and overcommit timeout settings.

Field	Description
Enabled Hosts	List of hosts where you can enable the YARN Optimizer Agent. Use regex patterns to find hosts efficiently.
Disabled Hosts	List of hosts where you can disable the YARN Optimizer Agent. Use regex patterns to find hosts efficiently.

Field

Description

Enabled Hosts

List of hosts where you can enable the YARN Optimizer Agent.

Use regex patterns to find hosts efficiently.

Disabled Hosts

List of hosts where you can disable the YARN Optimizer Agent.

Use regex patterns to find hosts efficiently.

Summary

The YARN Agent Configuration page provides full control over how metrics are collected, monitored, and emitted for YARN optimization.

By fine-tuning connection settings, security options, and monitoring frequencies, you can ensure optimal resource management and performance visibility across your YARN environment.

Last updated on

Was this page helpful?