Pulse Agents

Agents are lightweight services, and they follow a push model and send system and service-level metrics, logs, and health signals to Pulse databases.

Pulse Agents follow a master–worker model. The Hydra Server acts as the master and controls the configuration and management of the Agents (workers). The Hydra Server must be deployed, regardless of whether Agents are deployed using the Hydra Server or any other methods.

The different types of Pulse Agents are as follows:

• Node Agent: Collects system and service-level metrics
• Logs Agent: Collects service logs
• JMX Agent: Collects service JMX metrics
• AXN Agent: Runs Pulse Actions
• YARNMetrics Agent: Collects Node and YARN job metrics

All agents are managed by the Hydra Server, which handles their configuration, lifecycle, and updates using VARs.yml and Override.yml.

The VARs.yml file consists of the default configuration values. However, you can modify the configurations using the Override.yml file.

Hydra Server

This service acts as a central service that manages the lifecycle of all Pulse Agents.

This service's responsibilities include:

• Deploying and configuring agents on target nodes
• Handling agent refreshes and updates

This service ensures secure communication between Agents and Pulse services.

Pulse Hydra Agent

This Agent ensures that all Pulse agents are correctly configured and synchronized.

This Agent's responsibilities include:

• Applying configurations from the vars.yml and override.yml files
• Managing agent lifecycle (start, stop, restart)
• Monitoring and reporting agent health
• Performing remote updates and configuration refreshes
• Ensuring consistency of agent behavior across the cluster

Pulse AXN Agent

This Agent executes the stock and custom Pulse Actions triggered from the UI.

This Agent's responsibilities include: Executing automated actions.

Pulse AXN Server

PulseAXN Server collects action details and instructs the Agent to execute them. This supports schedule-based execution models.

Pulse Node Agent

This Agent collects real-time system and service-level metrics from nodes.

This agent collects the following system-level metrics:

• CPU utilization
• Memory usage
• Disk I/O (if applicable)
• Other service metrics

Pulse JMX Agent

This Agent collects Java Management Extensions (JMX) metrics from JVM-based services.

This agent monitors Java-based applications and collects the following metrics:

• Captures JMX metrics including GC time, heap usage, thread counts, etc.

Pulse Logs Agent

This Agent collects and forwards logs from nodes and clusters to centralized storage.

This Agent with log aggregation tools such as Logstash and ElasticSearch, where Pulse Logs sends data to LogStash for processing, which then routes it to ElasticSearch for searchable access in the Pulse UI.

This agent monitors applications and collects the following types of logs:

• Service logs (e.g., Spark, YARN, HDFS)
• System logs

Pulse YARNMetrics Agent

This Agent collects resource utilization metrics from the YARN ResourceManager and NodeManagers.

This agent monitors YARN components and collects the following metrics:

• YARN container metrics
• YARN Node Metrics