Acceldata
Pulse

Last updated: Nov 11, 2026 07:58 UTC

Enable NATS (ad-events) in Cluster Mode

This document explains the steps to enable NATS (ad-events service) in a Pulse for a multi-node setup. This setup requires you to have at least three nodes in your cluster. One of the three nodes must be the Pulse core node.

NATS (ad-events) provides messaging and event streaming for Pulse services. You can configure NATS in cluster mode to distribute the messaging workload across multiple nodes.

This guide describes how to migrate an existing single-node NATS deployment to a multi-node NATS cluster in Pulse.

Prerequisites

Before you begin, ensure that:

  • You have provisioned at least three nodes for the NATS cluster.
  • The cluster contains an odd number of nodes, such as 3 or 5.
  • Each node has a fully qualified domain name (FQDN) that the other nodes can resolve.
  • You have access to the Pulse master node and standalone nodes.
  • You have planned a maintenance window for the migration.

Important: Migrating from a single-node NATS deployment to a multi-node cluster requires temporarily stopping the ad-events service. The recovery time depends on the volume of data stored in NATS.

Step 1: Create the NATS Server Configuration

On each NATS host, create a nats-server.conf file in the $AcceloHome directory.

Add the following configuration:

debug: true
trace: false
server_name: ns1

port: 4222
http_port: 8222

jetstream = {
  store_dir: "/data/nats-server/"
  max_file: 500G
}

cluster = {
  name: "JSC"
  port: 6222
  listen: "0.0.0.0:6222"

  routes = [
    nats://<hostname_FQDN2>:19008
    nats://<hostname_FQDN3>:19008
  ]
}

Parameter

Description

server_name

Specify a unique server name for each host, such as ns1, ns2, or ns3.

cluster.routes

Specify the FQDNs of all other NATS hosts, excluding the current host.

For example, when configuring ns1, include the routes for ns2 and ns3.

Note: Keep the configuration file in $AcceloHome temporarily. You will move it to the required directory during migration.

Step 2: Migrate to a Multi-Node NATS Cluster

2.1 Move the configuration files

On each NATS host, move the nats-server.conf file to:

$AcceloHome/config/db/nats-server.conf

2.2 Generate the service configuration files

On the Pulse master node, run:

accelo admin makeconfig ad-core
$AcceloHome/config/docker/ad-core.yml
accelo admin makeconfig ad-sa-events

$AcceloHome/config/docker/addons/ad-sa-events.yml

2.3 Stop the NATS service

On the Pulse master node, stop the running ad-events container:

docker stop ad-events_default

2.4 Remove the NATS metadata directory

On the Pulse master node, run:

rm -rf "$AcceloHome/data/nats/\$SYS"

2.5 Configure host resolution (optional)

If a DNS resolver is unavailable, ensure that /etc/hosts contains entries for all NATS hosts.

On the Pulse master node, open:

vim $AcceloHome/config/docker/ad-core.yml
ad-events:
  volumes:
    - /etc/hosts:/etc/hosts
vim $AcceloHome/config/docker/addons/ad-sa-events.yml
ad-sa-events:
  volumes:
    - /etc/hosts:/etc/hosts

2.6 Start the NATS service

On the Pulse master node, run:

docker start ad-events_default

2.7 Deploy NATS on standalone nodes

On each standalone Pulse node, run:

accelo deploy addons

Complete the deployment by following the prompts.

Step 3: Update NATS Endpoints in Pulse

For each cluster configured in Pulse, open the corresponding configuration file:

$AcceloHome/config/acceldata_<clusterName>.conf
nats.server.host.list="hostname_FQDN1:19009,hostname_FQDN2:19009,hostname_FQDN3:19009"

Push the updated configurations to the database:

accelo admin database push-config -a

Step 4: Update NATS Configuration for Pulse Microservices

If any of the following microservices are deployed, update their NATS connection settings:

  • ad-pulse-profiler
  • ad-sparkstats
  • ad-yarn-optimizer
  • ad-alfred
  • ad-druid-analyzer
  • ad-axnserver

For each deployed service, generate the configuration file:

accelo admin makeconfig <serviceName>
accelo admin makeconfig ad-yarn-optimizer
$AcceloHome/config/docker/addons/

After updating the required microservice configurations, restart the Pulse services:

accelo restart all

Step 5: Update the NATS Configuration for Pulse Agents

For each cluster configured in Pulse, open the override.yml file:

$AcceloHome/work/<clusterName>/override.yml
base:
  nats_endpoint: hostname_FQDN1:19009,hostname_FQDN2:19009,hostname_FQDN3:19009

agent:
  impala_stream_replica: "3"
  hbazer_stream_replica: "3"

components:
  impala_agent_objectstore_replication: "3"

Parameter

Description

base.nats_endpoint

Specify the FQDNs and ports of all NATS hosts, separated by commas.

agent.impala_stream_replica

Set the value to the number of NATS hosts.

agent.hbazer_stream_replica

Set the value to the number of NATS hosts.

components.impala_agent_objectstore_replication

Set the value to the number of NATS hosts.

For example, if your NATS cluster contains three hosts, set the replication values to 3.

Save the configuration file and run:

accelo reconfig cluster -f

Note: Repeat this procedure for every cluster configured in Pulse.

Step 6: Update NATS Endpoints for Integration Hooks

If you have configured integration hooks for Hive, Tez, Spark, Trino, or Airflow, update their NATS endpoints to include all NATS hosts.

Hive and Tez Hooks

  1. Open the Hive and Tez hook configurations.
  2. Update the following property:
ad.events.streaming.servers=hostname_FQDN1:19009,hostname_FQDN2:19009,hostname_FQDN3:19009
  1. Save the changes.
  2. Restart the Hive and Tez services.

Spark Hook

  1. Open the Spark configuration in the appropriate location:
    • For standalone Spark deployments, update the custom spark-defaults configuration.
    • For managed Spark deployments, update the configuration through Cloudera Manager or Ambari.
  2. Update the following property:
spark.ad.nats.server.host.list=http://hostname_FQDN1:19009,http://hostname_FQDN2:19009,http://hostname_FQDN3:19009
  1. Save the changes.

Trino Hook

  1. Open the following configuration file:
/usr/odp/current/trino/conf/event-listener.properties
  1. Update the following property:
ad.events.streaming.servers=hostname_FQDN1:19009,hostname_FQDN2:19009,hostname_FQDN3:19009
  1. Save the changes.
  2. Restart the Trino coordinator.

Airflow Hook

  1. Locate the config.ini file associated with the Airflow listener under:
/usr/local/airflow/plugins
  1. Update the following property:
ad.events.url=hostname_FQDN1:19009,hostname_FQDN2:19009,hostname_FQDN3:19009
  1. Save the changes.

Step 7: Verify the NATS Cluster Configuration

After completing the migration, verify that:

  • The NATS containers are running on the master and standalone nodes.
  • All configured NATS hosts can communicate with one another.
  • The NATS cluster recognizes the expected number of nodes.
  • Pulse microservices can connect to the updated NATS endpoints.
  • Pulse agents use the updated NATS configuration.
  • Configured integration hooks can send events to NATS.