CDP Multi-node, Multi-KDC, Multi-Cluster

This document provides you with a step by step process to deploy single Pulse instance for multiple Cloudera clusters with multiple KDCs.

Prerequisites

Ensure you have the following information for both clusters:

  1. CM URL (https://<Alias/FQDN of the CM URL>:<CM Port>)

  2. CM Username

  3. CM Password

  4. Spark History HDFS path & Spark3 History HDFS path

  5. Kafka Version

  6. Hbase Version

  7. Hive Version

  8. Hive Metastore DB Connection URL

  9. Hive Metastore Database Name

  10. Hive Metastore DB Username

  11. Hive Metastore DB Password

  12. Oozie DB Name

  13. Oozie DB URL

  14. Oozie DB Username

  15. Oozie DB Password

  16. Kerberos Keytab

  17. krb5.conf file

  18. Principal

  19. Kerberos Username

  20. cacerts/jssecacerts

  21. YARN Scheduler Type

  22. Kafka Interbroker protocol

Note

To enable (TLS) HTTPS for Pulse Web UI Configuration, ensure you have the following present:

  1. Certificate File: cert.crt

  2. Certificate Key: cert.key

  3. CA Certificate: ca.crt (optional)

  4. Decide whether to keep the HTTP port (Default: 4000) open or not

  5. Decide on which port to use (default: 443)

For details, see Enable Native SSL/TLS for Pulse Web UI.

Uninstallation of Agents

Perform the following:

  1. To uninstall agents, you must follow the Cloudera Parcel Agent document.

  2. You must also remove the Pulse JARs and the configurations for Hive and Tez.

  3. The Acceldata team must then run the following commands for backup and uninstalling the existing Pulse application.

a. Create a backup directory:

mkdir -p /data01/backup

b. To backup, copy the entire config and workdirectory:

cp -R $AcceloHome/config /data01/backup/ cp -R $AcceloHome/work /data01/backup/

c. Uninstall the existing Pulse setup by running the following command:

accelo uninstall local

OUTPUT

[root@nifihost1:data01 (ad-default)]$ accelo uninstall local ✗ You're about to uninstall the local AccelData setup. This will also DELETE all persistent data from the current node. However, NONE of the remote no ✔ You're about to uninstall the local AccelData setup. This will also DELETE all persistent data from the current node. However, NONE of the remote no ✔ You're about to uninstall the local AccelData setup. This will also DELETE all persistent data from the current node. However, NONE of the remote no You're about to uninstall the local AccelData setup. This will also DELETE all persistent data from the current node. However, NONE of the remote nodes will be affected. Please confirm your action [y/n]: : y WARN: Gauntlet is running in dry run mode. Disable this to delete indices from elastic and purge data from mongo DB Uninstalling the AccelData components from local machine ...

  1. Logout from the terminal session.

  2. Perform the aforementioned steps for all Pulse server nodes.

Download and Load the Binaries and Docker Images

To download and load the binaries and Docker images, perform the following:

  1. Download the jars, hystaller, accelo binaries and docker images from the download links provided by the Acceldata team.

  2. Move the Docker images and jars into the following directory:

mkdir -p /data01/images

  1. Copy the binaries and tar files into the /data01/images folder:

cp </path/to/binaries/tar> /data01/images

  1. Change the directory:

cd /data01/images

  1. Extract the single tar file:

tar xvf <name_of_tar_file>.tar

OUTPUT

[root@nifihost1 images]# tar xvf pulse-333-beta.tar ./ad-alerts.tgz ./ad-connectors.tgz ./ad-dashplots.tgz ./ad-database.tgz ./ad-deployer.tgz ./ad-director.tgz ./ad-elastic.tgz ./ad-events.tgz ./ad-fsanalyticsv2-connector.tgz ./ad-gauntlet.tgz ./ad-graphql.tgz ./ad-hydra.tgz ./ad-impala-connector.tgz ./ad-kafka-0-10-2-connector.tgz ./ad-kafka-connector.tgz ./ad-ldap.tgz ./ad-logsearch-curator.tgz ./ad-logstash.tgz ./ad-notifications.tgz ./ad-oozie-connector.tgz ./ad-pg.tgz ./ad-pulsemon-ui.tgz ./ad-recom.tgz ./ad-sparkstats.tgz ./ad-sql-analyser.tgz ./ad-streaming.tgz ./ad-vminsert.tgz ./ad-vmselect.tgz ./ad-vmstorage.tgz ./accelo.linux ./admon ./hystaller

  1. Load the Docker images by running the following command:

ls -1 *.tgz | xargs --no-run-if-empty -L 1 docker load -i

  1. Ensure that all the images are loaded to the server by running the following command:

docker images | grep 3.3.3

Cluster Configuration

To configure the cluster, perform the following:

  1. Validate all the host files.

  2. Create the acceldata directory by running the following command:

cd /data01/ mkdir -p acceldata

  1. Place the accelo binary in the /data01/acceldata directory:

cp </path/to/accelo/binary> /data01/acceldata

  1. Rename the accelo.linux binary to accelo:

mv /data01/acceldata/accelo.linux accelo chmod +x /data01/acceldata/accelo

  1. Change the directory:

cd /data01/acceldata/accelo

  1. Run the following accelo init command:

./accelo init

  1. Enter the appropriate answers when prompted.

  2. Source the ad.sh file:

source /etc/profile.d/ad.sh

  1. To enter the Pulse version, run the init command:

accelo init

OUTPUT

[root@nifihost1:~ (ad-default)]$ accelo init Enter the AccelData ImageTag: : 3.3.3 ✓ Done, AccelData Init Successful.
Note

Provide the correct Pulse version, in this case its 3.3.3

  1. To get the initial information, run the accelo info command:

accelo info

OUTPUT

[root@nifihost1:~ (ad-default)]$ accelo info WARN: Gauntlet is running in dry run mode. Disable this to delete indices from elastic and purge data from mongo DB ___ ____________________ ____ ___ _________ / | / ____/ ____/ ____/ / / __ \/ |/_ __/ | / /| |/ / / / / __/ / / / / / / /| | / / / /| | / ___ / /___/ /___/ /___/ /___/ /_/ / ___ |/ / / ___ | /_/ |_\____/\____/_____/_____/_____/_/ |_/_/ /_/ |_| Accelo CLI Version: 3.3.3 Accelo CLI Build Hash: 8ba4727f11e5b3f3902547585a37611b6ec74e7c Accelo CLI Build ID: 1700746329 Accelo CLI Builder ID: ZEdjMmxrYUdGdWRGOWhZMk5sYkdSaEVLCg== Accelo CLI Git Branch Hash: TXdLaTlCVDFBdE56STNvPQo= AcceloHome: /data01/acceldata AcceloStack: ad-default AccelData Registry: 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata AccelData ImageTag: 3.3.3 Active Cluster Name: NotFound AcceloConfig Mongo DB Retention days: 15 AcceloConfig Mongo DB HDFS Reports Retention days: 15 AccelConfig TSDB Retention days: 31d Number of AccelData stacks found in this node: 0

  1. To enable multi-KDC, perform the following:

a. Modify the accelo.yml file.

vi $AcceloHome/config/accelo.yml

b. Change IsMutliKDCclusterEnabled to true.

ContainerRegistry: 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata ImageTag: 3.3.3 ContainerRegistryID: "" ContainerRegistryKey: "" ContainerUserID: 1000:1000 ContainerLogType: json-file ContainerLogOpts: max-buffer-size: 4m max-file: "3" max-size: 10m mode: non-blocking IsMultiKDCclusterEnabled: true KerberosAddons: - ad-connectors - ad-sparkstats - ad-kafka-connector - ad-kafka-0-10-2-connector - ad-impala-connector - ad-fsanalyticsv2-connector config: enable_gauntlet: true retention: mongo_retention_days: 15 mongo_hdfs_retention_days: 15 tsdb_retention_days: 31d snap_mongo_cleanup_frequency_in_hours: "0"

c. Save the file.

  1. To configure the cluster in Pulse, run the config cluster command.

accelo config cluster

  1. Provide appropriate answers when prompted.

[root@pulsecdp01:acceldata (ad-default)]$ accelo config cluster INFO: Configuring the cluster ... INFO: Using default API Version v10 for CM API Is the 'Database Service' up and running? [y/n]: : n WARN: Gauntlet is running in dry run mode. Disable this to delete indices from elastic and purge data from mongo DB ✔ Cloudera Enter Your Cluster's Display Name: : cdp1 Enter Cloudera URL (with http/https): : https://cdpssl01.acceldata.dvl:7183 ✔ Enter Cloudera Username: : admin█ IMPORTANT: This password will be securely encrypted and stored in this machine. Enter Cloudera User Password: : ***** Enter the cluster name to use (MUST be all lowercase & unique): : cdp1 ERROR: stat /data01/acceldata/.activecluster: no such file or directory INFO: Creating Post dirs. ✔ Cluster1 INFO: Using lower case for CDP Service name API INFO: Using lower case for CDP Service name API INFO: Using lower case for CDP Service name API INFO: Using lower case for CDP Service name API INFO: Using lower case for CDP Service name API INFO: Using lower case for CDP Service name API INFO: Using lower case for CDP Service name API INFO: Using lower case for CDP Service name API INFO: Using lower case for CDP Service name API INFO: Using lower case for CDP Service name API Enter the installed Kafka version (ex: 0.10.2): : 0.11.0 : 0.11.0█ Enter the installed HBase service version (ex: 0.9.4): : 0.9.4 Enter the installed Hive service version (ex: 2.0.0): : 2.0.0 Enter the installed Hive service version (ex: 2.0.0): : 2.0.0 ✓ Found Kerberos Realm: ADSRE.COM Enter the Spark History HDFS path: : /user/spark/applicationHistory Oozie DB URL: : jdbc:postgresql://cdpssl01.acceldata.dvl:7432/oozie_oozie_server ✔ Oozie DB URL: : jdbc:postgresql://cdpssl01.acceldata.dvl:7432/oozie_oozie_server█ Enter the Oozie DB Username: : oozie_oozie_server Enter the Oozie DB Password: : ********** Enter the Oozie DB JODA Timezone (Example: Asia/Kolkata): : Asia/Kolkata ✔ Enter the hive metastore Database Name : : hive█ ✔ Hive Metastore PostgreSQL DB Connection URL: : jdbc:postgresql://cdpssl01.acceldata.dvl:7432/hive█ Enter the hive metastore DB Username : : hive ✔ Enter the hive metastore DB Password : : **********█ ✔ Enter the hive metastore DB Password : : **********█ INFO: core-site.xml file has been updated INFO: hdfs-site.xml file has been updated ---------------------------Discovered configurations---------------------------------------- ✓ Cluster Type: CDH ✓ CDH Version: 7.1.7 ✓ Discovered Cluster Name: cdp1 ✓ Discovered Services: ✓ PULSEHYDRAAGENT ✓ SOLR ✓ SPARK_ON_YARN ✓ KAFKA ✓ LIVY ✓ HUE ✓ HIVE_ON_TEZ ✓ HBASE ✓ QUEUEMANAGER ✓ RANGER ✓ IMPALA ✓ ATLAS ✓ ZOOKEEPER ✓ OOZIE ✓ HIVE ✓ YARN ✓ HDFS ✓ Yarn RM URI: https://cdpssl02.acceldata.dvl:8090,https://cdpssl03.acceldata.dvl:8090 ✓ MapReduce Job History URI: https://cdpssl02.acceldata.dvl:19890 ✗ Yarn ATS is not enabled ✓ HDFS Namenode URI: swebhdfs://nameservice1 ✓ Hive Metastore URI: thrift://cdpssl02.acceldata.dvl:9083 ✗ Hive LLAP is not enabled ✓ Spark History Server URIs: https://cdpssl02.acceldata.dvl:18488 ✓ Impala URI: http://cdpssl04.acceldata.dvl:25000,http://cdpssl05.acceldata.dvl:25000,http://cdpssl01.acceldata.dvl:25000 ✓ Kafka Broker URI: https://cdpssl04.acceldata.dvl:9093,https://cdpssl05.acceldata.dvl:9093,https://cdpssl03.acceldata.dvl:9093 ✓ Zookeeper Server URI: http://cdpssl01.acceldata.dvl:2181,http://cdpssl02.acceldata.dvl:2181,http://cdpssl03.acceldata.dvl:2181 Would you like to continue with the above configuration? [y/n]: : y Is Kerberos enabled in this cluster? [y/n]: : y ✓ Found Kerberos Realm: ADSRE.COM Enter your Kerberos keytab username (Must have required HDFS permissions): : hdfs INFO: min-reports is set to default value 10 INFO: Purging old config files ✓ acceldata.conf file generated successfully. Setting up Kerberos Config Setting up kerberos.. Enter the principal: : hdfs/cdpssl03.acceldata.dvl@ADSRE.COM Enter full path to the Keytab file (eg: /root/hdfs.keytab): : /data01/security/kerberos_cluster1.keytab Enter the krb5Conf file path: : /data01/security/krb5_cluster1.conf WARN: /data01/acceldata/config/users/passwd already being generated ✓ Done, Kerberos setup completed. INFO: Creating post config files INFO: Writing the .dist files INFO: Clustername : cdp1 INFO: Performing PreCheck of Files Is HTTPS Enabled in the Cluster on UI Endpoint? [Y/N]: : Y Enter the Java Keystore cacerts File Path: : /data01/security/cacerts Enter the Java Keystore jsseCaCerts File Path: : /data01/security/cacerts INFO: Setting the active cluster WARN: Cannot find the pulse.yaml file, getting the values from acceldata.conf file WARN[1090] cannot find the spark on yarn thriftserver service ports WARN[1090] Atlas Server not installed WARN[1090] Hive Server Interactive not installed Creating hydra inventory ✔ Is the agent deployment Parcel Based? [Y/N] : : Y█ pulsecdp01.acceldata.dvl is the hostname of the Pulse Server, Is this correct? [Y/N]: : y ? Select the components you would like to install: Impala, Metastore, Hdfs, HiveServer2, Zookeeper, Yarn, Hbase Is Kerberos Enabled for Impala?: y Enter the JMX Port for hive_metastore: : 8009 ✔ Enter the JMX Port for zookeeper_server: : 9010█ Enter the Kafka Broker Port: : 9092 Do you want to enable Impala Agent: [Y/N]? : Y Would you like to setup LogSearch? [y/n]: : y ? Select the logs for components that are installed/enabled in your target cluster: kafka_server, yarn_timelinereader, impala_catalogd, yarn_timelineserver, hue_runcpserver, hive_server, oozie_jpa, ranger_audit, yarn_resourcemanager, hdfs_audit, oozie_error, hbase_regionserver, hue_error, impala_impalad, hdfs_datanode, yarn_nodemanager, mapred_historyserver, hbase_master, kafka_state_change, hdfs_namenode, kafka_server_gc, kafka_controller, kafka_err, yarn_application, kafka_log_cleaner, hive_server_interactive, oozie_audit, zookeeper, oozie_tomcat, hue_migrate, hue_access, syslog, oozie_ops, oozie_server ✓ Generated the vars.yml file successfully INFO: /data01/acceldata/work/cdp1/fsanalytics/update_fsimage.sh - ✓ Done INFO: /data01/acceldata/work/cdp1/fsanalytics/kinit_fsimage.sh - ✓ Done INFO: /data01/acceldata/work/cdp1/fsanalytics/update_fsimage_csv.sh - ✓ Done Configuring notifications ✓ Generated the notifications.yml file successfully Configuring notifications ✓ Generated the actions notifications.yml file successfully INFO: Please run 'accelo deploy core' to deploy APM core using this configuration.

  1. Run the config cluster for the second cluster.

[root@pulsecdp01:acceldata (ad-default)]$ accelo config cluster INFO: Configuring the cluster ... INFO: Using default API Version v10 for CM API Is the 'Database Service' up and running? [y/n]: : n WARN: Gauntlet is running in dry run mode. Disable this to delete indices from elastic and purge data from mongo DB ✔ Cloudera Enter Your Cluster's Display Name: : cdp2 ✔ Enter Cloudera URL (with http/https): : https://10.90.9.65:7183█ The hostname for the Cloudera is : : cdp-1.iti.adsre.com ✔ Enter Cloudera Username: : admin█ IMPORTANT: This password will be securely encrypted and stored in this machine. Enter Cloudera User Password: : ***** Enter the cluster name to use (MUST be all lowercase & unique): : cdp2 Enter the cluster name to use (MUST be all lowercase & unique): : cdp2 INFO: Creating Post dirs. ✔ CDP_Odin INFO: Using lower case for CDP Service name API INFO: Using lower case for CDP Service name API INFO: Using lower case for CDP Service name API INFO: Using lower case for CDP Service name API INFO: Using lower case for CDP Service name API INFO: Using lower case for CDP Service name API INFO: Using lower case for CDP Service name API INFO: Using lower case for CDP Service name API INFO: Using lower case for CDP Service name API ✔ Enter the installed Kafka version (ex: 0.10.2): : 0.11.0█ ✔ Enter the installed HBase service version (ex: 0.9.4): : 0.9.4█ Enter the installed Hive service version (ex: 2.0.0): : 2.0.0 ✓ Found Kerberos Realm: ADSRE.COM Enter the Spark History HDFS path: : /user/spark/applicationHistory INFO: core-site.xml file has been updated INFO: hdfs-site.xml file has been updated ✔ Oozie DB URL: : jdbc:postgresql://cdp-1.iti.adsre.com:7432/oozie_oozie_server2█ Enter the Oozie DB Username: : oozie_oozie_server2 Enter the Oozie DB Password: : ********** ✔ Enter the hive metastore Database Name : : hive2█ Hive Metastore PostgreSQL DB Connection URL: : jdbc:postgresql://cdp-1.iti.adsre.com:7432/hive2 Enter the hive metastore DB Username : : hive2 ✔ Enter the hive metastore DB Password : : **********█ ---------------------------Discovered configurations---------------------------------------- ✓ Cluster Type: CDH ✓ CDH Version: 7.1.7 ✓ Discovered Cluster Name: cdp2 ✓ Discovered Services: ✓ LIVY ✓ PULSEHYDRAAGENT ✓ RANGER ✓ ATLAS ✓ HIVE_ON_TEZ ✓ QUEUEMANAGER ✓ SPARK_ON_YARN ✓ HUE ✓ KAFKA ✓ HDFS ✓ OOZIE ✓ SOLR ✓ HBASE ✓ YARN ✓ ZOOKEEPER ✓ HIVE ✓ Yarn RM URI: https://cdp-1.iti.adsre.com:8090,https://cdp-2.iti.adsre.com:8090 ✓ MapReduce Job History URI: https://cdp-1.iti.adsre.com:19890 ✗ Yarn ATS is not enabled ✓ HDFS Namenode URI: swebhdfs://nameservice123 ✓ Hive Metastore URI: thrift://cdp-1.iti.adsre.com:9083 ✗ Hive LLAP is not enabled ✓ Spark History Server URIs: https://cdp-1.iti.adsre.com:18488 ✗ Impala is not installed ✓ Kafka Broker URI: https://cdp-3.iti.adsre.com:9093,https://cdp-1.iti.adsre.com:9093,https://cdp-2.iti.adsre.com:9093 ✓ Zookeeper Server URI: http://cdp-1.iti.adsre.com:2181 Would you like to continue with the above configuration? [y/n]: : y Is Kerberos enabled in this cluster? [y/n]: : y ✓ Found Kerberos Realm: ADSRE.COM Enter your Kerberos keytab username (Must have required HDFS permissions): : hdfs INFO: min-reports is set to default value 10 INFO: Purging old config files ✓ acceldata.conf file generated successfully. Setting up Kerberos Config Setting up kerberos.. Enter the principal: : hdfs/cdp-1.iti.adsre.com@ADSRE.COM Enter full path to the Keytab file (eg: /root/hdfs.keytab): : /data01/security/kerberos_cdp2.keytab Enter the krb5Conf file path: : /data01/security/krb5_cdp2.conf WARN: /data01/acceldata/config/users/group already being generated WARN: /data01/acceldata/config/users/passwd already being generated ✓ Done, Kerberos setup completed. INFO: Creating post config files INFO: Writing the .dist files INFO: Clustername : cdp2 INFO: Performing PreCheck of Files Is HTTPS Enabled in the Cluster on UI Endpoint? [Y/N]: : Y Enter the Java Keystore cacerts File Path: : /data01/security/cacerts Enter the Java Keystore jsseCaCerts File Path: : /data01/security/cacerts Enter the Java Keystore jsseCaCerts File Path: : /data01/security/cacerts INFO: Setting the active cluster WARN[0541] Atlas Server not installed WARN[0541] cannot find the spark on yarn thriftserver service ports WARN[0541] Hive Server Interactive not installed Creating hydra inventory ✔ pulsecdp01.acceldata.dvl is the hostname of the Pulse Server, Is this correct? [Y/N]: : y█ ? Select the components you would like to install: HiveServer2, Hdfs, Zookeeper, Yarn, Hbase, Metastore Enter the JMX Port for hive_metastore: : 8009 Enter the JMX Port for hive_server: : 8008 Enter the JMX Port for zookeeper_server: : 9010 Enter the Kafka Log Dirs: : /var/local/kafka/data Enter the Kafka JMX Port (Ex: 9999): : 9393 Would you like to install Kapxy? [y/n]: : n Would you like to enable NTP Stats? [y/n]: : y Would you like to setup LogSearch? [y/n]: : y ? Select the logs for components that are installed/enabled in your target cluster: kafka_log_cleaner, oozie_tomcat, hbase_regionserver, hdfs_datanode, oozie_ops, kafka_server, hue_access, mapred_historyserver, hue_migrate, zookeeper, oozie_error, oozie_server, hive_server, hdfs_namenode, oozie_jpa, yarn_timelineserver, oozie_audit, hbase_master, yarn_resourcemanager, ranger_audit, kafka_server_gc, kafka_state_change, yarn_timelinereader, syslog, yarn_nodemanager, kafka_err, hue_error, hdfs_audit, hive_server_interactive, kafka_controller, yarn_application, hue_runcpserver ✓ Generated the vars.yml file successfully INFO: /data01/acceldata/work/cdp2/fsanalytics/kinit_fsimage.sh - ✓ Done INFO: /data01/acceldata/work/cdp2/fsanalytics/update_fsimage_csv.sh - ✓ Done INFO: /data01/acceldata/work/cdp2/fsanalytics/update_fsimage.sh - ✓ Done Configuring notifications ✓ Generated the notifications.yml file successfully Configuring notifications ✓ Generated the actions notifications.yml file successfully INFO: Please run 'accelo deploy core' to deploy APM core using this configuration.

  1. Perform the below mentioned steps for both the clusters.

  2. Edit the acceldata.conf file for the changes for MultiNode deployment.

vi $AcceloHome/config/acceldata_<clustername>.conf

  1. Update the elastic section of the connections collection.

elastic = [ { name = "default" host = "<PULSE SERVER FQDN WHERE THE LOGSEARCH IS DEPLOYED>" port = <ELASTIC PORT> }, { name = "fsanalytics" host = "<PULSE SERVER FQDN WHERE THE FSELASTIC IS DEPLOYED>" port = <FSELASTIC PORT NUMBER> }, { name = "nifi" host = "ad-elastic" port = 9200 } ],

  1. Save the file.

  2. Change the directory to work/<clustername>

cd $AcceloHome/work/<clustername>

  1. Create the override.yml file if not yet created.

vi override.yml

  1. Enter the following code and edit as required:

base: logstash_url: <LOGSEARCH HOSTNAME>:19012 yarn_app_es_urls: http://<LOGSERCH HOSTNAME>:19051

  1. Save the file.

Deploy Core

  1. Deploy the Pulse core components by running the following command:

accelo deploy core

OUTPUT

[root@nifihost1:acceldata (ad-default)]$ accelo deploy core ERROR: Cannot connect to DB, Because: cannot connect to mongodb WARN: Gauntlet is running in dry run mode. Disable this to delete indices from elastic and purge data from mongo DB Have you verified the acceldata config file at '/data01/acceldata/config/acceldata_spark341.conf' ? [y/n]: : y ✓ accelo.yml file found and parsed ✓ AcceloEvents - events.json file found and parsed ✓ acceldata conf file found and parsed ✓ .dist file found and parsed ✓ hydra_hosts.yml file found and parsed ✓ vars.yml file found and parsed ✓ alerts notification.yml file found and parsed ✓ actions notification.yml file found and parsed ✓ alerts default-endpoints.yml file found and parsed ✓ override.yml file found and parsed ✓ gauntlet_mongo_spark341.yml file found and parsed ✓ gauntlet_elastic.yml file found and parsed INFO: No existing AccelData networks found. Current stack 'ad-default' is missing. INFO: Trying to create a new network .. INFO: If you're setting up AccelData for the first time give 'y' to the below. Would you like to initiate DB with the config file '/data01/acceldata/config/acceldata'? [y/n]: : y Creating group monitors [================================================================================================>-------------------] 83.33%INFO: Pushing the hydra_hosts.yml to mongodb Deployment Completed [==============================================================================================>--------------------] 81.82% 28s ✓ Done, Core services deployment completed. Now, you can access the AccelData APM Server at the configured port of this node. To deploy the AccelData addons, Run './accelo deploy addons'

  1. Push the configurations for all the clusters.

accelo admin database push-config -a

Configure SSL for Pulse Services and Connectors

If you have TLS/SSL enforced for any of the Hadoop components in the target cluster, you need to place the certificate and certificate password files in the cd /data01/acceldata/config/security path.

For details, see Manage SSL Certificates for Pulse Services and Connectors.

Deploy Add-ons

accelo deploy addons

OUTPUT

[root@nifihost1:acceldata (ad-default)]$ accelo deploy addons WARN: Gauntlet is running in dry run mode. Disable this to delete indices from elastic and purge data from mongo DB INFO: Active Cluster: spark341 ? Select the components you would like to install: Alerts (Agents MUST be configured), Core Connectors, Dashplot, Director (Agents MUST be configured), HYDRA, LogSearch, Notifications Starting the deployment .. Completed [==============================================================================================================================] 137.50% 29s ✓ Done, Addons deployment completed. [root@pulsecdp01:~ (ad-default)]$ docker ps | grep cdp1 fb489fdb8c09 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-sparkstats:3.3.3-beta "/opt/docker/bin/spa…" 18 minutes ago Up 18 minutes ad-sparkstats_cdp1_default 3cae513b8910 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-connectors:3.3.3-beta "/opt/docker/bin/ad-…" 18 minutes ago Up 18 minutes ad-connectors_cdp1_default c8f128057341 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-kafka-connector:3.3.3-beta "/opt/docker/bin/ad-…" 18 minutes ago Up 18 minutes ad-kafka-connector_cdp1_default
accelo set

Deploy the Pulse add-ons, and select the components that are needed for CDP Cluster2.

accelo deploy addons

OUTPUT

[root@nifihost1:acceldata (ad-default)]$ accelo deploy addons WARN: Gauntlet is running in dry run mode. Disable this to delete indices from elastic and purge data from mongo DB INFO: Active Cluster: spark341 ? Select the components you would like to install: Alerts (Agents MUST be configured), Core Connectors, Dashplot, Director (Agents MUST be configured), HYDRA, LogSearch, Notifications Starting the deployment .. Completed [==============================================================================================================================] 137.50% 29s ✓ Done, Addons deployment completed. [root@pulsecdp01:~ (ad-default)]$ docker ps | grep cdp2 73fafe32a650 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-kafka-connector:3.3.3-beta "/opt/docker/bin/ad-…" 6 hours ago Up 6 hours ad-kafka-connector_cdp2_default f089aca93f9f 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-sparkstats:3.3.3-beta "/opt/docker/bin/spa…" 6 hours ago Up 6 hours ad-sparkstats_cdp2_default 83d707813431 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-connectors:3.3.3-beta "/opt/docker/bin/ad-…" 6 hours ago Up 6 hours ad-connectors_cdp2_default

Database Push Configuration

To push the configuration to the database, run the following code:

accelo admin datbase push-config -a

Updating Gauntlet in Dry Run Mode

To update Gauntlet in dry run mode, perform the following:

  1. Check if the ad-core.yml file is present or not by running the following command:

ls -al $AcceloHome/config/docker/ad-core.yml

  1. If the above file is not present then generate it by running the following command:

accelo admin makeconfig ad-core

  1. Edit the ad-core.yml file by performing the following:

a. Open the file.

vi $AcceloHome/config/docker/ad-core.yml

b. Update the DRY_RUN_ENABLE environment variable in the ad-gauntlet section as shown below:

DRY_RUN_ENABLE=false

Note This makes Gauntlet delete the older elastic indices and MongoDB data.

c. The updated file must appear as shown below:

ad-gauntlet: image: ad-gauntlet container_name: ad-gauntlet environment: - MONGO_URI=ZN4v8cuUTXYvdnDJIDp+R8Z+ZsVXXjv8zDOvh8UwQXosC8vfVkGYGWGPNnX64ZVSp9yHgErQknPBAfYZ9cOG1A== - MONGO_ENCRYPTED=true - ELASTIC_ADDRESSES=http://ad-elastic:9200 - DRY_RUN_ENABLE=false - CRON_TAB_DURATION=*/5 * * * * volumes: - /etc/localtime:/etc/localtime:ro - /root/acceldata/config/logsearch/gauntlet_elastic.yml:/gauntlet/config/config.yml - /root/acceldata/logs/logsearch/:/gauntlet/logs/ ulimits: {} ports: [] depends_on: [] opts: {} restart: "" extra_hosts: [] network_alias: []

d. Save the file.

  1. Restart Gauntlet service by running the following command:

accelo restart ad-gauntlet

Configure Gauntlet

To update the Gauntlet Contrab duration, perform the following:

  1. Check if the ad-core.yml file is present or not by running the following command:

ls -al $AcceloHome/config/docker/ad-core.yml

  1. If the above file is not present then generate it by running the following command:

accelo admin makeconfig ad-core

  1. Edit the ad-core.yml file by performing the following:

a. Open the file

vi $AcceloHome/config/docker/ad-core.yml

b. Update the CRON_TAB_DURATION environment variable in the ad-gauntlet section as shown below:

CRON_TAB_DURATION=*/5 * * * *

Note This makes Gauntlet run every two days at midnight.

The updated file must appear as shown below:

ad-gauntlet: image: ad-gauntlet container_name: ad-gauntlet environment: - MONGO_URI=ZN4v8cuUTXYvdnDJIDp+R8Z+ZsVXXjv8zDOvh8UwQXosC8vfVkGYGWGPNnX64ZVSp9yHgErQknPBAfYZ9cOG1A== - MONGO_ENCRYPTED=true - ELASTIC_ADDRESSES=http://ad-elastic:9200 - DRY_RUN_ENABLE=true - CRON_TAB_DURATION=*/5 * * * * volumes: - /etc/localtime:/etc/localtime:ro - /root/acceldata/config/logsearch/gauntlet_elastic.yml:/gauntlet/config/config.yml - /root/acceldata/logs/logsearch/:/gauntlet/logs/ ulimits: {} ports: [] depends_on: [] opts: {} restart: "" extra_hosts: [] network_alias: []

c. Save the file.

  1. Restart the Gauntlet service by running the following command:

accelo restart ad-gauntlet

Configuring Gauntlet for Multi-node and Multi-cluster Deployment

Perform the following:

  1. To generate the Gauntlet config files, run the following command:

accelo admin database push-config -s -a

  1. Change the directory to config/gauntlet/

cd $AcceloHome/config/gauntlet

  1. Check if all the files are present or not for all the clusters or not:

[root@cdp5007:gauntlet (ad-default)]$ accelo admin database push-config -a -s Is the 'Database Service' up and running? [y/n]: : y INFO: Working on cluster: cl1 Creating group monitors [========================================================================================================================================================================================================>----------------------------------------] 83.33%INFO: Pushing the hydra_hosts.yml to mongodb INFO: Pushing the LDAP configuration to the mongo DB Done [=====================================================================================>---------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 33.33% 0s Push completed successfully! INFO: Working on cluster: cl2 Creating group monitors [========================================================================================================================================================================================================>----------------------------------------] 83.33%INFO: Pushing the hydra_hosts.yml to mongodb INFO: Pushing the LDAP configuration to the mongo DB Done [=====================================================================================>---------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 33.33% 0s Push completed successfully! [root@cdp5007:gauntlet (ad-default)]$ ls -al total 28 drwxr-xr-x. 2 root root 130 Nov 24 09:58 . drwxr--r--. 14 root root 4096 Nov 24 09:56 .. -rw-r--r--. 1 root root 866 Nov 24 09:56 gauntlet_elastic_cl1.yml -rw-r--r--. 1 root root 866 Nov 24 09:56 gauntlet_elastic_cl2.yml -rw-r--r--. 1 root root 6404 Nov 24 09:58 gauntlet_mongo_cl1.yml -rw-r--r--. 1 root root 6404 Nov 24 09:58 gauntlet_mongo_cl2.yml

  1. Modify the gauntlet_elastic_<clustername>.yml file by running the file:

vi gauntlet_elastic_<clustername>.yml

  1. Edit the elastic address in the file for multi-node setup.

INFO: Working on cluster: cl1 version: 1 elastic_servers: - version: v8 address: "http://<Elastic Server Hostname>:<Elastic Server Port>" basic_auth: true username: "pulse" #EncryptedPassword password: "pPBrVKaoB0QsmCJZNZyYAw==" enable_tls: false client_certificate_path: "" client_key_path: "" client_ca_cert: ""

  1. Modify the Elastic address for both the clusters.

  2. Push the configuration to the database:

accelo admin database push-config -a

  1. Restart the Gauntlet service:

accelo restart ad-gauntlet

Updating MongoDB Clean Up and Compaction Frequency In Hours

By default, when dry run is disabled, MongoDB cleanup and compaction will occur once a day. To adjust the frequency, perform the following:

  1. Run the following command:

accelo config retention

  1. Answer the following prompts, if you are unsure about the number of days you wish to retain, then proceed with the default values.

✔ How many days of data would you like to retain at Mongo DB ?: 15 ✔ How many days of data would you like to retain at Mongo DB for HDFS reports ?: 15 ✔ How many days of data would you like to retain at TSDB ?: 31

  1. When presented with the following prompt, indicate the hours of the day when you want MongoDB cleanup and compaction to occur. The value must be a comma-separated list of hours in accordance with the 24-hour time notation.

✔ How often should Mongo DB clean up & compaction run, provide a comma separated string of hours (valid values are [0,23] (Ex. 8,12,15,18)?: 0,6,12,18

  1. Execute the following command, and when Gauntlet runs the next time, MongoDB cleanup and compaction will be scheduled to run at the specified hours, once per hour:

accelo admin database push-config

Configure and Deploy FSAnlytics in the Second Pulse Server

To configure and deploy FSAnalytics in the second Pulse server, perform the following:

  1. Create the acceldata directory by running the following command:

cd /data01/ mkdir -p acceldata

  1. Place the accelo.linux binary in the /data01/acceldata directory:

cp </path/to/accelo/binary> /data01/acceldata

  1. Rename the accelo.linux binary to accelo

mv /data01/acceldata/accelo.linux accelo chmod +x /data01/acceldata/accelo

  1. Change the directory.

cd /data01/acceldata/accelo

  1. Run the following command to run accelo init.

./accelo init

  1. Provide appropriate answers when prompted.

  2. Source the ad.sh file

source /etc/profile.d/ad.sh

  1. To enter the Pulse version, run the init command:

./accelo init

OUTPUT

[root@nifihost1:~ (ad-default)]$ accelo init Enter the AccelData ImageTag: : 3.3.3 ✓ Done, AccelData Init Successful.

Note Provide the correct Pulse version number, in this case it is 3.3.3.

  1. Run accelo info to get the initial information.

accelo info

OUTPUT

[root@nifihost1:~ (ad-default)]$ accelo info WARN: Gauntlet is running in dry run mode. Disable this to delete indices from elastic and purge data from mongo DB ___ ____________________ ____ ___ _________ / | / ____/ ____/ ____/ / / __ \/ |/_ __/ | / /| |/ / / / / __/ / / / / / / /| | / / / /| | / ___ / /___/ /___/ /___/ /___/ /_/ / ___ |/ / / ___ | /_/ |_\____/\____/_____/_____/_____/_/ |_/_/ /_/ |_| Accelo CLI Version: 3.3.3 Accelo CLI Build Hash: 8ba4727f11e5b3f3902547585a37611b6ec74e7c Accelo CLI Build ID: 1700746329 Accelo CLI Builder ID: ZEdjMmxrYUdGdWRGOWhZMk5sYkdSaEVLCg== Accelo CLI Git Branch Hash: TXdLaTlCVDFBdE56STNvPQo= AcceloHome: /data01/acceldata AcceloStack: ad-default AccelData Registry: 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata AccelData ImageTag: 3.3.3 Active Cluster Name: NotFound AcceloConfig Mongo DB Retention days: 15 AcceloConfig Mongo DB HDFS Reports Retention days: 15 AccelConfig TSDB Retention days: 31d Number of AccelData stacks found in this node: 0

  1. Get the Pulse Master hostname and generate the Mongo URL by editing the below code:

mongodb://accel:<MONGO_PASSWORD>@<PULSE_MASTER_HOST>:27017

  1. Encrypt the above string by running the following command and provide the string when prompted:

accelo admin encrypt

  1. Edit the ad.sh for enabling the Pulse Standalone deployment. Add the following information, to the ad.sh.

vi /etc/profile.d/ad.sh

  1. Replace the MONGO_URI with the encrypted string obtained from step 10.

#!/bin/sh export PULSE_SA_NODE='true' export AcceloHome='/data01/acceldata' export AcceloStack='ad-default' export PATH=${PATH}:${AcceloHome} export MONGO_URI= export MONGO_ENCRYPTED=true export PS1='[\u@\h:\W (ad-default)]$ '

  1. Source the file.

source /etc/profile.d/ad.sh

  1. Now set the cluster.

accelo set

  1. Copy the fsanalytics directory from the Pulse Master Server present in the below location:

$AcceloHome/work/<clustername>/fsanalytics

Add the following to the below directory in the second cluster:

$AcceloHome/work/<clustername>/fsanalytics

Note Create the directory if not present.

  1. Copy the /krb/security directory from the Pulse Master Server present in the below location:

$AcceloHome/work/<clustername>/krb/security

Add the following to the below directory in the second cluster:

$AcceloHome/work/<clustername>/krb/security

Note Create the directory if not present.

  1. Generate the ad-fsanalyticsv2-connector.yml.

accelo admin makeconfig ad-fsanalyticsv2-connector

OUTPUT

[root@pulsecdp02:acceldata (ad-default)]$ accelo admin makeconfig ad-fsanalyticsv2-connector WARN: Gauntlet is running in dry run mode. Disable this to delete indices from elastic and purge data from mongo DB ✓ Done, Configuration file generated IMPORTANT: Please edit/verify the file '/data01/acceldata/config/docker/addons/ad-fsanalyticsv2-connector.yml'. If the addon is already up and running, use './accelo deploy addons' to remove and recreate the addon service.

  1. Edit the file.

vi /data01/acceldata/config/docker/addons/ad-fsanalyticsv2-connector.yml

  1. Update the following environment variables to the ad-fs-elastic:

  • MONGO_URI (Acceldata team will provide the right URI)

  • MONGO_ENCRYPTED=false

  • ES_HOST=<host_running_ES>

  • ES_PORT=19013

version: "2" services: ad-fsanalyticsv2-connector: image: ad-fsanalyticsv2-connector container_name: "" environment: - FSANALYTICS_TREE_VISIT_LEVEL2_PARALLELISM=32 - MONGO_SECRET=Ah+MqxeIjflxE8u+/wcqWA== - MONGO_URI=ZN4v8cuUTXYvdnDJIDp+R8Z+ZsVXXjv8zDOvh8UwQXqyScAm+LrS8Y9EWT8A8/30 - JAAS_LOGIN_CONF_LOCATION=/krb/security/krb5JAASLogin.conf - FSANALYTICS_RUN_MODE=raw - KRB5_CONFIG=/krb/security/krb5.conf - FSANALYTICS_IMAGE_PARSING_PARALLELISM=32 - AD_STREAMING=DEFAULT - KRB5_CONF_LOCATION=/krb/security/krb5.conf - JAVA_OPTS=-XX:+UseG1GC -XX:+UseStringDeduplication - ES_HOST=ad-fs-elastic - ES_PORT=9200 - FSANALYTICS_ENABLE_OFFHEAP_MAP=true - FSANALYTICS_IMAGE_PARSING_BATCHSIZE=400000 - FSANALYTICS_TREE_VISIT_LEVEL3_PARALLELISM=64 - MONGO_ENCRYPTED=true - FSANALYTICS_TREE_VISIT_LEVEL1_PARALLELISM=8 - KERBEROS_ENABLED=true volumes: - /etc/localtime:/etc/localtime:ro - /etc/hosts:/etc/hosts:ro - /data01/acceldata/data/fsanalytics:/etc/fsanalytics - /data01/acceldata/config/krb/security:/krb/security - /data01/acceldata/work:/opt/docker/work ulimits: {} ports: - 19027:19027 depends_on: [] opts: {} restart: "" extra_hosts: [] network_alias: [] label: FS Analytics V2

  1. Append below hostname entry in /etc/hosts and check if the /etc/hosts file is mounted under volume section in above created file (<ACCELO_HOME>/config/docker/addons/ad-fsanalyticsv2-connector.yml)

    1. <PULSE_CORE_HOST> ad-streaming

  2. Run the deploy add-ons command and select FSAnalytics and FSElastic.

accelo deploy addons

OUTPUT

[root@pulsecdp02:acceldata (ad-default)]$ accelo deploy addons WARN: Gauntlet is running in dry run mode. Disable this to delete indices from elastic and purge data from mongo DB INFO: Active Cluster: cdp1 ? Select the SA components you would like to install: FS Elastic, FS Analytics V2 Enter the NameNode URI: : https://cdpss02.acceldata.dvl:8090 Starting the deployment .. Deploying ad-fsanalyticsv2-connector service [=============================================================================================================>--------------------------------------------------------------------------------------------------------------] 50.00%WARN: Files/Dir not found : /data01/acceldata/data/fs-elastic Completed [============================================================================================================================================================================================================================================================] 100.00% 1s ✓ Done, Addons deployment completed. [root@pulsecdp02:~ (ad-default)]$ docker ps | grep cdp1 8cbd4cb3f25c 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-fsanalyticsv2-connector:3.3.3-beta "/opt/docker/bin/ad-…" 2 hours ago Up 2 hours 0.0.0.0:19027->19027/tcp, :::19027->19027/tcp ad-fsanalyticsv2-connector_cdp1_default

  1. Since the FSAnalyticsV2 Connector has a port exposed to the outside, you will need to modify the port bound to the host. To do this, open the ad-fsanalyticsv2-connector.yml file.

vi $AcceloHome/config/docker/addons/ad-fsanalyticsv2-connector.yml

  1. Update the port section of the file.

version: "2" services: ad-fsanalyticsv2-connector: image: ad-fsanalyticsv2-connector container_name: "" environment: - FSANALYTICS_TREE_VISIT_LEVEL2_PARALLELISM=32 - MONGO_SECRET=Ah+MqxeIjflxE8u+/wcqWA== - MONGO_URI=ZN4v8cuUTXYvdnDJIDp+R8Z+ZsVXXjv8zDOvh8UwQXqyScAm+LrS8Y9EWT8A8/30 - JAAS_LOGIN_CONF_LOCATION=/krb/security/krb5JAASLogin.conf - FSANALYTICS_RUN_MODE=raw - KRB5_CONFIG=/krb/security/krb5.conf - FSANALYTICS_IMAGE_PARSING_PARALLELISM=32 - AD_STREAMING=DEFAULT - KRB5_CONF_LOCATION=/krb/security/krb5.conf - JAVA_OPTS=-XX:+UseG1GC -XX:+UseStringDeduplication - ES_HOST=ad-fs-elastic - ES_PORT=9200 - FSANALYTICS_ENABLE_OFFHEAP_MAP=true - FSANALYTICS_IMAGE_PARSING_BATCHSIZE=400000 - FSANALYTICS_TREE_VISIT_LEVEL3_PARALLELISM=64 - MONGO_ENCRYPTED=true - FSANALYTICS_TREE_VISIT_LEVEL1_PARALLELISM=8 - KERBEROS_ENABLED=true volumes: - /etc/localtime:/etc/localtime:ro - /etc/hosts:/etc/hosts:ro - /data01/acceldata/data/fsanalytics:/etc/fsanalytics - /data01/acceldata/config/krb/security:/krb/security - /data01/acceldata/work:/opt/docker/work ulimits: {} ports: - 19029:19027 depends_on: [] opts: {} restart: "" extra_hosts: [] network_alias: [] label: FS Analytics V2

  1. Save the file.

  2. Set the cluster to the second cluster.

accelo set

  1. Run the deploy add-ons command and select FSAnalyticsV2 Connector add-on.

accelo deploy addons

OUTPUT

[root@pulsecdp02:fsanalytics (ad-default)]$ accelo deploy addons WARN: Gauntlet is running in dry run mode. Disable this to delete indices from elastic and purge data from mongo DB INFO: Active Cluster: cdp2 ? Select the SA components you would like to install: FS Elastic, FS Analytics V2 Starting the deployment .. Completed [============================================================================================================================================================================================================================================================] 100.00% 0s ✓ Done, Addons deployment completed. 15b78f10be66 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-fsanalyticsv2-connector:3.3.3-beta "/opt/docker/bin/ad-…" 2 hours ago Up 2 hours 0.0.0.0:19029->19027/tcp, :::19029->19027/tcp ad-fsanalyticsv2-connector_cdp2_default

  1. Check if the two connectors are running or not.

docker ps

OUTPUT

[root@pulsecdp02:fsanalytics (ad-default)]$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 15b78f10be66 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-fsanalyticsv2-connector:3.3.3-beta "/opt/docker/bin/ad-…" 2 seconds ago Up 1 second 0.0.0.0:19029->19027/tcp, :::19029->19027/tcp ad-fsanalyticsv2-connector_cdp2_default 8cbd4cb3f25c 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-fsanalyticsv2-connector:3.3.3-beta "/opt/docker/bin/ad-…" 4 minutes ago Up 4 minutes 0.0.0.0:19027->19027/tcp, :::19027->19027/tcp ad-fsanalyticsv2-connector_cdp1_default b141fd86e4b3 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-elastic:3.3.3-beta "/bin/tini -- /usr/l…" 4 minutes ago Up 4 minutes 0.0.0.0:19038->9200/tcp, :::19038->9200/tcp, 0.0.0.0:19039->9300/tcp, :::19039->9300/tcp ad-fs-elastic_default

  1. Check if both the containers are bound to 19027 and 19029 ports respectively.

docker ps | grep -i ad-fsanalyticsv2-connector

  1. For running fsa load, do set the following:

accelo admin fsa load

  1. Set the cluster to the second cluster.

accelo set

  1. Load the second cluster using the following fsa command:

accelo admin fsa load

Enabling (TLS) HTTPS for Pulse Web UI Configuration

For details on the configuration, see Enable Native SSL/TLS for Pulse Web UI.

Set up LDAP for the Pulse User Interface

For details on the configuration, see Set up LDAP for Pulse UI.

Configure Alerts Notifications

To configure alerts notifications, perform the following:

  1. To set the active cluster, run the following command:

accelo set

  1. Configure the alerts notifications by running the following command:

accelo config alerts notifications

OUTPUT

[root@nifihost1:acceldata (ad-default)]$ accelo config alerts notifications Enter the JODA Timezone value (Example: Asia/Jakarta): : Asia/Kolkata ? Select the metric groups you would like to enable: druid, nifi, ntpd, anomaly, chrony, customApp ? Select the notifications you would like to enable: email INFO: Configuring Email Notifications: Enter Email DefaultToEmailIds (comma separated list): : Enter Email DefaultSnoozeIntervalInSecs: : 0 Enter Email MaxEmailThreshold: : 1 ✓ Done, Alerts Notifications Configuration file generated ✓ Done, Alerts Notifications pushed to Pulse DB

  1. Set the cluster2 as the active cluster.

accelo set

  1. Configure alerts for the second cluster as shown below:

[root@nifihost1:acceldata (ad-default)]$ accelo config alerts notifications Enter the JODA Timezone value (Example: Asia/Jakarta): : Asia/Kolkata ? Select the metric groups you would like to enable: druid, nifi, ntpd, anomaly, chrony, customApp ? Select the notifications you would like to enable: email INFO: Configuring Email Notifications: Enter Email DefaultToEmailIds (comma separated list): : Enter Email DefaultSnoozeIntervalInSecs: : 0 Enter Email MaxEmailThreshold: : 1 ✓ Done, Alerts Notifications Configuration file generated ✓ Done, Alerts Notifications pushed to Pulse DB

  1. Set the cluster3 as the active cluster by running the following code:

accelo set

  1. Configure the alerts for the third cluster as shown below:

[root@nifihost1:acceldata (ad-default)]$ accelo config alerts notifications Enter the JODA Timezone value (Example: Asia/Jakarta): : Asia/Kolkata ? Select the metric groups you would like to enable: druid, nifi, ntpd, anomaly, chrony, customApp ? Select the notifications you would like to enable: email INFO: Configuring Email Notifications: ✔ Enter Email DefaultSnoozeIntervalInSecs: : 0█ mEnter Email MaxEmailThreshold: : 11█ ✔ Enter Email MaxEmailThreshold: : 1█ ✓ Done, Alerts Notifications Configuration file generated ✓ Done, Alerts Notifications pushed to Pulse DB

  1. Restart the alerts notifications.

accelo restart ad-alerts

OUTPUT

[root@nifihost1:spark341 (ad-default)]$ accelo restart ad-alerts WARN: Gauntlet is running in dry run mode. Disable this to delete indices from elastic and purge data from mongo DB ✗ You're about to restart AccelData services. This will restart all or any specified the service. However, any persistent data will be left untouched. ✔ You're about to restart AccelData services. This will restart all or any specified the service. However, any persistent data will be left untouched. ✔ You're about to restart AccelData services. This will restart all or any specified the service. However, any persistent data will be left untouched. You're about to restart AccelData services. This will restart all or any specified the service. However, any persistent data will be left untouched. Please confirm your action [y/n]: : y Completed [===============================================================================================================================] 100.00% 1s Restart ad-alerts completed ✓


Next to read:
Multi-Node Setup for NATS

For additional help, contact www.acceldata.force.com OR call our service desk +1 844 9433282

Copyright © 2026

On This Page
CDP Multi-node, Multi-KDC, Multi-ClusterPrerequisitesUninstallation of AgentsDownload and Load the Binaries and Docker ImagesCluster ConfigurationDeploy CoreConfigure SSL for Pulse Services and ConnectorsDeploy Add-onsDatabase Push ConfigurationUpdating Gauntlet in Dry Run ModeConfiguring Gauntlet for Multi-node and Multi-cluster DeploymentUpdating MongoDB Clean Up and Compaction Frequency In HoursConfigure and Deploy FSAnlytics in the Second Pulse ServerEnabling (TLS) HTTPS for Pulse Web UI ConfigurationSet up LDAP for the Pulse User InterfaceConfigure Alerts NotificationsConfigure Gauntlet