Non CDP - Hydra Services

Deploy Pulse Hydra Agent

There are three ways to install the Hydra Agent across all nodes in a cluster:

• With SSH Enabled: If SSH is enabled for all nodes, you can deploy the Hydra Agent using the Ansible playbook.
• Without SSH Enabled: If SSH is not enabled, the agent must be installed manually on each node, which is an inefficient approach.
• Using the Ambari Mpack: Ambari uses its internal implementation to install custom services to any of the nodes of the cluster through Mpack. This allows Hydra to be installed in all nodes through a single click from Ambari.

With SSH Access

If you have the SSH access to the cluster nodes use the following steps:

1. Deploy the Pulse Hydra agent by executing the following steps.
   1. Execute the following deploy command, here -f argument defines parallel ssh session to deploy agents
   2. Type y when the Have you run 'accelo configure agents' ?[y/n] question is displayed.
   3. Press the enter key.

xxxxxxxxxx

accelo deploy hydra -f 5

Without the SSH Access

If you do not have the SSH access to the cluster nodes, contact the Acceldata Support Team for offline Hydra agent packages (RPM or DEB) that you can install to all the cluster nodes with your preferred configuration management tool of your choice.

Install Agent(s)

1. Download and upload hystaller binary on mentioned node from provided link and credentials.
2. Place hystaller on /tmp directory and follow the below steps of command sequences, here replace <PULSE_HOSTNAME> to hostname running Pulse UI (main server).

chmod +x /tmp/hystaller

PULSE_HOME="/opt/pulse"

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin"

HYDRA_SERVER_URL="http://<PULSE_HOSTNAME>:19072"

HYDRA_HEARTBEAT_DURATION="60"

HYDRA_PARCEL_MODE="False"

HYDRA_HOSTNAME_CASE="lower"

HYDRA_HOSTNAME_METHOD="CMD"

HYDRA_HEARTBEAT_JITTER="10"

#sudo if this is non-root user

sudo env  "PULSE_HOME=$PULSE_HOME" "PATH=$PATH" "HYDRA_SERVER_URL=$HYDRA_SERVER_URL" "HYDRA_HEARTBEAT_DURATION=$HYDRA_HEARTBEAT_DURATION" "HYDRA_PARCEL_MODE=$HYDRA_PARCEL_MODE" "HYDRA_HOSTNAME_CASE=$HYDRA_HOSTNAME_CASE" "HYDRA_HOSTNAME_METHOD=$HYDRA_HOSTNAME_METHOD" "HYDRA_HEARTBEAT_JITTER=$HYDRA_HEARTBEAT_JITTER" /tmp/hystaller install

3. SSH to main Pulse server running UI and run below command to sync configuration from Pulse server to new node.

xxxxxxxxxx

accelo reconfig cluster

Uninstall Agent(s)

1. Using the same hystaller agent can be uninstalled

xxxxxxxxxx

/tmp/hystaller uninstall

Deploy Pulse Hydra using the Ambari Mpack

Prerequisites

Tested Versions:

• Ambari: 2.7.4.0, 2.6.2.2, 2.7.6.0
• Cluster: HDP 3.1, HDP 2.6, ODP 3.2

Deployment of Hydra

1. Download the Tarball in Ambari server and save it as /tmp/hystaller_mpack<version>.tar.gz .

xxxxxxxxxx

wget <S3 LINK FROM LICENSE UI> -O "hystaller_mpack<version>.tar.gz"

2. Install hystaller_mpack<version>.tar.gz in ambari-server .

xxxxxxxxxx

ambari-server install-mpack --mpack=/tmp/hystaller_mpack<version>.tar.gz -v

3. Add the Hydra Service symlink in the service list of Ambari.

cd /var/lib/ambari-server/resources/stacks/<Cluster_type>/<Cluster_version>/services

ln -s /var/lib/ambari-server/resources/mpacks/ambari-hydra-agent-mpack-<hydra_version>/common-services/PULSE_HYDRA_AGENT/<hydra_version> PULSE_HYDRA_AGENT

4. Restart ambari-server.

xxxxxxxxxx

ambari-server restart

It must now restart the server with logs stating Server started listening on 8080.

5. Navigate to the Ambari UI and select Add Service .

6. Select Hydra from the services list and select the nodes to deploy hydra.

7. Edit the configurations as per the requirements.

8. Select Deploy to install and start Hydra in respective hosts.

9. Once deployment is successful, we can get the status of each hydra agent as well as the status of pulsenode, pulselog, pulsejmx.

Installation of Hydra Agent on Newly Added Hosts

The Hydra Agent must be installed on at least one node in a cluster.

1. Run the following commands on Ambari node.

curl -s -u admin:admin http://0.0.0.0:8080/api/v1/hosts|grep host_name|sed -n 's/.*"host_name" : "\([^\"]*\)".*/\1/p'>hostcluster.txt

while read line ; do j=$line;  mkdir -p $j;  curl -u admin:admin -H "X-Requested-By:ambari" -i -X  POST http://0.0.0.0:8080/api/v1/clusters/<CLUSTER_NAME>/hosts/$j/host_components/PULSE_HYDRA_AGENT; done < hostcluster.txt

while read line ; do j=$line;  mkdir -p $j;  curl -u admin:admin -H "X-Requested-By:ambari" -i -X  PUT -d '{"HostRoles": {"state": "INSTALLED"}}' http://0.0.0.0:8080/api/v1/clusters/<CLUSTER_NAME>/hosts/$j/host_components/PULSE_HYDRA_AGENT; done < hostcluster.txt

while read line ; do j=$line;  mkdir -p $j;  curl -u admin:admin -H "X-Requested-By:ambari" -i -X  PUT -d '{"HostRoles": {"state": "STARTED"}}' http://0.0.0.0:8080/api/v1/clusters/<CLUSTER_NAME>/hosts/$j/host_components/PULSE_HYDRA_AGENT; done < hostcluster.txt

The Hydra Agent is installed on all the nodes. Restart if required.

Test your Agent Configuration Changes

If you made any changes to telegraph(node agent), filebeat(logs agent) or jmx(jmx agent), and you want to test your feature, then:

1. Build the Hydra test image from bitbucket and tag to running ad-hydra container and restart ad-hydra.
2. From the Ambari server, stop the Hydra service.
3. Delete the Hydra service.
4. Add the Hydra Service, follow the above Images and provide Hydra Server URL, Hydra Installer Repository, PULSE UI URL and deploy.
5. From Pulse node, do reconfig cluster, wait for ~2 minutes.
6. Then check your changes on the nodes for your changes.

Uninstallation of Hydra Agent

1. Stop all the Hydra Agents from the Ambari UI and delete the Hydra Service.
2. Run the uninstall mpack command on CLI of Ambari host .

xxxxxxxxxx

ambari-server uninstall-mpack --mpack-name=ambari-hydra-agent-mpack -v

Validate Pulse Installations

Run the following command to validate Pulse installation.

xxxxxxxxxx

accelo info

You get output as shown in the following image.