Install Agents: Non-CDP - Hydra Services

Deploy Pulse Hydra Agent

There are three ways to install the Hydra Agent across all nodes in a cluster:

• With SSH Enabled: If SSH is enabled for all nodes, you can deploy the Hydra Agent using the Ansible playbook.
• Without SSH Enabled: If SSH is not enabled, the agent must be installed manually on each node, which is an inefficient approach.
• Using the Ambari Mpack: Ambari uses its internal implementation to install custom services to any of the nodes of the cluster through Mpack. This allows Hydra to be installed in all nodes through a single click from Ambari.

With SSH Access

If you have the SSH access to the cluster nodes use the following steps:

1. Deploy the Pulse Hydra agent by executing the following steps.
   1. Execute the following deploy command, here -f argument defines parallel ssh session to deploy agents
   2. Type y when the Have you run 'accelo configure agents' ?[y/n] question is displayed.
   3. Press the enter key.

accelo deploy hydra -f 5

Without the SSH Access

If you do not have the SSH access to the cluster nodes, contact the Acceldata Support Team for offline Hydra agent packages (RPM or DEB) that you can install to all the cluster nodes with your preferred configuration management tool of your choice.

Install Agent(s)

1. Download and upload hystaller binary on mentioned node from provided link and credentials.
2. Place hystaller on /tmp directory and follow the below steps of command sequences, here replace <PULSE_HOSTNAME> to hostname running Pulse UI (main server).

chmod +x /tmp/hystaller

PULSE_HOME="/opt/pulse"

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin"

HYDRA_SERVER_URL="http://<PULSE_HOSTNAME>:19072"

HYDRA_HEARTBEAT_DURATION="60"

HYDRA_PARCEL_MODE="False"

HYDRA_HOSTNAME_CASE="lower"

HYDRA_HOSTNAME_METHOD="CMD"

HYDRA_HEARTBEAT_JITTER="10"

#sudo if this is non-root user

sudo env  "PULSE_HOME=$PULSE_HOME" "PATH=$PATH" "HYDRA_SERVER_URL=$HYDRA_SERVER_URL" "HYDRA_HEARTBEAT_DURATION=$HYDRA_HEARTBEAT_DURATION" "HYDRA_PARCEL_MODE=$HYDRA_PARCEL_MODE" "HYDRA_HOSTNAME_CASE=$HYDRA_HOSTNAME_CASE" "HYDRA_HOSTNAME_METHOD=$HYDRA_HOSTNAME_METHOD" "HYDRA_HEARTBEAT_JITTER=$HYDRA_HEARTBEAT_JITTER" /tmp/hystaller install

3. SSH to main Pulse server running UI and run below command to sync configuration from Pulse server to new node.

accelo reconfig cluster

Uninstall Agent(s)

1. Using the same hystaller agent can be uninstalled

/tmp/hystaller uninstall

Deploy Pulse Hydra using the Ambari Mpack

Prerequisites

Tested Versions:

• Ambari: 2.7.4.0, 2.6.2.2, 2.7.6.0
• Cluster: HDP 3.1, HDP 2.6, ODP 3.2

Deployment of Hydra

1. Download the Tarball in Ambari server and save it as /tmp/hystaller_mpack<version>.tar.gz .

wget <S3 LINK FROM LICENSE UI> -O "hystaller_mpack<version>.tar.gz"

2. Install hystaller_mpack<version>.tar.gz in ambari-server .

ambari-server install-mpack --mpack=/tmp/hystaller_mpack<version>.tar.gz -v

3. Add the Hydra Service symlink in the service list of Ambari.

cd /var/lib/ambari-server/resources/stacks/<Cluster_type>/<Cluster_version>/services

ln -s /var/lib/ambari-server/resources/mpacks/ambari-hydra-agent-mpack-<hydra_version>/common-services/PULSE_HYDRA_AGENT/<hydra_version> PULSE_HYDRA_AGENT

4. Restart ambari-server.

ambari-server restart

It must now restart the server with logs stating Server started listening on 8080.

5. Navigate to the Ambari UI and select Add Service .

6. Select Hydra from the services list and select the nodes to deploy hydra.

7. Edit the configurations as per the requirements.

8. Select Deploy to install and start Hydra in respective hosts.

9. Once deployment is successful, we can get the status of each hydra agent as well as the status of pulsenode, pulselog, pulsejmx.

Installation of Hydra Agent on Newly Added Hosts

The Hydra Agent must be installed on at least one node in a cluster.

1. Run the following commands on Ambari node.

curl -s -u admin:admin http://0.0.0.0:8080/api/v1/hosts|grep host_name|sed -n 's/.*"host_name" : "\([^\"]*\)".*/\1/p'>hostcluster.txt

​

while read line ; do j=$line;  mkdir -p $j;  curl -u admin:admin -H "X-Requested-By:ambari" -i -X  POST http://0.0.0.0:8080/api/v1/clusters/<CLUSTER_NAME>/hosts/$j/host_components/PULSE_HYDRA_AGENT; done < hostcluster.txt

​

while read line ; do j=$line;  mkdir -p $j;  curl -u admin:admin -H "X-Requested-By:ambari" -i -X  PUT -d '{"HostRoles": {"state": "INSTALLED"}}' http://0.0.0.0:8080/api/v1/clusters/<CLUSTER_NAME>/hosts/$j/host_components/PULSE_HYDRA_AGENT; done < hostcluster.txt

​

while read line ; do j=$line;  mkdir -p $j;  curl -u admin:admin -H "X-Requested-By:ambari" -i -X  PUT -d '{"HostRoles": {"state": "STARTED"}}' http://0.0.0.0:8080/api/v1/clusters/<CLUSTER_NAME>/hosts/$j/host_components/PULSE_HYDRA_AGENT; done < hostcluster.txt

The Hydra Agent is installed on all the nodes. Restart if required.

Test your Agent Configuration Changes

If you made any changes to telegraph(node agent), filebeat(logs agent) or jmx(jmx agent), and you want to test your feature, then:

1. Build the Hydra test image from bitbucket and tag to running ad-hydra container and restart ad-hydra.
2. From the Ambari server, stop the Hydra service.
3. Delete the Hydra service.
4. Add the Hydra Service, follow the above Images and provide Hydra Server URL, Hydra Installer Repository, PULSE UI URL and deploy.
5. From Pulse node, do reconfig cluster, wait for ~2 minutes.
6. Then check your changes on the nodes for your changes.

Uninstall Pulse Agents (Hydra) using Ambari UI

You can uninstall Hydra Agents from the nodes directly using the Ambari UI.

The Pulse Agents consist of all agents, such as Node, Logs, JMX, AXN, YARNMetrics, and Hydra.

Steps to uninstall Hydra Agents:

1. In the Amabari UI, navigate to Services > Hydra.

2. In the Hydra Summary page, click Hydra Agents. The list of hosts where Hydra Agents are installed appears on the screen.

3. Select each host and click Components. The list of components installed appears. Locate Hydra Agents and click the three dots next to them, and repeat the steps below for each host in the same order.

   1. Select Stop. This stops all Hydra Agents installed on the host.
   2. Select Uninstall. This uninstalls all Hydra Agents from each host.
   3. Select Delete. This deletes all Hydra Agents from each host.

4. In the Hydra Summary, click Actions > Delete Service.

5. Run the uninstall Mpack command of the Ambari host using the CLI

ambari-server uninstall-mpack --mpack-name=ambari-hydra-agent-mpack -v

Validate Pulse Installations

Run the following command to validate Pulse installation.

accelo info

You get output as shown in the following image.