Install Agents: CDP - Hydra Services (Parcel)

Parcel installation is a deployment strategy adopted by Acceldata to seamlessly deploy, install, and update Pulse in your environment. The document instructs how to deploy the Pulse Hydra agents into your cluster. After deploying, the Pulse Hydra agents automatically install and update Pulse in your environment.

Deploy Pulse Agents

For CDP clusters, the agents are provided in CDP parcels. The Cloudera Manager (CM) distributes it to all the nodes. It does not require any Configuration Management Tool(CMT). Post-installation Hydra Agent automatically updates every time configuration changes occur. The frequency of updates will depend on the Heartbeat Time interval selected.

Working of Parcel in Cloudera

Cloudera parcels distribute the service binaries and the required libraries in all the nodes of the Cloudera managed cluster.

1. The user installs the repository URL in the Cloudera Manager Parcels Repositories and Network setting page.
2. Cloudera Manager automatically downloads the service parcels and makes them available for distribution and activation.

Prerequisites

Ensure that Port - 19072 is either in the whitelist or the protected list.

Note: The node must be formatted if a new or migrated node is added.

Migration Steps

Use the following steps if your version is greater than or equal to 2.0.0:

1. Clean the work directory$AcceloHome/work. Ensure that it only contains the license file and the cluster directories that has the host.ini files.
2. Check if the Hydra directory is present by running the commandls -al $AcceloHome/config/hydra. If no directory is found run theaccelo infocommand to create all the necessary directories.
3. Create the hydra.yml file in the path$AcceloHome/config/hydra .
4. Copy the following code to the hydra.yml file in$AcceloHome/config/hydra/hydra.yml .

version: "<CURRENT PULSE VERSION>"

epoch: "1634736457"

server_port: "19072"

enable_logger: false

creds:

5. Run the command accelo deploy addons and select the Hydra server from the add-ons list to deploy.
6. Type the accelo reconfig cluster command to reconfigure the cluster.
7. Type the commandaccelo uninstall legacy to uninstall the legacy agents. If you have installed Hydra Agent via Parcel before, follow the Upgradation of Parcel and Hydra Services steps to install and update the Hydra services in all the nodes and the agents.

The migration steps are now complete. Follow the Installing Hydra Agent steps to install the Hydra Agent through the CDP Parcel deployment.

If you have installed Hydra Agent via Parcel before, follow the Upgradation of Parcel and Hydra Services steps to install and update the Hydra services in all the nodes and the agents.

Note: Post-installation of Hydra services, the user cannot roll back to the older versions of the Pulse.

Installing Hydra Agent Parcel

1. Open the Cloudera Manager and navigate to the Parcels page Clusters → Parcels

2. Click on Parcel Repository & Network Settings button.
3. Click on the +icon on the right side of the screen and paste the following repository URL to the Remote Parcel Repository URL’s. http://pulse:<PARCEL_REPO_PASSWORD>@<PULSE_SERVER_IP>:19072/files/parcels

In the preceding block, replace the <PULSE_SERVER_IP> with the IP address or the host-name of the Pulse core server host.

4. Click on Save & Verify Configuration. After adding the repo, PULSE_HYDRA_AGENT Parcel appears on the screen with the latest version displayed from the Hydra server.

5. Click on the Download button to download the Parcel.

6. Click on Distribute to distribute the parcel in all the nodes of the cluster.

7. Click on Activate, to activate the Parcel on all the nodes.

8. You will be prompted with a system message. Click on OK to continue.

9. Restart and login to the Cloudera Manager Server.
10. Enter the following command in the CLI: systemctl restart cloudera-scm-server
11. The Cloudera Manager user interface is displayed. Navigate to the Clusters page.

12. On the top right of the panel, click on the Actions button.
13. Select the Add Service option.

14. Select the radio button Pulse Hydra Agent service and click the Continue button.

15. Select the nodes where you want the Hydra agents to run.

16. Add inputs to the following two variables:
    • Acceldata Pulse NODE URL: It is the URL to the Pulse node. Update the URL to override the localhost to Pulse virtual machine reachable IP or a resolvable hostname. You must port from the default 8080 to 19072. Example:http://<PULSE_URL>:19072
    • Heartbeat Time :- It is the refresh interval (seconds) for the Pulse agent to signal the server for any new configuration.

17. Click on Continue.
18. Click on Finish.
19. Type ‘accelo reconfig cluster’ in the console where Pulse is installed to reconfigure the cluster.

Upgradation of Parcel and Hydra Services

If you have already installed Hydra agents by Parcel, the following procedures will guide you through reinstalling and updating the Hydra service.

1. Delete the Hydra service
2. Deactivate the Parcel
3. Remove the Parcel from the hosts
4. Delete the Parcel
5. Upgrade the Parcel
6. Install Hydra service

1. Delete the Hydra Service

1. In the Cloudera Manager navigate to the Pulse Hydra Service page.

2. Click on the Actions button and select the Stop tab to stop the Pulse Hydra agent service.

3. Click on Delete to delete the service.

The Hydra Service has now been removed from the Cloudera Manager.

2. Deactivate the Parcel

1. Navigate to the Parcels page.

2. Click on Deactivate to deactivate the Parcel for the cluster.

3. Remove the Parcel from the Hosts

Remove the hosts to un-distribute from all the hosts of the cluster.

4. Delete the Parcel

Click on Delete to delete the hosts from the node.

5. Upgrade the Parcel

1. Click on the Check for New Parcels.

2. Click on the Download.
3. Click on Distribute to distribute the parcel in all the nodes of the cluster.

4. Click on Activate, to activate the Parcel on all the nodes.

5. You will be prompted with a system message. Click on OK to continue.

6. Restart and login to the Cloudera Manager Server.
7. Enter the following command in the CLI:systemctl restart cloudera-scm-server
8. The Cloudera Manager user interface is displayed. Navigate to the Clusters page.

9. On the top right of the panel, click on the Actions button.
10. Select the Add Service option.

11. Select the radio button Pulse Hydra Agent service and click the Continue button

12. Select the nodes where you want the Hydra agents to run.

13. Add inputs to the following two variables.

• Acceldata Pulse NODE URL: It is the URL to the Pulse node. Update the URL to override the localhost to Pulse virtual machine reachable IP or a resolvable hostname. You must port from the default 8080 to 19072. Example:http://<PULSE_URL>:19072
• Heartbeat Time: It is the refresh interval (seconds) for the Pulse agent to signal the server for any new configuration.

14. Click on Continue.
15. Click on Finish.
16. Type accelo reconfig cluster in the console where Pulse is installed to reconfigure the cluster.

The Hydra service is now updated in all the selected nodes and the agents.

Troubleshooting

The Start or Stop button in the CM only starts or stops the CM service roles but not the backend telemetry and agents. Click on Stop and Uninstall to stop sending data to the Pulse from the node under maintenance or decommissioning the agent.

To check the agent's health or after adding a new parcel monitoring under the Cloudera Management list, you must always restart the Cloudera Management Service.

Validate Pulse Installations

Run the following command to validate Pulse installation.

accelo info

You get output as shown in the following image.

Add Multiple Clusters to Pulse

Once you complete the Pulse installation, the cluster you chose during installation is added to Pulse. However, you can add multiple clusters to Pulse. The procedure is the same as described above with the following minor change.

In Step 2 of the Configure the Acceldata Core Components section, you must enter the Cloudera Manager (CM) URL and credentials and cluster that you want to add, for the Which Cluster to use question.