Upgrade to Version 4.1.3

You can follow the appropriate steps to upgrade Pulse to version 4.1.3 based on your current deployment:

• Upgrade Pulse from Version 4.1.x to 4.1.3

• Upgrade Pulse from Version 4.0.x to 4.1.3

• Upgrade Pulse from Version 3.8.x. to 4.1.3

Note Before upgrading Pulse, ensure that your Docker version is upgraded. For details, see Upgrade Prerequisite.

Hook Migration

After upgrading Pulse to a newer Pulse version, migrate your existing Hive, Tez, and Spark hooks to the supported hook version required for that release to ensure continued compatibility and event collection.

For information about the supported hook version for each Pulse release, see Supported Hook Versions.

For details about the hook migration steps, see:

• For Hive and Tez

  • Configure ODP Hive and Tez

  • Configure HDP Hive, Tez, and Sqoop

  • Configure CDP Hive and Tez

  • Configure CDH Hive and Sqoop

• For Spark

  • Configure ODP Spark 2 and 3

  • Configure HDP Spark

  • Configure CDP Spark

  • Configure CDH Spark

Enable the runMigrations Property for Dashplots

Before performing a fresh installation or upgrading Pulse to a newer version, verify that the runMigrations property in the ad-dashplots-ui section of the ad-core configuration is set to true.

This property enables the required Dashplot database migrations. If it is not enabled during an upgrade, the ad-dashplots-ui service may fail to start and repeatedly restart with a database migration error similar to the following:

ERROR: column dashplot_reporting.cluster_key does not exist

For more information about the runMigrations property and other Dashplot configuration options, see Configure Dashplot Variables.

Upgrade Pulse from Version 4.1.x to 4.1.3

This section describes the required steps to upgrade from Pulse 4.1.x to 4.1.3.

1. Update ImageTag in:

Set it to:

2. Back up the existing accelo CLI file and download the latest accelo.linux.

3. Rename and make the CLI executable:

4. Push the updated config to the database:

5. For online installations:

Press Y when prompted.

6. For offline installations, download the latest image packs and load them:

7. Delete the existing Impala Stream NATS Consumers (Required for 4.1.0 Upgrades Only).

   1. Delete all existing Impala Stream NATS consumers only when upgrading from Pulse 4.1.0 to Pulse 4.1.3

   2. This step is not required when upgrading from Pulse 4.1.2 to Pulse 4.1.3.

   3. For details, see the template below.

For example:

8. Enable the ad-dashplots migration flags. For details, see Configure Dashplot Variables.

9. Restart all containers:

10. Uninstall the Hydra agent:

11. Install the Hydra agent:

12. Push the updated configuration again:

13. Confirm the ImageTag update:

Upgrade Pulse from Version 4.0.x to 4.1.3

This section describes the required steps to upgrade from Pulse 4.0.x to 4.1.3.

Step 1: Update Container Runtime Configuration

Update the /etc/profile.d/ad.sh file based on your container runtime.

For Podman:

For Docker:

Once modified, run the following command to apply the updated environment variables.

Step 2: Uninstall Hydra Agent

Hydra agents must be uninstalled to stop NATS stream data from being sent.

SSH-based deployment

Mpack and parcel-based deployments

• Uninstall Hydra agents from the CDP or Ambari UI.

• This step is required to stop NATS streams from being sent from the agent's side. Before doing further steps, ensure all NATS stream data has been processed before continuing.

Step 3: Update Accelo CLI

1. Take a backup of the existing accelo CLI binary.

2. Download the latest accelo.linux CLI.

3. Replace the existing binary and make it executable:

Step 4: Elasticsearch Migration: ES 8.18.4 → ES 8.19.0

Before upgrading Pulse from 4.0.x to 4.1.2 or later, both Elasticsearch instances must be upgraded from 8.18.4 to 8.19.0:

• ad-elastic

• ad-fs-elastic

For details about migrating the Elasticsearch DB from 8.18.4 → 8.19.0, see Upgrade ElasticSearch from ES 8.16.6 to ES 8.19.0.

Step 5: Update the Image Tag

Update the ImageTag in $AcceloHome/config/accelo.yml:

Set the version to:

Push the updated configuration to the database:

Verify the update:

Step 6: Offline Installations: Load Latest Images

For offline installations, download the latest image packs and load them:

Step 7: Run Migration

Run the migration command from $AcceloHome.

For example:

• -v: specifies the version you are migrating to (4.1.0). Note: Keep the version as -v 4.1.0, even when you are migrating to any update release version in 4.1.x.

• -i: specifies the version you are migrating from (4.0.0).

To migrate from version 4.0.0 to 4.1.3, the migration command looks as shown below.

For a single cluster, run:

For multi-cluster setups, run:

If any issues occur during migration, rerun the same command.

Step 8: Online Installations: Pull Latest Images

For online installations, run:

Press Y when prompted to download the most recent binaries.

Step 9: Restart Containers

Restart all containers to use the updated images:

Step 10: (Optional) Migrate from old Actions (ad-director) to the new Actions service

Pulse does not support Kerberos migration from old actions to new actions. For new actions, the Kerberos plugin had to be added before running any action plugin if the cluster was kerberized.

1. If the ad-director was previously configured, verify that the new Actions Service (ad-axnserver) container is running.

2. If the Actions Service container is not present, deploy it using the command:

3. When prompted, select Actions Service.

4. Press Enter to start deployment.

Verify the deployment

After the deployment completes, confirm that the container is running:

Expected output (status should be Up):

Check the container logs to confirm that the service has started successfully:

Reconfigure Notifications

If notifications (Email, Slack, Jira) were configured in ad-director, enable them again for the new Actions service.

Example:

After configuration completes:

• The notifications configuration file is generated.

• The configuration is pushed to the Pulse database.

Run the migration

Once the Actions Service is running and healthy, run:

Step 11: Install Hydra Agent

1. Deploy Hydra agents:

For mpack- or parcel-based deployments, install the Hydra agent using Ambari or Cloudera Manager.

2. Reconfigure the cluster to apply the updated agent configuration:

3. Push the updated configuration to the database:

Step 12: (Optional) Enable native SSL in Pulse UI

• If SSL was previously enabled using ad-proxy, enable native SSL support in Pulse UI.

• The ad-proxy service has been removed starting in Pulse 4.1.x. Native SSL must now be configured directly in Pulse UI.

• For instructions, see Enable Native SSL/TLS for Pulse Web UI.

Step 13: Configure Retention

To configure MongoDB cleanup and compaction scheduling:

Step 14: Create Database Indices

Run the following command to create indices for database collections:

If readOnlyRootFSEnabled is set to true in $AcceloHome/config/accelo.yml, follow these steps:

1. Open $AcceloHome/config/accelo.yml and set:

2. Run:

3. Reset readOnlyRootFSEnabled to true and rerun:

Upgrade Pulse from Version 3.8.x. to 4.1.3

Step 1: Update Container Runtime Configuration

Update /etc/profile.d/ad.sh based on the container orchestration tool in use.

For Docker:

For Podman:

Once modified, run the following command to apply the updated environment variables.

Step 2: Uninstall Hydra Agent

Hydra agents must be uninstalled to stop NATS stream data from being sent.

SSH-based deployment

Mpack and parcel-based deployments

• Uninstall Hydra agents from the CDP or Ambari UI.

• This step is required to Stop nats streams data being sent from agents side. Ensure all NATS stream data has been processed before continuing.

Step 3: Update Accelo CLI

1. Take a backup of the existing Accelo binary.

2. Download the latest accelo.linux CLI.

3. Replace the binary and make it executable:

Step 4: Elasticsearch Migration: ES 8.18.4 → ES 8.19.0

Before upgrading Pulse from 4.0.x to 4.1.2 or later, both Elasticsearch instances must be upgraded from 8.18.4 to 8.19.0:

• ad-elastic

• ad-fs-elastic

For details about migrating the Elasticsearch DB from 8.18.4 → 8.19.0, see Upgrade ElasticSearch from ES 8.16.6 to ES 8.19.0.

Step 5: Update ImageTag

Update the ImageTag in:

Set:

Push the updated configuration to the database:

Verify the update:

Step 6: Offline Installations: Load Latest Images

For offline installations, download the latest image packs and load them:

Step 7: Run Migration

For example:

• -v: specifies the version you are migrating to (4.1.0). Note: Keep the version as -v 4.1.0, even when you are migrating to any update release version in 4.1.x.

• -i: specifies the version you are migrating from (3.8.0).

To migrate from version 3.8.0 to 4.1.3, the migration command looks as shown below.

• For a single cluster, run:

• For multi-cluster setups, run:

If any issues occur, rerun the same migration command.

Step 8: Deploy Core Services

After migration, deploy core services:

Step 9: Online Installations: Pull Latest Images

For online installations:

Press Y when prompted to download the latest binaries.

Step 10: Restart Containers

Restart all containers to apply the updated images:

Step 11: (Optional) Migrate from old Actions (ad-director) to the new Actions service

1. If the ad-director was previously configured, verify that the new Actions Service (ad-axnserver) container is running.

2. If the Actions Service container is not present, deploy it using the command:

3. When prompted, select Actions Service.

4. Press Enter to start deployment.

Verify the deployment

After deployment completes, confirm that the container is running:

Expected output (status should be Up):

Check the container logs to confirm that the service has started successfully:

Reconfigure Notifications

If notifications (Email, Slack, Jira) were configured in ad-director, enable them again for the new Actions service.

Example:

After configuration completes:

• The notifications configuration file is generated.

• The configuration is pushed to the Pulse database.

Run the migration

Once the Actions Service is running and healthy, run:

Step 12: Install Hydra Agent

Deploy Hydra agents:

Reconfigure the cluster:

Push updated configuration to the database:

Step 13: (Optional) Enable native SSL in Pulse UI

• If SSL was previously enabled using ad-proxy, enable native SSL support in Pulse UI.

• The ad-proxy service has been removed starting in Pulse 4.1.x. Native SSL must now be configured directly in Pulse UI.

• For instructions, see Enable Native SSL/TLS for Pulse Web UI.

Step 14: Configure Retention

To configure MongoDB cleanup and compaction:

Step 15: Create Database Indexes

Create database indexes:

If readOnlyRootFSEnabled Is Set to true

If readOnlyRootFSEnabled is enabled in $AcceloHome/config/accelo.yml, perform the following steps:

1. Set readOnlyRootFSEnabled to false.

2. Push configuration and restart:

3. Create indexes:

4. Set readOnlyRootFSEnabled back to true.

5. Push configuration and restart again: