Upgrade from Version 3.0.3 to 3.1.0

This document describes the steps to migrate from Pulse 3.0.3 version to 3.1.0 version. You must perform the steps mentioned in this document in all your clusters.

If you are using Pulse 3.0.0, you cannot migrate from Pulse 3.0.0 to 3.1.0 directly. You must first migrate to 3.0.3 and then migrate to 3.1.0. To learn more about how to migrate from pulse 3.0.0 to 3.0.3, see Upgrade Pulse from 3.0.0 to 3.0.3.

Backup Steps

1. Take backup of Dashplots Charts using Export option.

2. Take backup of Alerts using the Export option.

Migration Steps

1. Execute the following command to run the migrations.

accelo migrate -v 3.1.0

Note : If you are using a multi-cluster setup, you must use the - a option. For example, accelo migrate -v 3.1.0 -a. This command automates migration for all the clusters.

2. (Applicable only for single-node Pulse deployment) Update the spark.events.url section in the acceldata<clustername>.conf file. You must replace the <clustername> in the field with the following URL.

spark.events.url = "http://ad-sparkstats:19004/events" 

​

# applicable for single node Pulse deployment

3. (Applicable only for multi-node Pulse deployment) Update the spark.events.url section in the acceldata<clustername>.conf file. You must replace the <clustername> in the field with the following URL.

http://<IP_WHERE_SPARKSATS_CONTAINER_IS_RUNNING>:19004/events

  # applicable for multi node Pulse deployment

4. Execute the following command to add the ad-events connection info in acceldata.conf file and Mongo.

accelo reconfig cluster

Deploy NATS Container

Pulse now uses the NATS queue. To enable the NATS queue, you must execute the following command.

accelo deploy core

Set Time Zone for Logs

The commands in this section facilitate you to update the environment variables such that they match the server time zone in JODA time format.

1. Execute the following command if the ad-logsearch.yml file is not present at the $AcceloHome/config/docker/addons directory.

accelo admin makeconfig ad-logsearch

2. Open the ad-logsearch.yml file from the $AcceloHome/config/docker/addons directory.
3. Update the following environment variable's value to match the server’s time zone in the joda format and add it to the ad-logstash section.

- TZ=<joda time zone for the server/node>

For example, if your server is located in India, you can set the joda time format as TZ=Asia/Kolkata.

3. Execute the following command to restart the logstash container. This ensures that the above changes are applied.

accelo restart ad-logstash

Oozie Connector Update

Add the following property to the acceldata/config/acceldata<CLUSTER NAME>.conf file, under the oozie.connectors section. Select the column based on the database applicable to your setup.

Pulse Hooks Deployment

Pulse now uses the NATS queue. To get the streaming data from NATS, you must configure and deploy the JAR files in various folders. Based on the configuration you use, you can refer to the respective sections (Hive, tez etc) for hook configuration. The links to each section are as follows.

• HDP 2.x, 3,x, and ODP 3.2.x
• CDH 5.x and 6.x
• CDP 7.x

Connector Level Configurations

This section describes the configurations you must perform on various connector microservices for Pulse deployments.

Sparkstats

You must generate the required connectors YAML file, if not available. You can execute the following command to generate the YAML file.

accelo admin makeconfig ad-core-connectors

Configuration for Sparkstats

You must configure the ad-core-connectors.yml file. To accomplish this, you must add the following environment variable under ad-sparkstats.

NATS_SERVER_HOST_LIST=(PULSE_IP:19009)

YARN applications and Hive-queries

You must perform the following configuration in Pulse configurations, under individual collectors.

nats.server.host.list = "(PULSE_IP:19009)" 

​

# You must use appropriate value for port number, applicable to your setup.