Prerequisites

Do not proceed for an upgrade without Ambari Upgrade. For more information, see Upgrade Ambari.

After upgrading to the latest version of Ambari, proceed with both express and rolling upgrades. Perform the following:

(Mandatory for Rolling Upgrade) Ensure that HA (High Availability) is enabled for HDFS. Optionally, you can also enable HA for YARN.
(Mandatory) To preserve the configurations of any service, including management packs (mpacks), you can use the following process, which provides two ways to store the configuration settings.

Create Separate Database and Store Configs (Recommended)

Pre-upgrade

Before proceeding with the upgrade, ensure to execute the following command:

Bash
    
 
create database service_conf_bck;use service_conf_bck;CREATE TABLE IF NOT EXISTS configs_prior_patch ASSELECT    cc.config_id,    cc.type_name,    cc.config_data,    cc.config_attributes,    service_name,    sc.version,    sc.note,    (SELECT MAX(version) FROM ambari.serviceconfig s     WHERE s.service_name = sc.service_name) AS max_versionFROM    ambari.serviceconfig scINNER JOIN    ambari.serviceconfigmapping scmON    sc.service_config_id = scm.service_config_idINNER JOIN    ambari.clusterconfig ccON    cc.config_id = scm.config_idWHERE    sc.service_name IN (<service-name-1>,<service-name-2>,.....);
Copy

Post-upgrade

After completing the upgrade, to restore the configurations for a specific service, execute the following code:

Bash
    
update clusterconfig a set config_data = (select config_data from service_conf_bck.configs_prior_patch b where a.type_name = b.type_name and version = (SELECT MAX(version) FROM service_conf_bck.configs_prior_patch WHERE service_name = <service-name>) and service_name = <service-name>),    config_attributes = (select config_attributes from service_conf_bck.configs_prior_patch b where a.type_name = b.type_name and version = (SELECT MAX(version) FROM serviceconfig WHERE service_name = <service-name>) and service_name = <service-name>)    where config_id in (select config_id from serviceconfigmapping where service_config_id in (select service_config_id from serviceconfig where service_name = <service-name> and version =    (select max(version) from serviceconfig where service_name = <service-name>)) );
Copy

For example, in the below code we have taken a backup for configurations of NiFi, Tez, and Spark3:

Bash
    
​x
 
create service_conf_bck;use service_conf_bck;​​CREATE TABLE IF NOT EXISTS configs_prior_patch ASSELECT    cc.config_id,    cc.type_name,    cc.config_data,    cc.config_attributes,    service_name,    sc.version,    sc.note,    (SELECT MAX(version) FROM ambari.serviceconfig s     WHERE s.service_name = sc.service_name) AS max_versionFROM    ambari.serviceconfig scINNER JOIN    ambari.serviceconfigmapping scmON    sc.service_config_id = scm.service_config_idINNER JOIN    ambari.clusterconfig ccON    cc.config_id = scm.config_idWHERE    sc.service_name IN ('NIFI','TEZ','SPARK3');
Copy

After the upgrade, to restore NIFI configurations, the following command was run in mysql:

Bash
    
update clusterconfig a set config_data = (select config_data from service_conf_bck.configs_prior_patch b where a.type_name = b.type_name and version = (SELECT MAX(version) FROM service_conf_bck.configs_prior_patch WHERE service_name = 'NIFI') and service_name = 'NIFI'),    config_attributes = (select config_attributes from service_conf_bck.configs_prior_patch b where a.type_name = b.type_name and version = (SELECT MAX(version) FROM serviceconfig WHERE service_name = 'NIFI') and service_name = 'NIFI')    where config_id in (select config_id from serviceconfigmapping where service_config_id in (select service_config_id from serviceconfig where service_name = 'NIFI' and version =    (select max(version) from serviceconfig where service_name = 'NIFI')) );
Copy

Create Database and Store Configs (FYI)

Run the following command before upgrade:

Bash
    
create table configs_prior_patch_<service-name> as select cc.config_id,cc.type_name,cc.config_data ,cc.config_attributes ,service_name,sc.version,sc.note  from serviceconfig sc inner join serviceconfigmapping scm on sc.service_config_id =scm.service_config_id  inner join clusterconfig cc on cc.config_id = scm.config_id where sc.service_name in (<service-name>);
Copy

On completion of the upgrade, to restore the configurations, run the following code:

Bash
    
update clusterconfig a set config_data = (select config_data from configs_prior_patch_nifi b where a.type_name = b.type_name and version = <latest-config-version-before-upgrade> and service_name = <service-name>),    config_attributes = (select config_attributes from configs_prior_patch_nifi b where a.type_name = b.type_name and version = <latest-config-version-before-upgrade> and service_name = <service-name>)    where config_id in (select config_id from serviceconfigmapping where service_config_id in (select service_config_id from serviceconfig where service_name = <service-name> and version =    (select max(version) from serviceconfig where service_name = <service-name>)) );
Copy

Example for service-name: NIFI with latest-config-version-before-upgrade:4

Bash
    
Before Upgrade:create table configs_prior_patch_nifi as select cc.config_id,cc.type_name,cc.config_data ,cc.config_attributes ,service_name,sc.version,sc.note  from serviceconfig sc inner join serviceconfigmapping scm on sc.service_config_id =scm.service_config_id  inner join clusterconfig cc on cc.config_id = scm.config_id where sc.service_name in ('NIFI');​After Upgradeupdate clusterconfig a set config_data = (select config_data from configs_prior_patch_nifi b where a.type_name = b.type_name and version = 4 and service_name = 'NIFI'),    config_attributes = (select config_attributes from configs_prior_patch_nifi b where a.type_name = b.type_name and version = 4 and service_name = 'NIFI')    where config_id in (select config_id from serviceconfigmapping where service_config_id in (select service_config_id from serviceconfig where service_name = 'NIFI' and version =    (select max(version) from serviceconfig where service_name = 'NIFI')) );
Copy

Ensure that you run service checks for all the installed stack services by performing the following steps in the UI:

Navigate to Actions > Service Check.

For example, here's how to perform a service check for HDFS. Remember to perform this check for all the installed services in your stack.

Common Steps to Perform Before the Upgrade

Ensure that you have successfully run service checks for all the services and uninstalled any existing mpacks before proceeding with the upgrade. If you face any service check issue kindly look at troubleshooting steps mentioned at the end of the document.
(For reference before the upgrade, check Ambari warnings.) The following warning may appear if you have mpacks installed and linked to your stack (e.g., SPARK3). Ensure that they are properly addressed or removed before proceeding with the upgrade, and only proceed once you are certain that the warnings are unnecessary or will not impact the upgrade.

Sample Exception

Bash
    
 
The following components are reporting unexpected versions:   manish1.odp.ce    SPARK3_CLIENT: UNKNOWN    SPARK3_JOBHISTORYSERVER: UNKNOWN    NIFI_MASTER: UNKNOWN  manish2.odp.ce    SPARK3_CLIENT: UNKNOWN  manish3.odp.ce    SPARK3_THRIFTSERVER: UNKNOWN    SPARK3_CLIENT: UNKNOWN    LIVY3_SERVER: UNKNOWN
Copy

Navigate to the Stack and Version page and click UPGRADE for the newly added stack.

Troubleshooting Steps for Service Checks

Follow these troubleshooting steps for the listed services below:

Zookeeper Service

Grant permission to zookeeper.log

Bash
    
 
chmod 664 /var/log/zookeeper/zookeeper.log
Copy

Kafka Service

Modify the file at location /usr/odp/3.2.3.1-3/kafka/bin/kafka-run-class.sh with content mentioned here.

Last updated on May 24, 2024

Was this page helpful?