If you have upgraded ODP from version 3.2.3.3-3 (with Ambari 2.7.8.2-3) to version 3.3.6.4-1 (with Ambari 3.0.0.1-1) and need to roll back to ODP 3.2.3.3-3 (with Ambari 2.7.8.2-3), follow the steps below.
Steps to Downgrade
In this case, the downgrade was tested right before the finalization.
Click Proceed with Downgrade and proceed further.
Pause the downgrade process before restoring backups.
Pause Downgrade Before Restoring Backups
Proceed with the Ambari downgrade.
Ambari Downgrade
Fetch the current Ambari version.
Response:
# apt list ambari-*
Listing... Done
ambari-agent/unknown,now 3.0.0.1-1 amd64 [installed]
ambari-infra-manager/unknown 3.0.0.1-1 amd64
ambari-infra-solr-client/unknown,now 3.0.0.1-1 amd64 [installed]
ambari-infra-solr/unknown,now 3.0.0.1-1 amd64 [installed]
ambari-server/unknown,now 3.0.0.1-1 amd64 [installed]
ambari-views-package/unknown 3.0.0.1-1 amd64
Stop Ambari Server and Agents.
Response:
ambari-server stop
Using python /usr/bin/python3.11
Stopping ambari-server
Waiting for server stop...
Ambari Server stopped
Response:
ambari-agent stop
Verifying Python version compatibility...
Using python /usr/bin/python3.11
Found ambari-agent PID: 838229
Stopping ambari-agent
Removing PID file at /run/ambari-agent/ambari-agent.pid
ambari-agent successfully stopped
Restore the older Ambari packages.
Set up the old Ambari repositories.
cat /etc/apt/sources.list.d/ambari.list
If you are downgrading the Ambari server to version 2.7.8.2-301, restore the repository URLs.
For Ubuntu 20:
cat /etc/apt/sources.list.d/ambari.list
deb https://mirror.odp.acceldata.dev/v2/ambari/python3/jdk8/3.0.0.1-1/releases/ubuntu22/ ODP main
#deb https://mirror.odp.acceldata.dev/staging/ODP-3.3.6.4-1-FINAL-STAGING/v2/ambari/python3/jdk11/2.7.9.2-1/releases/ubuntu22/ ODP main
For RHEL 8/9:
cat /etc/yum.repos.d/ambari.repo
[ambari]
baseurl = https://mirror.odp.acceldata.dev/v2/ambari/python3/jdk8/2.7.9.2-1/releases/rhel8/
gpgcheck = 0
name = ambari Version - ambari-2.7.9.2-1
Remove the previously installed Ambari packages.
For Ubuntu 20:
apt remove --purge ambari-server ambari-agent ambari-infra-solr ambari-infra-solr-client -y
For RHEL 8/9:
rpm -e --nodeps ambari-server-3.0.0.1-1.x86_64 ambari-agent-3.0.0.1-1.x86_64 ambari-infra-solr-3.0.0.1-1.noarch ambari-infra-solr-client-3.0.0.1-1.noarch
Install the required packages for Ambari downgrade.
yum install ambari-server ambari-agent ambari-infra-solr ambari-infra-solr-client
Reinstall Ambari Server to version 2.7.8.2-1 from 2.7.9.2-1.
[root@rl9upg1 backup]# rpm -qa | grep -i ambari
ambari-server-2.7.9.2-1.x86_64
ambari-infra-solr-client-2.7.9.2-1.noarch
ambari-infra-solr-2.7.9.2-1.noarch
ambari-agent-2.7.9.2-1.x86_64
Restore Ambari configurations:
root@ub22upg1:/etc/ambari-server# vim conf_bkp/conf/ambari.properties
root@ub22upg1:/etc/ambari-server# cp conf_bkp/conf/ambari.properties conf/
root@ub22upg1:/etc/ambari-server# cp conf_bkp/conf/password.dat conf/
Info
Make sure that you do not restore the Ambari database.
Reset the Version
Reset the version using the below command.
mysql> SELECT * FROM metainfo WHERE metainfo_key='version';
+--------------+----------------+
| metainfo_key | metainfo_value |
+--------------+----------------+
| version | 3.0.0 |
+--------------+----------------+
1 row in set (0.00 sec)
mysql> UPDATE metainfo SET metainfo_value='2.7.9.2' WHERE metainfo_key='version';
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> SELECT * FROM metainfo WHERE metainfo_key='version';
+--------------+----------------+
| metainfo_key | metainfo_value |
+--------------+----------------+
| version | 2.7.9.2 |
+--------------+----------------+
1 row in set (0.00 sec)
Reset Java Version to 1.8
From ambari.properties, reset java.home to 1.8. or JDK11 (previous)
java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.462.b08-3.el9.x86_64/jre
stack.java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.462.b08-3.el9.x86_64/jre
Host level
On the host level, set the node’s default Java version to 1.8.
root@ub22upg1:/etc/ambari-server# update-alternatives --config java
There are 2 choices for the alternative java (providing /usr/bin/java).
Selection Path Priority Status
------------------------------------------------------------
* 0 /usr/lib/jvm/java-11-openjdk-amd64/bin/java 1111 auto mode
1 /usr/lib/jvm/java-11-openjdk-amd64/bin/java 1111 manual mode
2 /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java 1081 manual mode
Press <enter> to keep the current choice[*], or type selection number: 2
update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java to provide /usr/bin/java (java) in manual mode
[root@rl9upg1 backup]# java -version
openjdk version "1.8.0_462"
OpenJDK Runtime Environment (build 1.8.0_462-b08)
OpenJDK 64-Bit Server VM (build 25.462-b08, mixed mode)
Start Ambari Server and Agents
Ambari Server
root@ub22upg1:/etc/ambari-server# ambari-server start
Using python /usr/bin/python3.11
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start..............
Server started listening on 8080
DB configs consistency check found warnings. See /var/log/ambari-server/ambari-server-check-database.log for more details.
Ambari Server 'start' completed successfully.
Agent
root@ub22upg1:/etc/ambari-server# ambari-agent start
Verifying Python version compatibility...
Using python /usr/bin/python3.11
Checking for previously running Ambari Agent...
Checking ambari-common dir...
Starting ambari-agent
Verifying ambari-agent process status...
Ambari Agent successfully started
Agent PID at: /run/ambari-agent/ambari-agent.pid
Agent out at: /var/log/ambari-agent/ambari-agent.out
Agent log at: /var/log/ambari-agent/ambari-agent.log
Reset Configuration Version
Reset the ODP stack configurations to the previous version (before the upgrade).
After resetting the older configurations, resume the downgrade.
For example, for the HDFS configuration version:
Reset the configuration version for all services.
Resume Downgrade
Restore Backups
Log in to the backend MySQL database and restore the backed-up SQL files.
-- 1. Restore Oozie Database
DROP DATABASE IF EXISTS oozie;
CREATE DATABASE oozie;
USE oozie;
SOURCE /root/basa/ODP_backup/oozie_bkp.sql;
-- 2. Restore Hive Database
DROP DATABASE IF EXISTS hive;
CREATE DATABASE hive;
USE hive;
SOURCE /root/basa/ODP_backup/hive_bkp.sql;
-- 3. Restore Ranger Database
DROP DATABASE IF EXISTS ranger;
CREATE DATABASE ranger;
USE ranger;
SOURCE /root/basa/ODP_backup/ranger_bkp.sql;
-- 4. Restore Ranger KMS Database
DROP DATABASE IF EXISTS rangerkms;
CREATE DATABASE rangerkms;
USE rangerkms;
SOURCE /root/basa/ODP_backup/rangerkms_bkp.sql;
-- 5. Restore Druid Database
DROP DATABASE IF EXISTS druid;
CREATE DATABASE druid;
USE druid;
SOURCE /root/basa/ODP_backup/druid_bkp.sql;
Proceed with the remaining downgrade steps.
Known Limitations
Issue: HDFS NameNode failed to start with the error:
Error
2025-08-16 18:13:11,103 INFO util.ExitUtil (ExitUtil.java:terminate(241)) - Exiting with status 1: java.io.IOException: File system image contains an old layout version -65. An upgrade to version -66 is required.
Resolution:
At restored hdfs namenode’s layoutVersion, set as -66
cat /hadoop/hdfs/namenode/current/VERSION
Response:
#Sat Aug 16 23:08:17 IST 2025
namespaceID=2113890819
clusterID=CID-089fc5ca-1bb7-458e-b63a-83ae0b4921e6
cTime=1755365897253
storageType=NAME_NODE
blockpoolID=BP-1385487665-10.100.11.39-1755221846094
layoutVersion=-65
Set layoutVersion -66 followed by restart the HDFS.
#Sat Aug 16 23:08:17 IST 2025
namespaceID=2113890819
clusterID=CID-089fc5ca-1bb7-458e-b63a-83ae0b4921e6
cTime=1755365897253
storageType=NAME_NODE
blockpoolID=BP-1385487665-10.100.11.39-1755221846094
layoutVersion=-66
Retry the HDFS restart from the Downgrade page, and proceed further.
