Ambari Server Upgrade

You can perform the following steps to upgrade the Ambari Server.

The following steps have to be performed on the installed Ambari Server node.

Stop Infra-Solr Service

Stop the Infra-Solr service from the Ambari UI by navigating to Ambari > Infra Solr > Actions > Stop.

Stop Ambari Server and Agents

Stop the Ambari server and agents by running the following command.

ambari-server stop ambari-agent stop

Warning Stop the Ambari Agents on all nodes in the cluster.

Configuration Backup

Take a backup of the Ambari-server configs.

cp -r /etc/ambari-server/config /etc/ambari-server/conf_bkp

Back up Ambari Database

mysqldump --databases ambari > /tmp/ambari-server_db_bckp/ambari_mysql_bckp.sql

Erase the Ambari Server

For RHEL:

yum erase ambari-server -y

For Ubuntu:

apt remove ambari-server -y

Add Ambari Repo on Cluster Nodes

Back up the Old Ambari Repository

For RHEL:

mv /etc/yum.repos.d/ambari.repo /tmp/

For Ubuntu:

mv /etc/apt/sources.list.d/ambari.list /tmp/

Update the Repository Files

Retrieve the latest Ambari repository details from Ambari Repositories and add it to all the cluster nodes.

Update ambari.repo (RedHat) or ambari.list (Ubuntu).

For RHEL:

vim /etc/yum.repos.d/ambari.repo

[ambari] async = 1 baseurl = https://mirror.odp.acceldata.dev/ODP/rhel/Ambari-3.0.0.1-1/ gpgcheck = 0 name = ambari Version - ambari-3.0.0.1-1

For Ubuntu:

vim /etc/apt/sources.list.d/ambari.list

deb https://mirror.odp.acceldata.dev/ODP/ubuntu20/Ambari-3.0.0.1-1/ ODP main deb https://mirror.odp.acceldata.dev/ODP/ubuntu20/Ambari-3.0.0.1-1/ ODP-UTILS main

List the New Packages

Verify the latest Ambari package for the version - 3.0.0.1-1.

For RHEL:

yum clean all yum list ambari-*

For Ubuntu 20:

apt clean all apt list ambari-*

Install the Updated Ambari Packages

For RHEL:

yum clean all yum install ambari-server -y

For Ubuntu:

apt clean apt install ambari-server -y

Restore the Configuration Files

Copy ambari.properties and password.dat to /etc/ambari-server/conf/.

Once the installation is successful, make sure to set up the following versions.

Set up Java 17

Make sure to install and update the default Java version as 17 on all the cluster nodes.

For RHEL:

yum install java-17-openjdk.x86_64 yum install java-17-openjdk-devel.x86_64

For Ubuntu:

apt-get install openjdk-17-jdk openjdk-17-jdk-headless

Warning

Ensure not to remove the jdk8 packages at this state.

update-alternatives --config java There are 2 choices for the alternative java (providing /usr/bin/java). Selection Path Priority Status ------------------------------------------------------------ * 0 /usr/lib/jvm/java-17-openjdk-amd64/bin/java 1111 auto mode 1 /usr/lib/jvm/java-11-openjdk-amd64/bin/java 1111 manual mode 2 /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java 1081 manual mode Press <enter> to keep the current choice[*], or type selection number: 1

Set up Python 3.11

yum install python3.11

Make sure to install and update the default Python version as 3.11 on all the cluster nodes.

update-alternatives --config python There are 4 programs which provide 'python'. Selection Command ----------------------------------------------- * 1 /usr/libexec/no-python 2 /usr/bin/python3 3 /usr/bin/python2 + 4 /usr/bin/python3.11 Enter to keep the current selection[+], or type selection number: 4

Update JSVC

Make sure to Instal the latest version of the bigtop-jsvc-1.2.4 on all the cluster nodes.

For RHEL:

yum install bigtop-jsvc* -y

For Ubuntu:

apt install bigtop-jsvc

Restore Config Files

Restore the ambari.properties and password.dat files, which are backed up at this step: Configuration Backup on this page.

Restore the ambari.properties file

cp /etc/ambari-server/conf_bkp/ambari.properties /etc/ambari-server/conf/

Please override the file by prompting 'y'.

Restore the password.dat file

cp /etc/ambari-server/conf_bkp/password.dat /etc/ambari-server/conf/

Update the java.home and stack.java.home

In this case, set JAVA_HOME to /usr/lib/jvm/java-17-openjdk

File path - /etc/ambari-server/conf/ambari.properties | grep -i java.home

java.home=/usr/lib/jvm/java-17-openjdk stack.java.home=/usr/lib/jvm/java-17-openjdk ambari.java.home=/usr/lib/jvm/java-17-openjdk mpacks-v2.staging.path=/var/lib/ambari-server/resources/mpacks-v2

Update the Ambari Database with New Tables and Columns

MySQL

mysql -u ambari -pambari USE ambari; CREATE TABLE IF NOT EXISTS registries( id BIGINT NOT NULL, registy_name VARCHAR(255) NOT NULL, registry_type VARCHAR(255) NOT NULL, registry_uri VARCHAR(255) NOT NULL, CONSTRAINT PK_registries PRIMARY KEY (id)); CREATE TABLE IF NOT EXISTS mpacks( id BIGINT NOT NULL, mpack_name VARCHAR(255) NOT NULL, mpack_version VARCHAR(255) NOT NULL, mpack_uri VARCHAR(255), registry_id BIGINT, CONSTRAINT PK_mpacks PRIMARY KEY (id), CONSTRAINT uni_mpack_name_version UNIQUE(mpack_name, mpack_version), CONSTRAINT FK_registries FOREIGN KEY (registry_id) REFERENCES registries(id)); ALTER TABLE stack ADD COLUMN mpack_id BIGINT NULL; ALTER TABLE stack ADD CONSTRAINT FK_mpacks FOREIGN KEY (mpack_id) REFERENCES mpacks(id); ALTER TABLE upgrade ADD COLUMN upgrade_package_stack VARCHAR(255) NOT NULL;

MariaDB

-- 0) Use Ambari schema USE ambari; -- 1) (Optional) Confirm current FK names on affected tables SELECT TABLE_NAME, CONSTRAINT_NAME FROM information_schema.TABLE_CONSTRAINTS WHERE CONSTRAINT_SCHEMA = DATABASE() AND CONSTRAINT_TYPE = 'FOREIGN KEY' AND TABLE_NAME IN ('servicecomponentdesiredstate', 'hostcomponentdesiredstate', 'hostcomponentstate') ORDER BY TABLE_NAME, CONSTRAINT_NAME; -- 2) Drop child FKs first, then parent FK ALTER TABLE hostcomponentdesiredstate DROP FOREIGN KEY hstcmpnntdesiredstatecmpnntnme; ALTER TABLE hostcomponentstate DROP FOREIGN KEY hstcomponentstatecomponentname; ALTER TABLE servicecomponentdesiredstate DROP FOREIGN KEY srvccmponentdesiredstatesrvcnm; -- 3) Alter service_name columns to VARCHAR(255) ALTER TABLE servicecomponentdesiredstate MODIFY COLUMN service_name VARCHAR(255) NOT NULL; ALTER TABLE hostcomponentdesiredstate MODIFY COLUMN service_name VARCHAR(255) NOT NULL; ALTER TABLE hostcomponentstate MODIFY COLUMN service_name VARCHAR(255) NOT NULL; -- 4) Recreate FKs ALTER TABLE servicecomponentdesiredstate ADD CONSTRAINT srvccmponentdesiredstatesrvcnm FOREIGN KEY (service_name, cluster_id) REFERENCES clusterservices (service_name, cluster_id); ALTER TABLE hostcomponentdesiredstate ADD CONSTRAINT hstcmpnntdesiredstatecmpnntnme FOREIGN KEY (component_name, service_name, cluster_id) REFERENCES servicecomponentdesiredstate (component_name, service_name, cluster_id); ALTER TABLE hostcomponentstate ADD CONSTRAINT hstcomponentstatecomponentname FOREIGN KEY (component_name, service_name, cluster_id) REFERENCES servicecomponentdesiredstate (component_name, service_name, cluster_id); -- 5) Verify column definitions SHOW COLUMNS FROM servicecomponentdesiredstate LIKE 'service_name'; SHOW COLUMNS FROM hostcomponentdesiredstate LIKE 'service_name'; SHOW COLUMNS FROM hostcomponentstate LIKE 'service_name';

Oracle 19C

sqlplus / as sysdba ALTER SESSION SET CONTAINER=<DB Name>; ALTER SESSION SET CURRENT_SCHEMA = <UserName>; CREATE TABLE registries( id NUMBER(19) NOT NULL, registy_name VARCHAR2(255) NOT NULL, registry_type VARCHAR2(255) NOT NULL, registry_uri VARCHAR2(255) NOT NULL, CONSTRAINT PK_registries PRIMARY KEY (id) ); CREATE TABLE mpacks( id NUMBER(19) NOT NULL, mpack_name VARCHAR2(255) NOT NULL, mpack_version VARCHAR2(255) NOT NULL, mpack_uri VARCHAR2(255), registry_id NUMBER(19), CONSTRAINT PK_mpacks PRIMARY KEY (id), CONSTRAINT uni_mpack_name_version UNIQUE(mpack_name, mpack_version), CONSTRAINT FK_registries FOREIGN KEY (registry_id) REFERENCES registries(id) ); ALTER TABLE stack ADD mpack_id NUMBER(19) NULL; ALTER TABLE stack ADD CONSTRAINT FK_mpacks FOREIGN KEY (mpack_id) REFERENCES mpacks(id);

Postgres

Ambari Server Upgrade Fails on PostgreSQL Due to Missing ambari Schema

Summary:

When Ambari is backed by PostgreSQL and ambari-server upgrade is executed, the Java SchemaUpgradeHelper process fails immediately because it attempts to set the PostgreSQL search path to a schema named ambari, but that schema does not exist. All Ambari tables are instead located in the default public schema.

This issue is specific to PostgreSQL schema behavior.

Other database platforms do not encounter this problem because:

MySQL treats the database and schema as the same object, so no separate schema creation is required.
Oracle automatically creates a schema for each database user.
PostgreSQL requires schemas to be explicitly created. Unless a schema is manually created beforehand, tables are created in the default public schema.

Error:

The following error appears during ambari-server upgrade execution and in /var/log/ambari-server/ambari-server.log:

SchemaUpgradeHelper:483 - Exception occurred during upgrade org.apache.ambari.server.AmbariException: ERROR: schema "ambari" does not exist Caused by: org.postgresql.util.PSQLException: ERROR: schema "ambari" does not exist at AbstractUpgradeCatalog.changePostgresSearchPath(AbstractUpgradeCatalog.java:397)

A secondary Python-level error may also appear:

ambari_simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

This is not the root cause. The Java upgrade process terminated unexpectedly and returned empty output, causing the Python wrapper to fail while attempting to parse the response.

Root Cause

The Ambari ambari.properties contains the property:

server.jdbc.postgres.schema=ambari

This causes AbstractUpgradeCatalog.changePostgresSearchPath() to execute:

SET search_path TO ambari;

The upgrade fails because the ambari schema does not exist. The Ambari PostgreSQL DDL was run before the schema was created, so PostgreSQL placed all Ambari tables in the default public schema.

The database should have been initialized with:

CREATE SCHEMA ambari AUTHORIZATION ambari; ALTER ROLE ambari SET search_path TO ambari, public;

Diagnosis

Read the connection details from ambari.properties

DB_HOST=$(grep server.jdbc.hostname /etc/ambari-server/conf/ambari.properties | cut -d= -f2) DB_PASS=$(cat /etc/ambari-server/conf/password.dat) echo "Host: $DB_HOST"

Check which schemas exist

PGPASSWORD=$DB_PASS psql -U ambari -d ambari -h $DB_HOST -p 5432 -c "\dn" -- Expected (broken state): -- Name | Owner -- --------+------------------- -- public | pg_database_owner -- Expected (correct state after fix): -- Name | Owner -- --------+-------- -- ambari | ambari -- public | pg_database_owner

Check where the tables are located

PGPASSWORD=$DB_PASS psql -U ambari -d ambari -h $DB_HOST -p 5432 -c " SELECT table_schema, count(*) AS table_count FROM information_schema.tables WHERE table_schema NOT IN ('information_schema','pg_catalog') GROUP BY table_schema;" ##-- Broken state: public | 113 ##-- Fixed state: ambari | 113

Fix Procedure

Info

Before making changes, take a full backup:

pg_dump -U ambari -h $DB_HOST ambari > /tmp/ambari_backup_ $(date +m%d).sql

Set the DB connection variables

Set connection variables and verify connectivity:

DB_HOST=$(grep server.jdbc.hostname /etc/ambari-server/conf/ambari.properties | cut -d= -f2) DB_PASS=$(cat /etc/ambari-server/conf/password.dat) # Verify connection PGPASSWORD=$DB_PASS psql -U ambari -d ambari -h $DB_HOST -p 5432 -c "SELECT 1;"

Create the ambari schema and move all objects

Create the schema, move tables and sequences, and set the search path:

PGPASSWORD=$DB_PASS psql -U ambari -d ambari -h $DB_HOST -p 5432 << 'EOF' -- Create schema CREATE SCHEMA IF NOT EXISTS ambari AUTHORIZATION ambari; -- Move all tables from public to ambari DO $$ DECLARE r RECORD; BEGIN FOR r IN SELECT tablename FROM pg_tables WHERE schemaname = 'public' LOOP EXECUTE 'ALTER TABLE public.' || quote_ident(r.tablename) || ' SET SCHEMA ambari'; RAISE NOTICE 'Moved table: %', r.tablename; END LOOP; END; $$; -- Move all sequences from public to ambari DO $$ DECLARE r RECORD; BEGIN FOR r IN SELECT sequence_name FROM information_schema.sequences WHERE sequence_schema = 'public' LOOP EXECUTE 'ALTER SEQUENCE public.' || quote_ident(r.sequence_name) || ' SET SCHEMA ambari'; RAISE NOTICE 'Moved sequence: %', r.sequence_name; END LOOP; END; $$; -- Set permanent search path for ambari role ALTER ROLE ambari SET search_path TO ambari, public; EOF

Verify

Verify all tables are now in the ambari schema:

# All tables should now be in ambari schema PGPASSWORD=$DB_PASS psql -U ambari -d ambari -h $DB_HOST -p 5432 -c " SELECT table_schema, count(*) AS table_count FROM information_schema.tables WHERE table_schema NOT IN ('information_schema','pg_catalog') GROUP BY table_schema;" # Expected output: # table_schema | table_count # --------------+------------- # ambari | 113 # Verify search path is set PGPASSWORD=$DB_PASS psql -U ambari -d ambari -h $DB_HOST -p 5432 -c "SHOW search_path;" # Expected: ambari, public

Retry the upgrade

ambari-server upgrade -v

Why does this only affect PostgreSQL?

Database	Schema behavior	Ambari impact
MySQL / MariaDB	Database and schema are the same concept	No issue
Oracle 19c	The schema is automatically created per user	No issue
PostgreSQL	The schema must be created manually	Upgrade fails if the schema is missing

Prevention

For new Ambari PostgreSQL deployments, create the schema before running the Ambari DDL:

CREATE DATABASE ambari; CREATE USER ambari WITH PASSWORD 'your_password'; \c ambari CREATE SCHEMA ambari AUTHORIZATION ambari; ALTER ROLE ambari SET search_path TO ambari, public;

Then run the Ambari DDL:

psql -U ambari -d ambari -f Ambari-DDL-Postgres-CREATE.sql

Correct Setup Checklist:

Before running Ambari PostgreSQL DDL, confirm that:

Database ambari exists.
User ambari exists.
password.dat contains the correct password.
Schema ambari exists.
Schema ambari is owned by user ambari.
ALTER ROLE ambari SET search_path TO ambari, public has been run.
Ambari tables are created in the ambari schema, not public.

Upgrade the Ambari Server Database Schema

ambari-server upgrade -v

Upon running the command, the response will be as shown in the screenshot.

Make sure to take the database backup as mentioned earlier and enter 'y' here.

Start the Ambari Server

Once started, you are able to access the Ambari UI using the same url as it was earlier.

ambari-server start

Access the Ambari UI

Use your browser to access the Ambari UI.

Check the Upgraded Version

To verify the upgraded Ambari version, navigate to Admin → About in the top-right corner of the Ambari UI.

All the services also must be working normally without any downtime.

Known Issues

Pip dependency missing

Error:

[root@ce19 ~]# ambari-server upgrade Using python /usr/bin/python3 Upgrading ambari-server Traceback (most recent call last): File "/usr/sbin/ambari-server.py", line 28, in <module> from ambari_commons.exceptions import FatalException, NonFatalException File "/usr/lib/ambari-server/lib/ambari_commons/init.py", line 21, in <module> from ambari_commons.os_check import OSCheck, OSConst File "/usr/lib/ambari-server/lib/ambari_commons/os_check.py", line 25, in <module> import distro ModuleNotFoundError: No module named 'distro'

Workaround:

pip3 install distro

If still hitting the distro module not found error, follow the below steps:

Step 1:

Download the get-pip.py script:

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py

Install pip for Python 3.11:

/usr/bin/python3.11 get-pip.py

Step 2: Install the distro module using pip3.11

Once `pip` is installed for Python 3.11, you can now use it to install the missing `distro` module:

/usr/bin/python3.11 -m pip install distro

Step 3: Verify the installation.

After installing the distro module, verify that it’s correctly installed:

/usr/bin/python3.11 -c "import distro"

The ambari-server upgrade -v upgrade is successfully done.

Mpacks uninstallation not done

Traceback (most recent call last): File "/usr/lib/ambari-server/lib/ambari_simplejson/scanner.py", line 49, in _scan_once nextchar = string[idx] IndexError: string index out of range During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/sbin/ambari-server.py", line 1083, in <module> mainBody() File "/usr/sbin/ambari-server.py", line 1053, in mainBody main(options, args, parser) File "/usr/sbin/ambari-server.py", line 999, in main action_obj.execute() File "/usr/sbin/ambari-server.py", line 78, in execute self.fn(*self.args, **self.kwargs) File "/usr/lib/ambari-server/lib/ambari_server/serverUpgrade.py", line 262, in upgrade retcode = run_schema_upgrade(args) File "/usr/lib/ambari-server/lib/ambari_server/serverUpgrade.py", line 162, in run_schema_upgrade upgrade_response = json.loads(stdout) File "/usr/lib/ambari-server/lib/ambari_simplejson/init.py", line 538, in loads return _default_decoder.decode(s) File "/usr/lib/ambari-server/lib/ambari_simplejson/decoder.py", line 378, in decode obj, end = self.raw_decode(s) File "/usr/lib/ambari-server/lib/ambari_simplejson/decoder.py", line 408, in raw_decode return self.scan_once(s, idx=_w(s, idx).end()) File "/usr/lib/ambari-server/lib/ambari_simplejson/scanner.py", line 93, in scan_once return _scan_once(string, idx) File "/usr/lib/ambari-server/lib/ambari_simplejson/scanner.py", line 51, in _scan_once raise JSONDecodeError(errmsg, string, idx) ambari_simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Workaround: Uninstall all the ambari-mpacks.

ambari-server uninstall-mpack --mpack-name=<installed mpack>

Last updated on Jun 17, 2026

Was this page helpful?