Working with Management Packs

Management packs allow you to deploy a range of services to your Ambari-managed cluster. You can use a management pack to deploy a specific component or service.

Accessing ODP Repositories

The ODP mpacks can be accessed via the links listed below :

Installing Management Pack

In general, when working on installing management packs, perform the following steps :

1. Download Mpack file.
2. Install the management pack using ambari-server install-mpack --mpack=[service_mpack.gz] --verbose.

#Install Ambari mpack

ambari-server install-mpack --mpack=[service_mpack.gz] --verbose

3. Restart the Ambari Server ambari-server restart.
4. Add service from Ambari UI.
5. Select host(s) for respective component(s).
6. Start all stopped or required services.

Uninstalling Management Pack

To uninstall ambari mpack, perform the following steps:

1. Login to Ambari service UI and go to respective service section.
2. Stop respective service and delete permanently.
3. Uninstall the service from CLI using ambari-server uninstall-mpack --mpack-name=[service_mpack.gz].

#Uninstall Ambari mpack

ambari-server uninstall-mpack --mpack-name=[service_mpack.gz]

4. Restart the Ambari Server ambari-server restart.

Hue

Install Hue with Ambari using the following management pack created by Acceldata for ODP clusters.

• Version Key: hue-4.10.0.3.2.2.0-2.tar.gz - Hue-4.10.0 mpack, compatible with ODP 3.2.3.1-2 release.
• Usage Notes: Minimal installation requires Ambari, HDFS, Hive, Tez, Yarn, and Zookeeper.

Install Hue Mpack

To install Ambari Hue Mpack, perform the following:

1. Download the branch from the code and zip with tar.gz extension.
2. Upload the gzipped file to the ambari-server.
3. Execute the below command for installation:

ambari-server install-mpack --mpack=ambari-mpacks-hue-4.10.0.tar.gz --verbose

ambari-server restart

4. Login to the Ambari UI and navigate to Add Hue service.
5. Select the hosts for Hue Server components.
6. Start all stopped or required services.

Hue Custom Database

To create a Hue custom database, perform the following:

1. Get the database details.
2. Create the Hue database and configure the Hue user along with your own password.

Example Using Hue with MySQL

1. Create a new user in MySQL:

• Log in to MySQL as the root user:

mysql -u root -p

• Create a new database for Hue:

CREATE DATABASE hue;

• Create a new user for Hue:

CREATE USER $HUEUSER IDENTIFIED BY '$HUEPASSWORD';

• Grant all privileges to the new user on all databases:

GRANT ALL PRIVILEGES ON *.* TO '$HUEUSER'@'localhost' WITH GRANT OPTION;

GRANT ALL ON $HUEUSER.* TO '$HUEUSER'@'localhost' IDENTIFIED BY '$HUEPASSWORD';

GRANT ALL ON $HUEUSER.* TO '$HUEUSER'@'%' IDENTIFIED BY '$HUEPASSWORD';

• Flush privileges to apply changes:

FLUSH PRIVILEGES;

Note Ensure to replace $HUEUSER with the desired Hue username and $HUEPASSWORD with the desired Hue user password.

Uninstall Hue Mpack

To uninstall Ambari Hue Mpack, perform the following steps:

1. Login to Ambari Service and Navigate to Hue Section:

   • Access the Ambari service web interface.
   • Navigate to the Hue section in the Ambari dashboard.

2. Stop the Hue Service and Delete Permanently:

   • Locate the option to stop the Hue service in the Ambari dashboard.
   • Stop the Hue service and confirm the action.
   • Delete the Hue service permanently from the Ambari dashboard.

3. Open a terminal or command prompt on the Ambari server.

   • Run the following command to uninstall the Hue Mpack:

ambari-server uninstall-mpack --mpack-name=hue-ambari.mpack

Replace hue-ambari.mpack with the actual name of the Hue Mpack if it differs.

4. Restart the Ambari Server:
   • After uninstalling the Hue Mpack, restart the Ambari server to apply the changes.

ambari-server restart

These steps will uninstall the Ambari Hue Mpack from your Ambari service.

Troubleshooting Steps for Hue Known Issues

Issue: A bug exists in User Group Management for Ambari 2.8 & ODP 3.x.

Workaround: Execute the following Python command before installing Hue:

/var/lib/ambari-server/resources/scripts/configs.py -u admin -p admin -n [CLUSTER_NAME] -l [CLUSTER_FQDN] -t 8080 -a set -c cluster-env -k ignore_groupsusers_create -v true

Make sure to replace [CLUSTER_NAME] and [CLUSTER_FQDN] with the correct values for your cluster.

Impala

Install Impala with Ambari using the following management pack created by Acceldata for ODP clusters.

Install Impala Mpack

To install Impala Mpack, perform the following steps:

1. Download the branch from the code repository and compress it into a tar.gz file.
2. Upload the compressed file to the Ambari server.
3. Execute the following command to install the Mpack:

ambari-server install-mpack --mpack=ambari-mpacks-impala-4.1.2.tar.gz -v

4. Restart the Ambari server:

ambari-server restart

5. Log in to the Ambari UI and navigate to Add Impala service.
6. Select hosts for the Catalog & Statestore components.
7. Choose daemons from the host list and proceed with service deployment to complete the installation.
8. Start any stopped or required services to finalize the installation.

Uninstall Impala Mpack

To uninstall Ambari Impala Mpack, perform the following:

1. Login to Ambari Service and Navigate to Impala Section:

   • Access the Ambari service web interface.
   • Navigate to the Impala section in the Ambari dashboard.

2. Stop the Impala Service and Delete Permanently:

   • Locate the option to stop the Impala service in the Ambari dashboard.
   • Stop the Impala service and confirm the action.
   • Delete the Impala service permanently from the Ambari dashboard.

3. Execute Uninstallation Command:

   • Open a terminal or command prompt on the Ambari server.
   • Run the following command to uninstall the Impala Mpack:

ambari-server uninstall-mpack --mpack=ambari-impala-mpack -v

Replace ambari-impala-mpack with the actual name of the Impala Mpack if it differs.

4. Restart the Ambari Server:
   • After uninstalling the Impala Mpack, restart the Ambari server to apply the changes:

ambari-server restart

These steps will uninstall the Ambari Impala Mpack from your Ambari service.

Troubleshooting Steps for Impala Known Issues

1. Issue 1: Ranger Authorization Enablement

The file ranger-hive-security.xml is missing in all daemons except on Hive servers, leading to start failures. Additionally, if audit is enabled, the file ranger-hive-audit.xml is also unavailable.

F0214 14:55:06.908636 40022 frontend.cc:148] InternalException: Unable to instantiate authorization provider: org.apache.impala.authorization.ranger.RangerAuthorizationFactory

CAUSED BY: InvocationTargetException: null

CAUSED BY: IllegalArgumentException: bound must be positive

. Impalad exiting.

Solution

Copy the ranger-hive-security.xml and ranger-hive-audit.xml files from hive-servers to /etc/hive/conf/ as enabled with Ranger.

2. Issue 2: Solr Audit Enablement

When Solr is enabled for Ranger audit, daemons fail to start on infra-solr-clients due to broken symlinks in the underlying impala-solr libraries.

Solution

To resolve the issue of broken symlinks for Solr in the impala-solr libraries, install ambari-infra-solr on all daemons.

3. Issue 3: Missing Ranger Cache Policies

Missing cache policies on other launched daemons or Impala components.

2023-02-06T10:19:21,074 ERROR [main] org.apache.ranger.plugin.contextenricher.RangerTagEnricher$RangerTagRefresher - failed to save service-tags to cache file '/etc/ranger/adodp_hive/policycache/impala_adodp_hive_tag.json'

java.io.FileNotFoundException: /etc/ranger/adodp_hive/policycache/impala_adodp_hive_tag.json (No such file or directory)

at java.io.FileOutputStream.open0(Native Method) ~[?:1.8.0_342]

at java.io.FileOutputStream.open(FileOutputStream.java:270) ~[?:1.8.0_342]

Solution

Create Ranger policy cache directories under /etc/ranger/conf and copy the policy cache files from the respective policy directories.

Accessing Impala Web UI Consoles

Each of the Impala daemons (impalad, statestored, and catalogd) includes a built-in web server that displays diagnostic and status information.

Catalogd WebUI Console

The catalogd Web UI includes information about the databases, tables, and other objects managed by Impala, in addition to the resource usage and configuration settings of the catalogd. Because there is only a single instance of the catalogd within any Impala cluster, you access the Web UI only on the particular host that serves as the Impala Catalog Server.

http://<catalog-host-name>:25020

Statesoted WebUI Console

The statestored Web UI includes information about memory usage, configuration settings, and ongoing health checks performed by statestored. Because there is only a single instance of the statestored within any Impala cluster, you can access the Web UI only on a particular host that serves as the Impala StateStore.

http://<Statestore-host-name>:25010

Impald WebUI Console

The impalad Web UI includes information about configuration settings, running and completed queries, and associated performance and resource usage for queries. In particular, the Details link for each query displays alternative views of the query including a graphical representation of the plan, and the output of the EXPLAIN, SUMMARY, and PROFILE statements from impala-shell. Each host that runs the impalad daemon has its own instance of the Web UI, with details about those queries for which that host served as the coordinator. The impalad Web UI is primarily used for diagnosing query problems that can be traced to a particular node.

http://<Impalad-host-name>:25000

For more information about accessing the Impala Web interface, see Impala Web User Interface for Debugging.

NiFi

Install NiFi with Ambari using the following management pack created by Acceldata for ODP clusters.

Version Key

• nifi-1.23.2.tar.gz - Nifi-1.23.2 mpack

Usage Notes

• Minimal installation requires Ambari, HDFS, and Zookeeper.

Install NiFi Mpack

To install Ambari NiFi Mpack, perform the following steps:

1. Download the branch from the code repository and compress it into a tar.gz file.
2. Upload the compressed file to the Ambari server.
3. Execute the following command to install the Mpack:

ambari-server install-mpack --mpack=ambari-mpacks-nifi-1.23.2.tar.gz --verbose

4. Restart the Ambari server:

ambari-server restart

5. Log in to the Ambari UI and navigate to Add NiFi service.
6. Select hosts for the NiFi components.
7. Start any stopped or required services to finalize the installation.

Uninstall NiFi Mpack

To uninstall Ambari NiFi Mpack, perform the following steps:

1. Log in to Ambari Service and Navigate to NiFi Section:

   • Access the Ambari service web interface.
   • Navigate to the NiFi section in the Ambari dashboard.

2. Stop the NiFi Service and Delete Permanently:

   • Locate the option to stop the NiFi service in the Ambari dashboard.
   • Stop the NiFi service and confirm the action.
   • Delete the NiFi service permanently from the Ambari dashboard.

3. Execute Uninstallation Command:

   • Open a terminal or command prompt on the Ambari server.
   • Run the following command to uninstall the Impala Mpack:

ambari-server uninstall-mpack --mpack-name=nifi-ambari-mpack

4. Restart the Ambari Server:
   • After uninstalling the NiFi Mpack, restart the Ambari server to apply the changes:

ambari-server restart

These steps will uninstall the Ambari NiFi Mpack from your Ambari service.

Troubleshooting Steps for NiFi Known Issues

1. Issue 1: User Group Management for Ambari 2.8 & ODP 3.x.

Solution

Run the following Python code before installing NiFi:

/var/lib/ambari-server/resources/scripts/configs.py -u admin -p admin -n [CLUSTER_NAME] -l [CLUSTER_FQDN] -t 8080 -a set -c cluster-env -k  ignore_groupsusers_create -v true

Replace the CLUSTER_NAME and CLUSTER_FQDN with the correct names.

Spark 3

Install Spark 3 with Ambari using this management pack created by Acceldata for ODP clusters.

Version Key

• spark3_mpack-3.3.3.tar.gz - Spark3.3.3 mpack, compatible with ODP 3.2.3.1-2 release.

Usage Notes

• Minimal installation requires Ambari, HDFS, Hive, Tez, Yarn, and Zookeeper.

Install Spark 3 Mpack

To install Ambari Spark 3 Mpack, perform the following steps:

1. Download the branch from the code repository and compress it into a tar.gz file.
2. Upload the compressed file to the Ambari server.
3. Execute the following command to install the Mpack:

ambari-server install-mpack --mpack=ambari-mpacks-spark3-3.3.3.tar.gz --verbose

In case you find the following errors, run the following commands as mentioned:

(mpack_metadata, mpack_name, mpack_version, mpack_staging_dir, mpack_archive_path) = _install_mpack(options, replay_mode)

  File "/usr/lib/ambari-server/lib/ambari_server/setupMpacks.py", line 798, in _install_mpack

    process_stack_addon_service_definitions_artifact(artifact, artifact_source_dir, options)

  File "/usr/lib/ambari-server/lib/ambari_server/setupMpacks.py", line 557, in process_stack_addon_service_definitions_artifact

    sudo.symlink(source_service_version_path, dest_link)

  File "/usr/lib/ambari-server/lib/resource_management/core/sudo.py", line 130, in symlink

    os.symlink(source, link_name)

OSError: [Errno 17] File exists

​

ambari-server uninstall-mpack --mpack-name=spark3-ambari-3.3.3.mpack

​

rm -rf /var/lib/ambari-server/data/tmp/ambari-mpacks-spark3-3.3.3*/ /var/lib/ambari-server/resources/stacks/ODP/3.0/services/SPARK3 /var/lib/ambari-server/resources/stacks/ODP/3.1/services/SPARK3 /var/lib/ambari-server/resources/common-services/SPARK3 /var/lib/ambari-agent/cache/stacks/ODP/3.0/services/SPARK3 /var/lib/ambari-agent/cache/stacks/ODP/3.1/services/SPARK3 /var/lib/ambari-agent/cache/common-services/SPARK3 /var/lib/ambari-agent/cache/stacks/ODP/3.2/services/SPARK3 /var/lib/ambari-server/resources/stacks/ODP/3.2/services/SPARK3

Incase of the above failures, run the installation command:

ambari-server install-mpack --mpack=ambari-mpacks-spark3-3.3.3.tar.gz --verbose

​

cd /var/lib/ambari-server/resources/stacks/ODP/3.0/services/

rm -f SPARK3

ln -s /var/lib/ambari-server/resources/mpacks/spark3-ambari-3.3.3.mpack-3.3.3/common-services/SPARK3/3.3.3 SPARK3

cd ../../3.1/services/

rm -f SPARK3

ln -s /var/lib/ambari-server/resources/mpacks/spark3-ambari-3.3.3.mpack-3.3.3/common-services/SPARK3/3.3.3 SPARK3

​

ambari-server restart

After running the above commands, the following message is shown:

INFO: Management pack spark3-ambari-3.3.3.mpack-3.3.3 successfully installed! Please restart ambari-server.

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

Ambari Server 'install-mpack' completed successfully.

4. Log in to the Ambari UI and navigate to Add Spark 3 service.
5. Select hosts for the Spark thrift and client components.
6. Start any stopped or required services to finalize the installation.

Uninstall Spark 3 Mpack

To uninstall Ambari Spark 3 Mpack, perform the following steps:

1. Log in to Ambari Service and Navigate to Spark 3 Section:

   • Access the Ambari service web interface.
   • Navigate to the Spark 3 section in the Ambari dashboard.

2. Stop the Spark 3 Service and Delete Permanently:

   • Locate the option to stop the Spark 3 service in the Ambari dashboard.
   • Stop the Spark 3 service and confirm the action.
   • Delete the Spark 3 service permanently from the Ambari dashboard.

3. Execute Uninstallation Command:

   • Open a terminal or command prompt on the Ambari server.
   • Run the following command to uninstall the Spark 3 Mpack:

ambari-server uninstall-mpack --mpack-name=spark3-ambari-3.3.3.mpack

4. Restart the Ambari Server:
   • After uninstalling the Spark 3 Mpack, restart the Ambari server to apply the changes:

ambari-server restart

These steps will uninstall the Ambari Spark 3 Mpack from your Ambari service.

Troubleshooting Steps for Spark 3 Known Issues

1. Issue 1: Issue in User Group Management for Ambari 2.8 & ODP 3.x.

Solution

Run the following Python command before installing Spark 3:

/var/lib/ambari-server/resources/scripts/configs.py -u admin -p admin -n [CLUSTER_NAME] -l [CLUSTER_FQDN] -t 8080 -a set -c cluster-env -k  ignore_groupsusers_create -v true

Replace the [CLUSTER_NAME] __and __[CLUSTER_FQDN] in the command above with the correct names.

HttpFS

Ambari HttpFS Service

This is a straightforward Ambari extension for the HttpFS HDFS service.

Install HttpFS Mpack

To install Ambari HttpsFS Mpack, perform the following steps:

1. Download the branch from the code repository and compress it into a tar.gz file.
2. Upload the compressed file to the Ambari server.
3. Execute the following command to install the Mpack:

ambari-server install-mpack --mpack=ambari-mpacks-httpfs-2.7.1.tar.gz --verbose

4. Restart the Ambari Service:

ambari-server restart

5. Log in to the Ambari UI and navigate to Add HttpFS service.
6. Select hosts for the Httpfs Server components.
7. Start any stopped or required services to finalize the installation.

Uninstall HttpFS Mpack

To uninstall Ambari HttpFS Mpack, perform the following steps:

1. Login to Ambari Service and Navigate to HttpFS Section:

   • Access the Ambari service web interface.
   • Navigate to the HttpFS section in the Ambari dashboard.

2. Stop the HttpFS Service and Delete Permanently:

   • Locate the option to stop the HttpFS service in the Ambari dashboard.
   • Stop the HttpFS service and confirm the action.
   • Delete the HttpFS service permanently from the Ambari dashboard.

3. Execute Uninstallation Command:

   • Open a terminal or command prompt on the Ambari server.
   • Run the following command to uninstall the HttpFS Mpack:

ambari-server uninstall-mpack --mpack-name=httpfs-ambari-mpack

4. Restart the Ambari Server:
   • After uninstalling the HttpFS Mpack, restart the Ambari server to apply the changes:

ambari-server restart

These steps will uninstall the Ambari HttpFS Mpack from your Ambari service.

Troubleshooting Steps for HttpFS Known Issues

Issue 1 : Bug in User Group Management for Ambari 2.8 & ODP 3.x.

Solution

Run the following Python command before installing HttpFS:

/var/lib/ambari-server/resources/scripts/configs.py -u admin -p admin -n [CLUSTER_NAME] -l [CLUSTER_FQDN] -t 8080 -a set -c cluster-env -k  ignore_groupsusers_create -v true

Ensure the correct [CLUSTER_NAME] and [CLUSTER_FQDN] names are used.

License: Apache License version 2.0

Ozone

Install Ozone with Ambari using this management pack created by Acceldata. Note Ozone provided in this distribution supports HA only.

Version Key

• ozone-1.4.0.tar.gz - Ozone 1.4.0 mpack, compatible from ODP v3.2.3.0-2 release onwards

Usage Notes

• Minimal install required Ambari
• For Authentication integration it is compatible with Kerberos
• For Authorization integration it requires Ranger with auditing disabled due to performance issues in upstream.

Install Ozone Mpack

To install Ambari Ozone Mpack, perform the following steps:

1. Download the branch from the code repository and compress it into a tar.gz file.
2. Upload the compressed file to the Ambari server.
3. Execute the following command to install the Mpack:

ambari-server install-mpack --mpack=ambari-mpacks-ozone-1.4.0.tar.gz --verbose

After running the above commands, the following message is shown:

INFO: Management pack ozone-ambari-mpack-1.4.0 successfully installed! Please restart ambari-server.

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

Ambari Server 'install-mpack' completed successfully.

4. Create Service Symlink directory:

cd /var/lib/ambari-server/resources/stacks/ODP/3.0/services/

rm -f OZONE

ln -s /var/lib/ambari-server/resources/mpacks/ozone-ambari.mpack-1.4.0/common-services/OZONE/1.4.0 OZONE

cd ../../3.1/services/

rm -f OZONE

ln -s /var/lib/ambari-server/resources/mpacks/ozone-ambari.mpack-1.4.0/common-services/OZONE/1.4.0 OZONE

cd ../../3.2/services/

rm -f OZONE

ln -s /var/lib/ambari-server/resources/mpacks/ozone-ambari.mpack-1.4.0/common-services/OZONE/1.4.0 OZONE

5. Restart the Ambari service:

ambari-server restart

6. Log in to the Ambari UI and navigate to Add Ozone service.
7. Select atleast three hosts for the Ozone Manager, three hosts for Storage Container Manager, hosts for S3 gateway, host for Recon server, and atleast three nodes for Data node.
8. Configure Ozone as per your use case and deploy the service.
9. Start any stopped or required services to finalize the installation.

Uninstall Ozone Mpack

To uninstall Ambari Ozone Mpack, perform the following steps:

1. Login to Ambari Service and Navigate to Ozone Section:

   • Access the Ambari service web interface.
   • Navigate to the Ozone section in the Ambari dashboard.

2. Stop the Ozone Service and Delete Permanently:

   • Locate the option to stop the Ozone service in the Ambari dashboard.
   • Stop the Ozone service and confirm the action.
   • Delete the Ozone service permanently from the Ambari dashboard.

3. Execute Uninstallation Command:

   • Open a terminal or command prompt on the Ambari server.
   • Run the following command to uninstall the Ozone Mpack:

ambari-server uninstall-mpack --mpack-name=ozone-ambari.mpack

4. Restart the Ambari Server:
   • After uninstalling the Ozone Mpack, restart the Ambari server to apply the changes:

ambari-server restart

Registry

Install Schema Registry with Ambari using this management pack created by Acceldata for ODP Clusters.

Version Key

• registry-1.0.0.tar.gz - Registry 1.0.0 mpack, compatible from ODP v3.2.3.1-2 release onwards.

Usage Notes

• Minimal installation requires Ambari.
• For authentication integration, it is compatible with Knox, OAUTH2 and Kerberos.
• For authorization integration, it requires Ranger and for audit enablement it requires HDFS.

Install Registry Mpack

To install Ambari Registry Mpack, perform the following steps:

1. Download the branch from the code repository and compress it into a tar.gz file.
2. Upload the compressed file to the Ambari server.
3. Execute the following command to install the Mpack:

ambari-server install-mpack --mpack=ambari-mpacks-registry-1.0.0.tar.gz --verbose

After running the above commands, the following message is shown:

INFO: Management pack registry-ambari-mpack-1.0.0 successfully installed! Please restart ambari-server.

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

Ambari Server 'install-mpack' completed successfully.

But it requires a few symlink changes before Ambari server restart:

cd /var/lib/ambari-server/resources/stacks/ODP/3.0/services/

rm -f REGISTRY

ln -s /var/lib/ambari-server/resources/mpacks/registry-ambari-mpack-1.0.0/common-services/REGISTRY/1.0.0 REGISTRY

cd ../../3.1/services/

rm -f REGISTRY

ln -s /var/lib/ambari-server/resources/mpacks/registry-ambari-mpack-1.0.0/common-services/REGISTRY/1.0.0 REGISTRY

cd ../../3.2/services/

rm -f REGISTRY

ln -s /var/lib/ambari-server/resources/mpacks/registry-ambari-mpack-1.0.0/common-services/REGISTRY/1.0.0 REGISTRY

​

ambari-server restart

4. Log in to the Ambari UI and navigate to Add Schema Registry service.
5. Select hosts for the Registy Server components.
6. Start any stopped or required services to finalize the installation.

Uninstall Registry Mpack

To uninstall Ambari Schema Registry Mpack, perform the following steps:

1. Login to Ambari Service and Navigate to Schema Registry Section:

   • Access the Ambari service web interface.
   • Navigate to the Schema Registry section in the Ambari dashboard.

2. Stop the Registry Service and Delete Permanently:

   • Locate the option to stop the Registry service in the Ambari dashboard.
   • Stop the Registry service and confirm the action.
   • Delete the Registry service permanently from the Ambari dashboard.

3. Execute Uninstallation Command:

   • Open a terminal or command prompt on the Ambari server.
   • Run the following command to uninstall the Registry Mpack:

ambari-server uninstall-mpack --mpack-name=registry-ambari-mpack

4. Restart the Ambari Server:
   • After uninstalling the Registry Mpack, restart the Ambari server to apply the changes:

ambari-server restart

These steps will uninstall the Schema Registry Mpack from your Ambari service.

Airflow

Install Apache Airflow with Ambari using this management pack created by Acceldata for ODP clusters.

Version Key

• Apache Airflow 2.8.1 - Apache Airflow 1.0.0 mpack, compatible from ODP v3.3.0 release onwards.

Usage Notes

• Minimal installation requires Ambari.
• For Authentication integration it is compatible with LDAP and Kerberos

Install Apache Airflow Mpack

To install Airflow Mpack, perform the following steps:

1. Download the branch from the code repository and compress it into a tar.gz file.
2. Upload the compressed file to the Ambari server.
3. Execute the following command to install the Mpack:

ambari-server install-mpack --mpack=ambari-mpacks-airflow-2.8.1.tar.gz --verbose

After running the above command, the following message is shown:

INFO: Management pack airflow-ambari-mpack-2.8.1 successfully installed! Please restart ambari-server.

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

Ambari Server 'install-mpack' completed successfully.

Change Simlinks before restarting the Ambari Server:

cd /var/lib/ambari-server/resources/stacks/ODP/3.0/services

unlink AIRFLOW

ln -s /var/lib/ambari-server/resources/mpacks/airflow-ambari-mpack-2.8.1/common-services/AIRFLOW/2.8.1 AIRFLOW

cd /var/lib/ambari-server/resources/stacks/ODP/3.1/services

unlink AIRFLOW

ln -s /var/lib/ambari-server/resources/mpacks/airflow-ambari-mpack-2.8.1/common-services/AIRFLOW/2.8.1 AIRFLOW

cd /var/lib/ambari-server/resources/stacks/ODP/3.2/services

unlink AIRFLOW

ln -s /var/lib/ambari-server/resources/mpacks/airflow-ambari-mpack-2.8.1/common-services/AIRFLOW/2.8.1 AIRFLOW

​

ambari-server restart

4. Log in to the Ambari UI and navigate to Add Apache Airflow service.
5. Select hosts for the Apache Airflow server components.
6. Start any stopped or required services to finalize the installation.

Uninstall Ambari Airflow Mpack

To uninstall Ambari Airflow Mpack, perform the following steps:

1. Log in to Ambari Service and Navigate to Apache Airflow Section:

   • Access the Ambari service web interface.
   • Navigate to the Apache Airflow section in the Ambari dashboard.

2. Stop the Apache Airflow Servers and Delete Permanently:

   • Locate the option to stop the Apache Airflow service in the Ambari dashboard.
   • Stop the Airflow service and confirm the action.
   • Delete the Airflow service permanently from the Ambari dashboard.

3. Execute Uninstallation Command:

   • Open a terminal or command prompt on the Ambari server.
   • Run the following command to uninstall the Apache Airflow Mpack:

ambari-server uninstall-mpack --mpack-name=airflow-ambari-mpack

4. Restart the Ambari Server:
   • After uninstalling the Airflow Mpack, restart the Ambari server to apply the changes:

ambari-server restart

Isilon OneFS

README for Isilon Ambari Management Pack 1.0.3.0 for OneFS

The Isilon Ambari Management Pack for OneFS is a software component that integrates OneFS as a service into a Hadoop cluster via Ambari. This management pack allows an Ambari administrator to manage OneFS as a HDFS storage service, providing native namenode and datanode capabilities similar to traditional HDFS. The management pack is OneFS release-independent and can be updated between releases if necessary.

Prerequisites

This management pack is compatible with the following OneFS versions:

• OneFS 8.1.2
• OneFS 8.2.0
• OneFS 8.2.1
• OneFS 8.2.2
• OneFS 9.0.0

Install OneFS Mpack

To install the Isilon management pack, perform the following steps:

1. Run the installation command:

ambari-server install-mpack --mpack=ambari-mpacks-isilon-onefs-1.0.3.0.tar.gz --verbose

2. Update the symbolic links for the OneFS service:

cd /var/lib/ambari-server/resources/stacks/ODP/3.0/services/

rm -f ONEFS

ln -s /var/lib/ambari-server/resources/mpacks/onefs-ambari-mpack-1.0.3.0/common-services/ONEFS/1.0.0/ ONEFS

​

cd ../../3.1/services/

rm -f ONEFS

ln -s /var/lib/ambari-server/resources/mpacks/onefs-ambari-mpack-1.0.3.0/common-services/ONEFS/1.0.0/ ONEFS

​

cd ../../3.2/services/

rm -f ONEFS

ln -s /var/lib/ambari-server/resources/mpacks/onefs-ambari-mpack-1.0.3.0/common-services/ONEFS/1.0.0/ ONEFS

3. Restart the Ambari Server:

ambari-server restart

After running the above commands, you will see the following message:

INFO: Management pack onefs-ambari-mpack successfully installed! Please restart ambari-server.

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

Ambari Server 'install-mpack' completed successfully.

To uninstall the management pack, run the following command:

ambari-server uninstall-mpack --mpack-name=onefs-ambari-mpack

Then, remove the following directories:

rm -rf /var/lib/ambari-server/data/tmp/isilon-onefs-mpack*/ /var/lib/ambari-server/resources/stacks/ODP/3.0/services/ONEFS /var/lib/ambari-server/resources/stacks/ODP/3.1/services/ONEFS /var/lib/ambari-server/resources/common-services/ONEFS /var/lib/ambari-agent/cache/stacks/ODP/3.0/services/ONEFS /var/lib/ambari-agent/cache/stacks/ODP/3.1/services/ONEFS /var/lib/ambari-agent/cache/common-services/ONEFS /var/lib/ambari-agent/cache/stacks/ODP/3.2/services/ONEFS

Important Notes

• A performance issue with OneFS and Ambari has been identified when Apache HBase is configured. Setting the hbase.wal.provider value to filesystem resolves this issue.
• When installing the 1.0.3.0 Ambari Management Pack on a Hadoop cluster with a new deployment of ODP, the hbase.wal.provider value will be automatically set to filesystem.
• If the Hadoop cluster already has Ambari and HBase configured, the hbase.wal.provider value must be manually configured in the Ambari UI. Follow these steps to configure it:
  1. Log in to the Ambari UI.
  2. Click the 'HBase' service in the left panel, then click the Configs tab, and finally click the Advanced tab.
  3. In the Custom hbase-site section, look for the hbase.wal.provider key. If it does not exist, add it by clicking Add Property.
  4. Enter the following in the Properties field: hbase.wal.provider=filesystem
  5. Click ADD and then Save to save the configuration. Restart Ambari if prompted to apply the setting.

Resolved Issues

• Setting the hbase.wal.provider value to filesystem in the Ambari UI resolves the performance issue.
• The OneFS Release Notes can be found on the customer support site: Support for PowerScale OneFS | Overview | Dell US
• The following documents, available on the customer support site, provide instructions for downloading and installing the Ambari Management Pack for Isilon OneFS:
  • PowerScale OneFS With Hadoop and Installation Guide
  • PowerScale OneFS With Hadoop and Hadoop for Kerberos Installation Guide

Upgrade Isilon Mpack

Run the following commands to upgrade to Isilon Ambari Management Pack 1.0.3.0 and then to restart the Ambari server:

ambari-server upgrade-mpack --mpack=

​

ambari-server restart

Kudu

Install Kudu with Ambari using this management pack created by Acceldata for ODP clusters.

Installing Kudu Mpack

To install Kudu Mpack, perform the following steps:

1. Download the branch from the code repository and compress it into a tar.gz file.
2. Upload the compressed file to the Ambari server.
3. Execute the following command to install the Mpack:

ambari-server install-mpack --mpack=ambari-mpacks-kudu-1.17.0.tar.gz --verbose

4. Log on to the Ambari UI and add the Kudu service.
5. Select the hosts for Kudu Masters and Tablet Servers. You must select at least one of each, with a recommended maximum of up to 100 tablet servers. You must select an odd number for a minimum of 3 master servers.

Uninstall Kudu Airflow Mpack

To uninstall Ambari Kudu Mpack, perform the following steps:

1. Log in to the Ambari UI and navigate to the Kudu section.
2. Stop the Kudu service, then delete the service.
3. Run the following command to remove the Kudu mpack.

ambari-server uninstall-mpack --mpack=kudu-ambari-mpack

ambari-server restart