Acceldata
 ODP
Search documentation… ⌘K

Last updated: Sep 21, 2026 18:51 UTC

Install Multiple Spark 3 Versions

Prerequisites

Before installing multiple Spark3 versions, ensure:

  • ✅ Ambari Server 2.7.0 or higher
  • ✅ ODP 3.0+ stack installed
  • ✅ Required dependencies: HDFS, YARN, HIVE
  • ✅ Sufficient cluster resources for multiple History Servers
  • ✅ Network ports 18082, 18083, 18084 available

Step 1: Obtain the Management Pack

Download the Spark3 management pack:

The mpack includes all three Spark versions ambari-mpacks-odp-5566-spark3-3.5.5.3.2.3.6-2.tar.gz

Step 2: Install the Management Pack

On your Ambari Server, run:

Install the mpack ambari-server install-mpack \\ --mpack=._ambari-mpacks-odp-5566-spark3-3.5.5.3.2.3.6-2.tar.gz \\ --verbose # Restart Ambari Server ambari-server restart

Expected Output: 

INFO: Installing management pack INFO: Loading properties from _etc_ambari-server_conf_ambari.properties INFO: Management pack installed successfully

Step 3: Add Spark3 Services via Ambari UI

Adding Spark3 3.5.5 (Recommended)

  • Log in to Ambari Web UI
    • Navigate to http://<ambari-server>:8080
    • Log in with admin credentials
  • Add Service
    • Click "Actions" → "Add Service"
    • Select "Spark3 3.5.5" from the service list
    • Click "Next"
  • Assign Components
    • History Server: Select host(s) for Spark History Server
    • Thrift Server: (Optional) Select host(s) for Thrift Server
    • Client: Select hosts that need Spark client libraries
    • Click "Next"
  • Configure Service: Review default configurations. Key settings:
    • History Server Port: 18082
    • Log Directory: /var/log/spark3
    • PID Directory: /var/run/spark3
    • Click "Next"
  • Deploy Service
    • Review the deployment plan
    • Click "Deploy"
    • Monitor installation progress
    • Click "Complete" when finished

Adding Spark3 3.3.3 or 3.5.1

Repeat the above steps, selecting: Spark 3.5.1 or Spark 3.3.3

Step 4: Verify Installation

Check Services in Ambari UI

Navigate to the Ambari dashboard and verify:

  • ✅ All installed Spark3 services show "Started" status
  • ✅ All components are healthy (green indicators)

Verify History Server UIs

Access each History Server:

Spark 3.5.5 http:__<host>:18082 # Spark 3.3.3 http:__<host>:18083 # Spark 3.5.1 http:__<host>:18084

Each UI should display the correct Spark version.

Verify Client Installation

On client nodes:

# Check installed packages
rpm -qa | grep spark3

# Verify symlinks
ls -la _usr_odp_current_ | grep spark3

# Test Spark Shell (3.5.5)
_usr_odp_current_spark3-client_bin_spark-shell --version

# Test Spark Shell (3.3.3)
_usr_odp_current_spark3_3_3_3-client_bin_spark-shell --version

# Test Spark Shell (3.5.1)
_usr_odp_current_spark3_3_5_1-client_bin_spark-shell --version