Install Multiple Livy 3

Prerequisites

Before installing multiple Livy3 versions, ensure:

  • ✅ Ambari Server 2.7.8.4-2/3

  • ✅ ODP 3.0+ stack installed

  • ✅ Required dependencies: HDFS, YARN, HIVE

  • ✅ Corresponding Spark3 versions installed

    • Livy 3.5.5 requires Spark 3.5.5
    • Livy 3.3.3 requires Spark 3.3.3
    • Livy 3.5.1 requires Spark 3.5.1

  • ✅ Network ports available:

    • Port 8355 for Livy 3.5.5 (recommended)
    • Port 8351 for Livy 3.5.1
    • Port 8333 for Livy 3.3.3

  • ✅ Sufficient cluster resources for multiple Livy servers

Step 1: Obtain the Management Pack

The Livy management packs are bundled with the corresponding Spark3 mpacks:

Bash
Copy

Step 2: Install the Management Packs

On your Ambari Server, run:

Bash
Copy

Expected Output:

Bash
Copy

Step 3: Add Spark3 Services with Livy via Ambari UI

  1. Log in to Ambari Web UI

    • Navigate to http://<ambari-server>:8080
    • Log in with admin credentials

  2. Add Service

    • Click "Actions" → "Add Service"
    • Select "Spark3 3.5.5" from the service list
    • Click "Next"

  3. Assign Components

    • History Server: Select host(s) for Spark History Server
    • Livy Server: ✅ Enable and select host(s) for Livy3 Server
    • Thrift Server: (Optional) Select host(s) for Thrift Server
    • Client: Select hosts that need Spark client libraries
    • Click "Next"

  4. Configure Service Review default configurations. Key Livy settings:

    • Livy Server Port: 8355 (default for 3.5.5)
    • Log Directory: /var/log/livy3
    • PID Directory: /var/run/livy3
    • Spark Home: /usr/odp/current/spark3-client
    • Click "Next"

  5. Deploy Service

    • Review deployment plan
    • Click "Deploy"
    • Monitor installation progress
    • Click "Complete" when finished

Adding Livy3 3.3.3 or 3.5.1

Repeat the above steps, selecting:

  • Spark 3.3.3 (includes Livy 3.3.3)
  • Spark 3.5.1 (includes Livy 3.5.1)

Ensure different ports are configured for each Livy version to avoid conflicts.

Step 4: Configure Port Allocation

To avoid port conflicts, configure unique ports for each Livy instance:

Livy VersionRecommended PortConfiguration Parameter
Livy 3.5.58355livy.server.port
Livy 3.5.18351livy.server.port
Livy 3.3.38333livy.server.port

Configure via Ambari UI:

  1. Navigate to Services → SPARK3_3_3_3 (or respective version)
  2. Click "Configs" tab
  3. Find "Advanced livy3-3.3.3-conf" section
  4. Set livy.server.port = 8333
  5. Save changes
  6. Restart Livy Server

Step 5: Verify Installation

Check Services in Ambari UI

Navigate to the Ambari dashboard and verify:

  • ✅ All installed Livy3 services show "Started" status
  • ✅ All components are healthy (green indicators)
  • ✅ No port conflicts reported

Verify Livy REST API Endpoints

Access each Livy server's REST API:

Bash
Copy

Each endpoint should return the correct Livy version information:

Bash
Copy

Verify Client Installation

On client nodes:

Bash
Copy

####

Type to search, ESC to discard
Type to search, ESC to discard
Type to search, ESC to discard
  Last updated on Jan 28, 2026
Next to read:
Using Multiple Livy Versions
On This Page
Install Multiple Livy 3PrerequisitesStep 1: Obtain the Management PackStep 2: Install the Management PacksStep 3: Add Spark3 Services with Livy via Ambari UIAdding Livy3 3.5.5 (Recommended)Adding Livy3 3.3.3 or 3.5.1Step 4: Configure Port AllocationStep 5: Verify InstallationCheck Services in Ambari UIVerify Livy REST API EndpointsVerify Client Installation