Troubleshooting Livy 3

Issue: Livy Server Not Visible in Ambari UI

Symptoms: Livy component doesn't appear when adding Spark service

Solution:

Bash
    
​x
 
# Verify mpack installationambari-server list-mpacks​# Expected output should include spark3 mpacks# spark3-ambari-3.5.5.mpack# spark3-ambari-3.3.3.mpack​# Clear Ambari cacheambari-server stoprm -rf /var/lib/ambari-server/cache/*ambari-server start
Copy

Issue: Port Conflict - "Address already in use"

Symptoms: Livy server fails to start with port binding error

Error Message:

Bash
    
 
Exception in thread "main" java.io.IOException: Failed to bind to /0.0.0.0:8333Caused by: java.net.BindException: Address already in use
Copy

Solution:

Check what's using the port:

Bash
    
 
sudo netstat -tulpn | grep 8333# orsudo lsof -i :8333
Copy

Update Livy configuration:

Bash
    
 
# Edit configuration via Ambari UI:# Services → SPARK3_3_3_3 → Configs → Advanced livy3-3.3.3-conf# Change: livy.server.port = 8334 (or another available port)​# Or edit directly:vi /etc/livy3_3_3_3/conf/livy.conf# Update: livy.server.port = 8334
Copy

Restart Livy Server:

Bash
    
 
# Via Ambari UI or:ambari-agent restart
Copy

Issue: Wrong Spark Version Used

Symptoms: Livy sessions use incorrect Spark version

Solution:

Verify SPARK_HOME in livy-env.sh:

Bash
    
 
# For Livy 3.3.3, should point to Spark 3.3.3cat /etc/livy3_3_3_3/conf/livy-env.sh | grep SPARK_HOME# Expected: export SPARK_HOME=/usr/odp/current/spark3_3_3_3-client
Copy

Update via Ambari:
- Services → SPARK3_3_3_3 → Configs → Advanced livy3-3.3.3-env
- Set correct SPARK_HOME path
- Save and restart Livy
Verify Spark version in session:

Bash
    
 
# Create test sessionSESSION_ID=$(curl -X POST http://localhost:8356/sessions \  -H "Content-Type: application/json" \  -d '{"kind":"spark"}' | jq -r '.id')​# Check Spark versioncurl http://localhost:8356/sessions/$SESSION_ID | jq '.appInfo.sparkUiUrl'
Copy

Issue: Session Creation Fails

Symptoms: REST API returns error when creating sessions

Common Errors and Solutions:

Error: "Session not found"

Bash
    
 
# Check if Livy server is runningcurl http://<host>:8333/version​# Check Livy logstail -f /var/log/livy333/livy-livy-server.out
Copy

Error: "YARN queue not found"

Bash
    
 
{  "conf": {    "spark.yarn.queue": "existing-queue-name"  }}
Copy

Error: "Insufficient resources"

Bash
    
 
# Check YARN capacityyarn application -status​# Reduce resource requirementscurl -X POST http://<host>:8333/sessions \  -H "Content-Type: application/json" \  -d '{    "kind": "spark",    "conf": {      "spark.executor.memory": "1g",      "spark.executor.cores": 1,      "spark.executor.instances": 1    }  }'
Copy

Issue: Sessions Timeout Unexpectedly

Symptoms: Livy sessions terminate before completion

Solution:

Increase session timeout:

Bash
    
 
# /etc/livy3_3_3_3/conf/livy.conflivy.server.session.timeout = 2hlivy.server.session.timeout-check = truelivy.server.session.timeout-check.interval = 60000  # 60 seconds
Copy

Or disable timeout for long-running jobs:

Bash
    
 
livy.server.session.timeout = 0  # Disable timeout
Copy

Issue: Livy UI Shows Wrong Version

Symptoms: Livy REST API reports incorrect version

Solution:

Verify correct binaries are being used:

Bash
    
 
ps aux | grep livy | grep java# Should show correct path like /usr/odp/current/livy3_3_3_3-server
Copy

Check symlinks:

Bash
    
 
ls -la /usr/odp/current/ | grep livy3
Copy

Restart Livy with the correct LIVY_HOME:

Bash
    
 
export LIVY_HOME=/usr/odp/current/livy3_3_3_3-server$LIVY_HOME/bin/livy-server restart
Copy

Session Pooling

Last updated on Jan 28, 2026

Was this page helpful?