Issue: Livy Server Not Visible in Ambari UI
Symptoms: Livy component doesn't appear when adding Spark service
Solution:
# Verify mpack installation
ambari-server list-mpacks
# Expected output should include spark3 mpacks
# spark3-ambari-3.5.5.mpack
# spark3-ambari-3.3.3.mpack
# Clear Ambari cache
ambari-server stop
rm -rf /var/lib/ambari-server/cache/*
ambari-server start
Issue: Port Conflict - "Address already in use"
Symptoms: Livy server fails to start with port binding error
Error Message:
Exception in thread "main" java.io.IOException: Failed to bind to /0.0.0.0:8333
Caused by: java.net.BindException: Address already in use
Solution:
Check what's using the port:
sudo netstat -tulpn | grep 8333
# or
sudo lsof -i :8333
Update Livy configuration:
# Edit configuration via Ambari UI:
# Services → SPARK3_3_3_3 → Configs → Advanced livy3-3.3.3-conf
# Change: livy.server.port = 8334 (or another available port)
# Or edit directly:
vi /etc/livy3_3_3_3/conf/livy.conf
# Update: livy.server.port = 8334
Restart Livy Server:
# Via Ambari UI or:
ambari-agent restart
Issue: Wrong Spark Version Used
Symptoms: Livy sessions use incorrect Spark version
Solution:
Verify SPARK_HOME in livy-env.sh:
# For Livy 3.3.3, should point to Spark 3.3.3
cat /etc/livy3_3_3_3/conf/livy-env.sh | grep SPARK_HOME
# Expected: export SPARK_HOME=/usr/odp/current/spark3_3_3_3-client
Update via Ambari:
Verify Spark version in session:
# Create test session
SESSION_ID=$(curl -X POST http://localhost:8356/sessions \
-H "Content-Type: application/json" \
-d '{"kind":"spark"}' | jq -r '.id')
# Check Spark version
curl http://localhost:8356/sessions/$SESSION_ID | jq '.appInfo.sparkUiUrl'
Issue: Session Creation Fails
Symptoms: REST API returns error when creating sessions
Common Errors and Solutions:
Error: "Session not found"
# Check if Livy server is running
curl http://<host>:8333/version
# Check Livy logs
tail -f /var/log/livy333/livy-livy-server.out
Error: "YARN queue not found"
{
"conf": {
"spark.yarn.queue": "existing-queue-name"
}
}
Error: "Insufficient resources"
# Check YARN capacity
yarn application -status
# Reduce resource requirements
curl -X POST http://<host>:8333/sessions \
-H "Content-Type: application/json" \
-d '{
"kind": "spark",
"conf": {
"spark.executor.memory": "1g",
"spark.executor.cores": 1,
"spark.executor.instances": 1
}
}'
Issue: Sessions Timeout Unexpectedly
Symptoms: Livy sessions terminate before completion
Solution:
Increase session timeout:
# /etc/livy3_3_3_3/conf/livy.conf
livy.server.session.timeout = 2h
livy.server.session.timeout-check = true
livy.server.session.timeout-check.interval = 60000 # 60 seconds
Or disable timeout for long-running jobs:
livy.server.session.timeout = 0 # Disable timeout
Issue: Livy UI Shows Wrong Version
Symptoms: Livy REST API reports incorrect version
Solution:
Verify correct binaries are being used:
ps aux | grep livy | grep java
# Should show correct path like /usr/odp/current/livy3_3_3_3-server
Check symlinks:
ls -la /usr/odp/current/ | grep livy3
Restart Livy with the correct LIVY_HOME:
export LIVY_HOME=/usr/odp/current/livy3_3_3_3-server
$LIVY_HOME/bin/livy-server restart
Session Pooling
