Acceldata
 ODP
Search documentation… ⌘K

Last updated: Sep 22, 2026 09:47 UTC

Livy 3 FAQs

Q: Can I install only one or two Livy versions?

A: Yes! You can install any combination of the three versions based on your requirements. Installation is independent and modular. Each Livy version comes with its corresponding Spark service installation.

Q: Which Livy version should I use for new applications?

A: Livy 3.5.5 is recommended for all new applications. It provides:

  • Latest bug fixes and features
  • Best compatibility with Spark 3.5.5
  • Active maintenance and updates
  • Improved performance and stability

Q: Can I run multiple sessions from the same Livy instance?

A: Yes! Each Livy instance supports multiple concurrent sessions. The limit depends on:

  • Available cluster resources (YARN capacity)
  • Livy server memory configuration
  • Network bandwidth
  • Typical range: 10-100+ concurrent sessions per Livy server

Q: Do I need separate YARN queues for different Livy versions?

A: Not required, but recommended for:

  • Resource isolation between versions
  • Capacity planning per version
  • Cost allocation tracking
  • Performance monitoring and troubleshooting

Configure queue per session:

curl -X POST http://<host>:8333/sessions \
 -d '{"kind":"spark","conf":{"spark.yarn.queue":"livy_3_3_3_queue"}}'

Q: How do I migrate applications from one Livy version to another?

A: Follow this migration checklist:

  • Test application on the new Livy version
  • Update client endpoint URLs
  • Deploy the updated application
  • Monitor for errors or behavior changes
  • Rollback to the old version if issues arise
  • Decommission the old version after the validation period

Q: Can different Livy versions share the same Spark installation?

A: Not recommended. Each Livy version should use its corresponding Spark version:

  • Livy 3.5.5 → Spark 3.5.5
  • Livy 3.3.3 → Spark 3.3.3
  • Livy 3.5.1 → Spark 3.5.1

Mixing versions can cause:

  • Classpath conflicts
  • API incompatibilities
  • Unpredictable behavior

Q: What happens if I restart the Livy server with active sessions?

A: Active sessions will be terminated:

  • Interactive sessions: Lost (cannot be recovered)
  • Batch jobs: Continue running in YARN (can be monitored via YARN UI)

Best practice

Drain sessions before restart:

[CODE_BLOCK:bash] 

# Check active sessions

curl http://<host>:8333/sessions

# Wait for sessions to complete or delete gracefully
curl -X DELETE http://<host>:8333/sessions/{session-id}

[END_CODE_BLOCK]

Q: Can I load balance across multiple Livy instances of the same version?

A: Yes! For high availability:

  • Deploy multiple Livy servers (same version, different hosts)
  • Use load balancer (HAProxy, Nginx, F5)
  • Configure health checks on /version endpoint

Example Nginx configuration: 

upstream livy_355_cluster {
 server livy-host1:8355 max_fails=3 fail_timeout=30s;
 server livy-host2:8355 max_fails=3 fail_timeout=30s;
}

server {
 listen 80;
 location / {
 proxy_pass http://livy_355_cluster;
 }
}

Q: How do I secure communication between the client and Livy?

A: Multiple security layers:

  • Network Security:
    • Use firewall rules to restrict access
    • VPN for remote access
  • Authentication: 
livy.server.auth.type = kerberos
  • SSL/TLS: 
livy.keystore = /path/to/keystore.jks
  • Authorization: 
livy.server.access-control.enabled = true
livy.server.access-control.users = authorized_users