Livy 3 Limitations

Independent Livy Instances ⚠️

Impact:

• Each Livy version runs as a separate process

• No session sharing between Livy instances

• Client applications must connect to the correct endpoint

Consideration:

• Plan client routing carefully

• Use load balancers if needed for each version

• Document endpoint URLs for teams

Port Management

Impact:

• Each Livy instance requires a unique port

• Firewall rules must allow access to all ports

• Client applications need port-specific configuration

Recommended Port Allocation:

Spark Version Coupling

Impact:

• Each Livy version should use its corresponding Spark version

• Mixing versions can cause compatibility issues

• The configuration must point to the correct Spark installation

Best Practice:

API Compatibility

Impact:

• Different Livy versions may have API differences

• Client applications may need version-specific code

• Behavioral changes between versions

Mitigation:

• Review Livy release notes for API changes

• Test client applications against the target Livy version

• Maintain version-specific client libraries if needed