Using Multiple Livy Versions

Create sessions on specific versions:

Livy 3.5.5 (Port 8355)

# Create interactive Scala session
curl -X POST http://<livy-host>:8355/sessions \
 -H "Content-Type: application/json" \
 -d '{
    "kind": "spark",
    "conf": {
      "spark.executor.memory": "2g",
      "spark.executor.cores": 2
    }
  }'

Livy 3.3.3 (Port 8333)

# Create interactive PySpark session
curl -X POST http://<livy-host>:8333/sessions \
 -H "Content-Type: application/json" \
 -d '{
    "kind": "pyspark",
    "conf": {
      "spark.executor.memory": "2g",
      "spark.executor.cores": 2
    }
  }'

Livy 3.5.1 (Port 8351)

# Create SparkR session
curl -X POST http://<livy-host>:8351/sessions \
 -H "Content-Type: application/json" \
 -d '{
    "kind": "sparkr",
    "conf": {
      "spark.executor.memory": "2g"
    }
  }'