Using Multiple Spark Versions

Submitting Jobs to Specific Versions

Using spark-submit

Spark 3.5.5:

Bash
    
 
/usr/odp/current/spark3-client/bin/spark-submit \  --class com.example.MyApp \  --master yarn \  --deploy-mode cluster \  myapp.jar
Copy

Spark 3.3.3:

Bash
    
 
/usr/odp/current/spark3_3_3_3-client/bin/spark-submit \  --class com.example.MyApp \  --master yarn \  --deploy-mode cluster \  myapp.jar
Copy

Spark 3.5.1:

Bash
    
 
/usr/odp/current/spark3_3_5_1-client/bin/spark-submit \  --class com.example.MyApp \  --master yarn \  --deploy-mode cluster \  myapp.jar
Copy

Setting Environment Variables

Add to your application launcher script:

For Spark 3.5.5:

Bash
    
 
export SPARK_HOME=/usr/odp/current/spark3-client export SPARK_CONF_DIR=/etc/spark3/conf
Copy

For Spark 3.3.3:

Bash
    
 
export SPARK_HOME=/usr/odp/current/spark3_3_3_3-client export SPARK_CONF_DIR=/etc/spark3_3_3_3/conf
Copy

For Spark 3.5.1:

Bash
    
 
export SPARK_HOME=/usr/odp/current/spark3_3_5_1-client export SPARK_CONF_DIR=/etc/spark3_3_5_1/conf
Copy

Last updated on Jan 28, 2026

Was this page helpful?