Manage Spark 3 Configurations

Version-Specific Configurations

Each Spark version has independent configuration files:

/etc/spark3/conf/              # Spark 3.5.5

/etc/spark3_3_3_3/conf/        # Spark 3.3.3

/etc/spark3_3_5_1/conf/        # Spark 3.5.1

Modifying Configurations via Ambari

1. Navigate to Services → Select Spark version
2. Click the "Configs" tab
3. Modify settings as needed
4. Save changes
5. Restart affected components

Example: Increase History Server memory for Spark 3.3.3

1. Go to Spark3 3.3.3 service
2. Configs → Advanced spark3-3.3.3-env
3. Set spark_daemon_memory = 2048 (MB)
4. Save and restart the History Server