Configure HDP Spark

This page helps you enable Spark metrics via the Graphite sink, allowing Pulse to capture Spark application and JVM metrics.

Configure Spark Metrics Sink

In the Ambari UI:

1. Navigate to Spark > Configs > Advanced spark2-metrics-properties.
2. Add the following properties:

Additional Notes

• On edge nodes not managed by Ambari, manually append the above properties to:

• Ensure the following properties are enabled in spark-defaults.conf for all Spark jobs:

• Apply the same properties in managed configurations for applications running on Spark 1.x and 3.x.

Result

• Spark sends metrics (including JVM metrics) to Graphite for Pulse ingestion.