Configure ODP Spark 2 and 3

This page helps you enable Spark metrics via the Graphite sink, allowing Pulse to capture Spark application and JVM metrics.

Configure Spark Metrics Sink

In the Ambari UI:

Navigate to Spark > Configs > Advanced spark2-metrics-properties.
Update the following parameters:

Bash
    
xxxxxxxxxx
 
# Graphite sink class*.sink.graphite.class=org.apache.spark.metrics.sink.GraphiteSink# Location of your graphite instance*.sink.graphite.host=localhost*.sink.graphite.port=12003*.sink.graphite.protocol=tcp*.sink.graphite.prefix=spark.metrics*.sink.graphite.period=20master.source.jvm.class=org.apache.spark.metrics.source.JvmSourceworker.source.jvm.class=org.apache.spark.metrics.source.JvmSourcedriver.source.jvm.class=org.apache.spark.metrics.source.JvmSourceexecutor.source.jvm.class=org.apache.spark.metrics.source.JvmSource
Copy

Result

Spark 2.x/3.x sends metrics (including JVM metrics) to Graphite for Pulse ingestion.

Last updated on

Was this page helpful?