Configure ODP Hive and Tez

This page describes how to enable Hive query statistics, configure JMX for HiveServer2 and Metastore, and place Pulse hook JARs for Hive and Tez so that Pulse can collect query data and metrics.

View Hive Table Details in Pulse

To display Hive table details with data in Pulse:

  1. Enable automatic statistics gathering. In the hive-site.xml file, set the following properties to true:

hive.stats.autogather hive.stats.column.autogather

This allows Hive to compute table statistics automatically.

  1. Compute statistics manually (optional). You can also run the following Hive command to compute table statistics manually:

ANALYZE TABLE <table_name> COMPUTE STATISTICS;

Configure HiveServer2 JMX

In the Ambari UI:

  1. Navigate to Hive > Configs > Advanced hive-env > hive-env template.

  2. In hive-envtemplate``, choose and add one of the following JMX configurations based on your security requirements.

Enable JMX without Security on JMX Remote Port

To enable JMX port without any security, add the following parameters at the end of the file:

if [ "$SERVICE" = "hiveserver2" ]; then export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=8008" fi
Warning

Avoid JMX Changes for Hive 1.x Using the MR Engine

Do not enable JMX for Hive 1.x when using the MapReduce (MR) execution engine. A known bug can cause query failures if JMX is enabled.

Enable Basic Authentication on JMX Remote Port (Optional)

To enable basic authentication on the JMX remote port, add the following parameters:

if [ "$SERVICE" = "hiveserver2" ]; then export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=true -Dcom.sun.management.jmxremote.access.file=</path/to/jmxremote.access/file> -Dcom.sun.management.jmxremote.password.file=</path/to/jmxremote.password/file> -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=8008" fi

Enable TLS/SSL on JMX Remote Port (Optional)

To enable TLS/SSL authentication on the JMX remote port, add the following parameters:

if [ "$SERVICE" = "hiveserver2" ]; then export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=true -Dcom.sun.management.jmxremote.access.file=</path/to/jmxremote.access/file> -Dcom.sun.management.jmxremote.password.file=</path/to/jmxremote.password/file> -Dcom.sun.management.jmxremote.ssl=true -Dcom.sun.management.jmxremote.registry.ssl=true -Djavax.net.ssl.keyStore=</path/to/keystore.jks/file> -Djavax.net.ssl.keyStorePassword=<Keystore Password> -Djavax.net.ssl.trustStore=</path/to/truststore.jks/file> -Djavax.net.ssl.trustStorePassword=<Truststore Password> -Dcom.sun.management.jmxremote.port=8008" fi

Configure Hive Metastore JMX

In the Ambari UI:

  1. Navigate to Hive > Configs > Advanced hive-env > hive-env template.

  2. In hive-env template, choose and add one of the following JMX configurations based on your security requirements.

Enable JMX without Security on JMX Remote Port

To enable JMX port without any security, add the following parameters at the end of the file:

if [ "$SERVICE" = "metastore" ]; then export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=8009" fi

Enable JMX With Basic Authentication (Optional)

To enable basic authentication on the JMX remote port, add the following parameters:

if [ "$SERVICE" = "metastore" ]; then export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=true -Dcom.sun.management.jmxremote.access.file=</path/to/jmxremote.access/file> -Dcom.sun.management.jmxremote.password.file=</path/to/jmxremote.password/file> -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=8008" fi

Enable JMX With TLS/SSL Authentication (Optional)

To enable TLS/SSL authentication on the JMX remote port, add the following parameters:

if [ "$SERVICE" = "metastore" ]; then export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=true -Dcom.sun.management.jmxremote.access.file=</path/to/jmxremote.access/file> -Dcom.sun.management.jmxremote.password.file=</path/to/jmxremote.password/file> -Dcom.sun.management.jmxremote.ssl=true -Dcom.sun.management.jmxremote.registry.ssl=true -Djavax.net.ssl.keyStore=</path/to/keystore.jks/file> -Djavax.net.ssl.keyStorePassword=<Keystore Password> -Djavax.net.ssl.trustStore=</path/to/truststore.jks/file> -Djavax.net.ssl.trustStorePassword=<Truststore Password> -Dcom.sun.management.jmxremote.port=8008" fi

Place Pulse hook JARs and Set Properties (Hive and Tez)

Pulse uses Hive and Tez hooks to capture query statistics.

Hook JAR Mapping

Distro Version

Tez Version

Hive Version

Pulse Hook Jar Name

ODP

0.10.1

3.1.4

ad-hive-hook_odp_3.1.4-assembly-X.Y.Z.jar

ODP

0.10.1.3.2.3.5-3

3.1.4.3.2.3.5-3

ad-hive-hook_odp_ 3.1.4.3.2.3.5-3-assembly-X.Y.Z.jar

ODP

0.10.2.3.3.6.4-2 (JAVA 17)

3.1.3.3.3.6.4-2 (JAVA 17)

ad-hive-hook_odp_3.1.3-assembly-X.Y.Z.jar

Note This is supported on ODP 3.1.3 and JDK>=17

ODP

0.10.3,0.10.3

4.0.0,4.0.1

ad-hive-hook_odp_ 3.3.6_java11

Note If you are using Pulse 3.8.x or later with ODP 3.3.6.x.x and Java 11 or later, ensure to use this hook.

Info

X.Y.Z >> For details about supported hook versions on each Pulse version, see Supported Hook Versions.

  • For Pulse 4.0.x, use the hook version 2.0.1.

  • For Pulse 4.1.x, use the hook version 2.1.2

Steps to Place Hive Hook JARs

  1. Get the Hive hook JARs from Acceldata (see mapping above).

  2. Remove the existing Hive/Tez hook JAR from /opt/acceldata/ before placing the new JAR.

  3. Place the JARs on HiveServer2 and Hive interactive nodes in:

/opt/acceldata

  1. Ensure the hook directory is readable and executable by all users.

  2. Add the hook JARs to AUX_CLASSPATH:

Info

Update the hook JAR name in the following properties based on your installed ODP Tez and Hive versions.

  • For HiveServer2, navigate to Hive > Configs > Advanced hive-env, add the following hook jar.

For example:

export AUX_CLASSPATH=/opt/acceldata/<JARNAME>

<JARNAME>: Select the corresponding Hook JAR from the mapping table based on your Hive and Tez versions.

  • For Hive Interactive, navigate to Hive > Configs > Advanced hive-interactive-env, add the following hook jar.

For example:

export AUX_CLASSPATH=/opt/acceldata/<JARNAME>

<JARNAME>: Select the corresponding Hook JAR from the mapping table based on your Hive and Tez versions.

Info

X.Y.Z >> For details about supported hook versions on each Pulse version, see Supported Hook Versions.

  • For Pulse 4.0.x, use the hook version 2.0.1.

  • For Pulse 4.1.x, use the hook version 2.1.2

Configuration

Set the following parameters and values for Hive and Tez:

  1. In the Ambari UI, go to Services > Hive.

  2. On the Hive page, click the Configs tab.

  3. In Configs, select Custom-hive-site and set the following:

    1. ad.events.streaming.servers=(<Pulse IP>:19009)

    2. ad.cluster=(cluster name as specified in Pulse installation)

    3. (Optional: Add this only if the YARN Optimizer is enabled) Click Add Property to add a new property, and add ad.tez.fingerprinting: true

  4. Repeat the same configuration steps for Tez under Services > Tez > Configs > Custom-tez-site in the Ambari UI.

  5. Save the configuration changes.

  6. Restart the affected services to apply the new configuration.

Hive hook

In the Ambari UI:

  1. Navigate to Hive > Configs > Advanced > General.

  2. Ensure io.acceldata.hive.AdHiveHook is added (comma-separated) under:

hive.exec.failure.hooks hive.exec.pre.hooks hive.exec.post.hooks

Tez hook

In the Ambari UI:

  1. Navigate to Tez > Configs > Advanced > General.

  2. Ensure the property tez.history.logging.service.class is configured to io.acceldata.hive.AdTezEventsNatsClient.

Configure ODP Tez for Pulse

This section describes how to configure Tez so that Pulse can collect query events.

Prerequisites

  • Obtain the Hive hook JARs from the Acceldata team.

  • Review the mapping table to identify the correct JAR version for your HDP distribution. For details, see the section Hook JAR Mapping on this page.

Steps for Configuration

  1. Identify the hook JAR location.

For example:

  • For Hive 3.1.4 on ODP:

Use the JAR ad-hive-hook_odp_3.1.4-assembly-X.Y.Z.jar on the HDFS path:

/odp/apps/${odp.version}/tez/tez.tar.gz

For details on the supported hooks for different Hive and Tez versions, see Place Pulse hook JARs and Set Properties (Hive and Tez).

Info

X.Y.Z >> For details about supported hook versions on each Pulse version, see Supported Hook Versions.

  • For Pulse 4.0.x, use the hook version 2.0.1.

  • For Pulse 4.1.x, use the hook version 2.1.2

  1. Update the Tez tarball with the Pulse hook JAR.

Info

During upgrades or hook updates, ensure the Tez tarball contains only the latest Acceldata hook JAR. Remove any older hook files to avoid conflicts.

# Create a directory mkdir -p tez_pack/ && cd tez_pack # Take backup of existing tez tarball in HDFS /tmp hdfs dfs -cp /odp/apps/<cluster_version>/tez/tez.tar.gz /tmp # Download tez tarball from HDFS to local, switch to accesible user hdfs dfs -get /odp/apps/<cluster_version>/tez/tez.tar.gz . # Unpack the tarball tar -zxvf tez.tar.gz # Remove existing Pulse hook jar from tez libs/ (if exists) rm ./lib/<existing_pulse_hook.jar> # Copy Pulse hook jar to tez libs/ cp </location../../pulse_hook.jar> ./lib/ # Package tez tarball tar -cvzf /tmp/tez.tar.gz . # Upload back and provide right permissions and ownership hdfs dfs -put -f /tmp/tez.tar.gz /odp/apps/<cluster_version>/tez/tez.tar.gz hdfs dfs -chown hdfs:hadoop /odp/apps/<cluster_version>/tez/tez.tar.gz hdfs dfs -chmod 755 /odp/apps/<cluster_version>/tez/tez.tar.gz

  1. Update the tez-site properties.

    • In the Ambari UI, navigate to: Tez > Configs > Custom tez-site.

    • Add or update the following properties:

tez.history.logging.service.class=io.acceldata.hive.AdTezEventsNatsClient ad.events.streaming.servers (PULSE_IP:19009) ad.cluster (your cluster name, ex: ad_odp_dev)

  • Optional (Hive 3.x only): The property ad.hdfs.sink is set to true by default. If set to false, Tez will not publish query metadata proto logs to HDFS.

Result

  • View Hive table details in Pulse.

  • HiveServer2 and Metastore expose JMX metrics.

  • Pulse hooks (Hive/Tez) emit execution events to Pulse.

  • Pulse hooks (Hive/Tez) send fingerprinting of applications to Pulse.


Next to read:
Configure ODP Spark 2 and 3

For additional help, contact www.acceldata.force.com OR call our service desk +1 844 9433282

Copyright © 2026

On This Page
Configure ODP Hive and TezConfigure ODP Tez for PulseView Hive Table Details in PulseConfigure HiveServer2 JMXConfigure Hive Metastore JMXPlace Pulse hook JARs and Set Properties (Hive and Tez)PrerequisitesSteps for ConfigurationResultEnable JMX without Security on JMX Remote PortEnable Basic Authentication on JMX Remote Port (Optional)Enable TLS/SSL on JMX Remote Port (Optional)Enable JMX without Security on JMX Remote PortEnable JMX With Basic Authentication (Optional)Enable JMX With TLS/SSL Authentication (Optional)Hook JAR MappingSteps to Place Hive Hook JARsConfigurationHive hookTez hook