Configure CDH Hive and Sqoop

This page describes how to place Pulse Hive hook JARs, update Hive and Sqoop configurations, and enable metrics collection for CDH clusters.

Configure CDH Hive for Pulse

Hook Version Mapping (CDH Hive)

Distro Version	Hive Version	Tez Version	Pulse Hook Jar Name
CDH 6.2.x	2.1.x	2.1.1-cdh6.2.1	ad-hive-hook__2.1.1__cdh6.2.1-assembly-x.y.z.jar
CDH 6.3.4	2.1.x	2.1.1-cdh6.3.4	ad-hive-hook__cdh__3.0.0-assembly-x.y.z.jar

Info

X.Y.Z >> For details about supported hook versions on each Pulse version, see Supported Hook Versions.

For Pulse 4.0.x, use the hook version 2.0.1
For Pulse 4.1.x, use the hook version 2.1.2

Place Hive Hook JARs

Obtain the Hive hook JARs from the Acceldata team (refer to the table above).
Remove the existing Hive/Tez hook JAR from /opt/acceldata/ before placing the new JAR.
Place the JARs on all HiveServer2 nodes under:

/opt/acceldata

Ensure the hook directory is readable and executable by all users.

Update Hive Environment with Hook JAR

In CM, search for:

Gateway Client Environment Advanced Configuration Snippet (Safety Valve) for hive-env.sh

Add:

AUX_CLASSPATH=${AUX_CLASSPATH}:/opt/acceldata/<AD HIVE 1.x or HIVE 2.x hook jar name>

Update Hive Site Properties

In CM, search for:

Hive Client Advanced Configuration Snippet (Safety Valve) for hive-site.xml
HiveServer2 Advanced Configuration Snippet (Safety Valve) for hive-site.xml

Switch to XML view and add:

<property> <name>hive.exec.failure.hooks</name> <value>io.acceldata.hive.AdHiveHook</value> <description>for Acceldata APM</description> </property> <property> <name>hive.exec.post.hooks</name> <value>io.acceldata.hive.AdHiveHook</value> <description>for Acceldata APM</description> </property> <property> <name>hive.exec.pre.hooks</name> <value>io.acceldata.hive.AdHiveHook</value> <description>for Acceldata APM</description> </property>

Update Pulse Details in Hive Site Properties

In the Advanced hive-site XML section, add the following properties:

ad.events.streaming.servers=(<Pulse IP>:19009) ad.cluster=(cluster name as specified in Pulse installation)

Restart Hive Services

Restart the affected Hive components.
Deploy the updated client configuration.

Configure CDH Sqoop for Pulse

Place the hook JAR in the classpath libraries of the Sqoop client on the designated edge nodes.
Ensure that the Sqoop client can access the Pulse hook JAR, as configured in the Hive steps above.

Result

CDH Hive and Sqoop are configured with Pulse hook JARs and metric sinks. Pulse captures metrics and Hive query statistics from your CDH 5.x/6.x cluster.

Last updated on

Was this page helpful?