Troubleshooting ODP

This section helps you with solving and configuring common issues related to component installation in ODP.

Configure /tmp

If /tmp is required to be mounted as exec but compliance issues necessitate mounting it as noexec, follow the steps below on every node to create an alternative tmp directory:

  1. Create an alternative tmp directory:

sudo mkdir -p /hadoop/tmp sudo chown root:hadoop /hadoop/tmp sudo chmod 1770 /hadoop/tmp
  1. Update Ambari configuration for respective services:

  • For HBase, YARN-HBase (Timeline Reader), and Ambari Metrics (AMS): In Ambari, update the configurations YARN yarn-hbase-env, AMS ams-hbase-env, and HBase hbase-env by adding:

export HBASE_OPTS="$HBASE_OPTS -Dorg.apache.hbase.thirdparty.io.netty.native.workdir=/hadoop/tmp"
  • For Knox: Directly update gateway.sh and knoxcli.sh in the file system on all Knox hosts by appending:

-Djava.io.tmpdir=/hadoop/tmp -D*jna*.tmpdir=/hadoop/tmp
  • For HDFS/YARN/MR: Override HADOOP_OPTS and append:

-Djava.io.tmpdir=/hadoop/tmp

The above adjustments ensure compatibility and functionality despite the noexec mount constraint.

Impala

  1. Update/add these configurations respectively for smooth impala installation

Pre-installation > Update in impala-env, if not set :

is_coordinator = true

is_executor = true

Post-installation > Add custom properties in hadoop core-site :

hadoop.proxyuser.impala.groups=*

hadoop.proxyuser.impala.hosts=*

  1. Impala service failed to register ranger authorization :

E0524 21:09:47.710661 104717 catalog.cc:87] InternalException: Unable to instantiate authorization provider: org.apache.impala.authorization.ranger.RangerAuthorizationFactory CAUSED BY: InvocationTargetException: null CAUSED BY: IllegalArgumentException: bound must be positive

Ensure ranger is enabled in impala configs. If issue issue still persists, current workaround requires handlers to manage and update hive clients and ranger policies.

  • Created dir /etc/ranger/<cluster_name>_hive/policycache and copied respective files from HiveServer2 to impala components

  • Copied files /etc/hive/3.2.2.0-2/0/ranger-hive-audit.xml, ranger-hive-security.xml from HS2 to impala components

Ranger

  1. If KMS ranger policy fails to create. Perform the following steps:

  • Remove the empty KMS policy folder from CLI

  • Restart Ranger from Ambari UI

  • Then, Restart Ranger KMS from Ambari UI 2. If service repo creation fails with missing rangerlookup user, create rangerlookup user manually from Ranger UI.

Spark2

Note

During the installation of Spark2, if the user interface displays "installing Livy3" instead of "Livy2," it is a typographical error on the UI front and can be disregarded.


  Last updated