Retain Custom Changes in FS Scripts After reconfig cluster

Starting from ODP 3.3.6, when WebHDFS is enabled, disabling SPNEGO authentication is no longer supported. This can cause FSImage loading failures in FSAnalytics because the FSImage fetch request may fail when accessing the NameNode.

A practical workaround is to modify the FSImage fetch request URL to include the parameter:

&user.name=hdfs

Why This Is Needed

When WebHDFS is enabled and SPNEGO cannot be disabled, the default FSImage fetch request may fail due to authentication restrictions. By appending user.name=hdfs to the request URL, the NameNode can process the request successfully under simple authentication environments.

This enables FSImage retrieval without requiring customers to disable WebHDFS, ensuring FSAnalytics continues to function correctly.

Steps

1. Navigate to the FSAnalytics script directory

$ACCELDATA_HOME/work/<clusterName>/fsanalytics

2. Back up the existing script

cp update_fsimage.sh update_fsimage.sh.original

3. Modify the FSImage fetch logic

Replace the existing block in update_fsimage.sh.

Existing block

# Trigger the fsimage build command

if [[ $IsKerberosEnabled == 1 ]]

then

  # Kerberos is enabled

  /usr/bin/gurl -X GET -u "hdfs:" -k -kt /krb/security/kerberos.keytab -kp  -o /etc/fsanalytics/$1/fsimage -l "http://odp.jags.latest.com:50070/imagetransfer?getimage=1&txid=latest"

else

  /usr/bin/gurl -X GET -o /etc/fsanalytics/$1/fsimage -l "http://odp.jags.latest.com:50070/imagetransfer?getimage=1&txid=latest"

fi

Updated block

# Trigger the fsimage build command

/usr/bin/gurl -X GET -o /etc/fsanalytics/$1/fsimage -l "http://odp.jags.latest.com:50070/imagetransfer?getimage=1&txid=latest&user.name=hdfs"

This modification appends the user.name=hdfs parameter to the request URI.

4. Verify the changes inside the FSAnalytics container

docker exec -it <containerName> bash

Check the updated script:

less /opt/docker/work/<clusterName>/fsanalytics/update_fsimage

Exit the container:

exit

5. Reload the Image

Run the following command to reload the FSImage:

accelo admin fsa load

If required, specify the FSAnalytics endpoint:

accelo admin fsa load --uri=http://<PulseHostIP>:<FsContainerPort>

Result

After applying this change, FSAnalytics can successfully fetch the FSImage regardless of whether WebHDFS is enabled or disabled, ensuring stable FSAnalytics operation in ODP 3.3.6 and later environments.