Configure HDFS Small File Threshold

Pulse identifies an HDFS file as a small file with a size threshold. You can configure the threshold via setting the variable hdfs.analytics.file.size = xxxxxx (in Bytes) in the cluster configuration file to set the small file size threshold. Files meeting this threshold or smaller are identified as small files and displayed on the HDFS Analytics, HDFS Explorer, etc., pages.

Steps to configure the threshold:

1. Open the _$AcceloHome/config/acceldata_<clustername>.conf_ file.
2. Add the following variable and configure the file size in Bytes. By default, the small file value is set to 1 MB.

xxxxxxxxxx

common.configs = {

      hdfs.analytics.file.size = 1048576 (example, in Bytes) # update this value

    }

3. Save and close the file.
4. Run the following command to push the changes. To reflect the changes on the FS Analytics page, run the FSImage command to reprocess the reports.

xxxxxxxxxx

accelo admin database push-config

accelo admin fsa process

accelo admin fsa reports