Configure HDFS Small File Threshold
Pulse identifies an HDFS file as a small file with a size threshold. You can configure the threshold via setting the variable hdfs.analytics.file.size = xxxxxx
(in Bytes) in the cluster configuration file to set the small file size threshold. Files meeting this threshold or smaller are identified as small files and displayed on the HDFS Analytics, HDFS Explorer, etc., pages.
Steps to configure the threshold:
- Open the
_$AcceloHome/config/acceldata_<clustername>.conf_
file. - Add the following variable and configure the file size in Bytes. By default, the small file value is set to 1 MB.
x
common.configs = {
hdfs.analytics.file.size = 1048576 (example, in Bytes) # update this value
}
The configured file size is pushed as a real-time metric under hdfs_root_analysis
in the database.
- Save and close the file.
- Run the following command to push the changes. To reflect the changes on the FS Analytics page, run the FSImage command to reprocess the reports.
accelo admin database push-config
accelo admin fsa process
accelo admin fsa reports
Was this page helpful?