FSImage Loading Takes More Time

Issue

FSImage processing in Pulse takes more time (for example, more than 10 hours) to complete for large FSImages (for example, ~40 GB).

Resolution

Update FS Analytics and Elasticsearch configuration

Apply the following configuration changes to improve the performance of FSImage parsing and Elasticsearch ingestion.

1. Step 1: Verify configuration files

On the Pulse node, check whether the following files exist at:

• ad-fsanalyticsv2-connector.yml

• ad-fs-elastic.yml

2. Step 2: Generate configuration files (if they do not already exist)

If the files do not exist, generate them using the following commands:

3. Update environment variables

Open the configuration files and locate the environment section in each file.

• FS Analytics (ad-fsanalyticsv2-connector.yml)

Add the following entries:

• Elasticsearch (ad-fs-elastic.yml)

Add the following entry:

Result

These settings significantly improve FSImage parsing speed and Elasticsearch ingestion throughput, reducing overall FSImage processing time.