Acceldata
Pulse

Last updated: Sep 27, 2026 00:48 UTC

Update Real-time Monitoring Parameters of HDFS

This page describes how to enable and configure real-time monitoring of HDFS in Pulse.


Step 1: Edit Configuration File

On the Pulse node, navigate to the following path and edit the configuration file:

$AcceloHome/config/acceldata_<clustername>.conf


Step 2: Configure Real-Time Monitoring

By default, the following parameter is enabled:

fsanalytics.inotify.enabled=true

This setting enables real-time monitoring of HDFS using the inotify mechanism.

You may update other related parameters in the configuration file as needed, based on your environment and monitoring requirements.

{
  name = "fsanalytics-connector"
  trigger = "scheduled"
  fsanalytics.off.heap.storage = "/etc/fsanalytics"
  fsanalytics.metastore {
     # HIVE METASTORE is ENABLED during the setup
    enabled = true
    encrypted = true
    DAYS = 7
  }
  hive.schema.prefix = "hive."
  # Inotify configuration
  fsanalytics.inotify.enabled = true   # Enable/Disable Inotify feature
  fsanalytics.inotify.poll.interval.ms = 1000  # Time between consecutive polls to HDFS Inotify
  fsanalytics.inotify.poll.timeout.ms = 1000   # Sets poll timeout
  fsanalytics.inotify.es.flush.interval.ms = 1
  fsanalytics.inotify.es.batch.size = 100
  fsanalytics.inotify.es.buffer.capacity = 10000
  slick-mysql {
    profile = "slick.jdbc.MySQLProfile$"
    db {
      dataSourceClass = "slick.jdbc.DriverDataSource"
      properties = {
        driver = "org.mariadb.jdbc.Driver"
        # Check the server timezone
        # Check & validate the metastore DB Name
        url = "jdbc:mysql://odp311.acceldata.dvl:3306/hive?serverTimezone=UTC&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&useUnicode=true"
        user = "hive"
        password = "ColrOFzIvF+y0RK0807jpg=="
      }
    }
  }
}

Info

Limitation: Snapshot Processing Snapshot functionality will NOT be available in the real-time system as HDFS inotify does not provide snapshot-related events & nodeId. The system focuses exclusively on regular file system operations (fsimage processing).

Limitation: Non-HDFS Operations Permission changes or file system operations performed outside standard HDFS APIs (i.e., not executed via HDFS commands or NameNode-backed interfaces) will not generate inotify events and will not be captured in real time. Such changes will only be reflected during subsequent fsimage processing.