Real-Time Data Support Without Compromising System Stability

Starting with 4.1.0, Pulse displays real-time data in the UI. Previously, we intentionally introduced a caching layer to reduce latency and improve performance when querying Elasticsearch, as direct queries could take up to 1–2 minutes and increase load on Elasticsearch, especially for large clients.

Earlier Cache Behavior

The cache was designed with index validation to ensure correctness:

• We fetch data from Elasticsearch only if the current index does not match the alias index.
• If the alias index and cached index match, the system serves data directly from cache.
• This avoids unnecessary Elasticsearch calls and significantly improves performance.

Current Problem

Although the backend now updates Elasticsearch in real time, the UI still displays cached data. This happens because:

• Alias index may remain the same even when the underlying data changes.
• Cache continues serving older data.

Completely removing the cache is not feasible because:

• Queries may again take 1–2 minutes
• Elasticsearch load may increase significantly
• This could impact performance and stability for large clients

Implemented Solution

To support real-time requirements while maintaining system stability, we introduced controlled enhancements:

1. Refresh Button in UI

• A Refresh button has been added.

• When clicked, the system:
  • Clears the existing cache
  • Fetches fresh data from Elasticsearch
  • Updates the cache with the latest data
  • Displays updated data in the UI

2. Conditional Visibility Based on Metadata

The refresh button is shown only when real-time mode is enabled via metadata:

{

  "metadata": {

    "docType": "_metadata",

    "inotifyRealtime": true,

    "lastUpdated": 1771311185471

  }

• If inotifyRealtime is true, the refresh button is displayed.
• If inotifyRealtime is false, the refresh button is hidden and normal cache behavior continues.

Resulting Behavior

• Default behavior continues using cache for optimal performance.
• Elasticsearch is protected from unnecessary load.
• Users can manually fetch real-time data when required.
• Cache remains effective while still supporting real-time updates.

This approach ensures a balance between performance, scalability, and real-time data requirements without impacting overall system stability.

Configurable Refresh and Elasticsearch Timeout Controls

To support real-time data access while maintaining system stability, you can configure refresh and Elasticsearch timeout behavior from the Pulse node.

Update Configuration

1. Navigate to the following file.

$AcceloHome/config/docker/ad-core.yml

2. Edit the ad-pulse-ui section and update the environment variables:

environment:

- REFRESH_COOLDOWN_MS=10000

- ES_REQUEST_TIMEOUT_MS=120000

If the ad-core.yml file does not exist, create it using:

accelo admin makeconfig ad-core

Configuration Parameters

• REFRESH_COOLDOWN_MS (default: 10000 ms)

  • Enforces a cooldown period between consecutive refresh requests from the frontend to prevent rapid repeated calls and reduce unnecessary load on Elasticsearch.

• ES_REQUEST_TIMEOUT_MS (default: 120000 ms)

  • Defines the timeout for Elasticsearch requests. Requests exceeding this duration fail gracefully to avoid long-hanging connections and maintain system stability.

Apply Changes

Restart the service for the updates to take effect:

accelo restart ad-pulse-ui