Manage Pulse Cluster using APIs

This page describes API details for managing and scaling Pulse services and components. It includes operations such as restarting services, managing shards, indexing databases, and processing FSImage data.

Restart Pulse Services

This API endpoint restarts the specified Pulse services without applying any configuration changes. It restarts only the services that are already deployed and uses the configuration defined in the annotations of the Custom Resource (CR).

Accepted Inputs: A list of services that can be restarted.

Core Connectors
SparkStats
Kafka Connector
FS Analytics v2
Ozone Connector
Oozie Connector
Impala Connector
Druid Analyzer
LDAP
SQL Analyser
Trino Connector
Streaming
Hydra
Dashplot
Dashplot-UI
Notifications
Alerts
Pulse-UI
Actions

{ "services": [ "PULSE UI" ]

curl --request PUT \ --url <Ingress API for the manager Server>/api/v1/mserver/restart/<clustername> --header '<basic credentials>' --header 'Content-Type: application/json' \ --data '{ "services": [ "PULSE UI" ] }'

Remove Pulse Add-On Services

This API endpoint removes specified add-ons from the Pulse cluster. It deletes the corresponding pods, deployments, or stateful sets, and clears the related annotations for the selected service.

Accepted Inputs: A list of add-on services can be removed.

SparkStats
Kafka Connector
FS Analytics v2
Ozone Connector
Oozie Connector
Impala Connector
Druid Analyzer
LDAP
SQL Analyser
Trino Connector

{ "addons": [ "OZONE CONNECTOR", "SQL ANALYSER", "CORE CONNECTORS", "SPARKSTATS", "FS ANALYTICS V2", "DRUID ANALYZER" ] }

curl --request PUT \ --url <Ingress API for the manager Server>/api/v1/mserver/remove/addons/<clustername> \ --header '<basic credentials>' --header 'Content-Type: application/json' \ --data '{ "addons": [ "OZONE CONNECTOR", "SQL ANALYSER", "CORE CONNECTORS", "SPARKSTATS", "FS ANALYTICS V2", "DRUID ANALYZER" ] }'

Refresh Core

This API endpoint refreshes the configuration of core services and restarts all of them.

Affected Services:

GraphQL
Streaming
Hydra
Dashplot
Dashplot-UI
Notifications
Alerts
Pulse-UI
Actions

{ "extra_hosts": { "<IP Address 1>": "<host name 1>", "<IP Address 2>": "<host name 2>", "<IP Address 2>": "<host name 3>", "<IP Address 3>": "<host name 4>", } }

curl --request PUT \ --url <Ingress API for the manager Server>/api/v1/mserver/refresh/core/ \ --header '<basic credentials>' --header 'Content-Type: application/json' \ --data '{ "extra_hosts": { "<IP Address 1>": "<host name 1>", "<IP Address 2>": "<host name 2>", "<IP Address 2>": "<host name 3>", "<IP Address 3>": "<host name 4>", } }'

Scale Factor

This API endpoint updates the scale factor for scalability-related services.

Affected services:

Core Connectors
Alerts
Streaming
SparkStats
Impala Connector

Input:

{ "scale_factor": 2 }

curl --request PUT \ --url <Ingress API for the manager Server>/api/v1/mserver/configsf/cluster/<clustername> \ --header '<basic credentials>' --header 'Content-Type: application/json' \ --data '{ "scale_factor": 2 }'

NATS Partition Index

This API Endpoint updates the partition index for the microservices. The microservices those will be affected by are the following ones.

Core Connectors
Alerts
Streaming
Spark
Stats
Impala Connector

Input:

{ "partition_idx": 1 }

curl --request PUT \ --url http://<Ingress API for the Manager Server>/manager/api/v1/mserver/configpidx/cluster/<clustername> \ -u "pulse:<basic credentials>" \ --header 'Content-Type: application/json' \ --data '{ "partition_idx": 1 }'

For Alerts and Streaming, the NATS partition count is automatically set to the maximum NATS partition count configured across all available clusters.

Info

After updating the NATS partition count using the API described above, the affected services must be restarted for the changes to take effect.

Shard Count

This API endpoint updates the shard count for scalability-related services.

Affected services:

Core Connectors
Alerts
Streaming
SparkStats
Impala Connector

Input:

{ "shard_count": 12 }

curl --request PUT \ --url <Ingress API for the manager Server>/api/v1/mserver/configsc/cluster/odp_smokey \ --header '<basic credentials>' --header 'Content-Type: application/json' \ --data '{ "shard_count": 12 }'

Index Databases

Use the following command to index the databases.

curl --request PUT \ --url http://<Ingress API for the Manager Server>/api/v1/mserver/init/indexdb \ --header '<basic credentials>' --header 'Content-Type: application/json'

Reload Events

This operation reloads event configurations for Pulse services without requiring a full restart.

curl --request PUT \ --url http://<Ingress API for the Manager Server>/api/v1/mserver/init/reloadevents \ --header '<basic credentials>' --header 'Content-Type: application/json'

Init Sharding

This command initializes sharding for the supported Pulse services. It should be used during the initial setup or when resetting shard configurations.

curl --request PUT \ --url http://<Ingress API for the Manager Server>/api/v1/mserver/init/sharding \ --header '<basic credentials>' --header 'Content-Type: application/json'

FSImage Load Command

This command loads the FSImage data into the system.

curl --request PUT \ --url http://<Ingress API for the Manager Server>/api/v1/mserver/fsa/load/<clustername> \ --header 'Basic Credentials' \ --header 'Content-Type: application/json'

FSImage Process Command

This command processes the loaded FSImage data for analysis.

curl --request PUT \ --url http://<Ingress API for the Manager Server>/api/v1/mserver/fsa/process/<clustername> \ --header 'Basic Credentials' \ --header 'Content-Type: application/json'

FSImage Reports Command

This command generates reports based on the processed FSImage data.

curl --request PUT \ --url http://<Ingress API for the Manager Server>/api/v1/mserver/fsa/reports/<clustername> \ --header 'Basic Credentials' \ --header 'Content-Type: application/json'

Dyanamic NATS Allocation

Use this API to configure the environment variables required to enable dynamic NATS allocation for backend services.

curl --location --request PUT 'http://<Ingress API for the Manager Server>/api/v1/mserver/configdna' \ -u "pulse:<secret>" \ --header 'Content-Type: application/json' \ --data '{ "dynamic_nats_allocations" : true }'

Events Health Check

List Streams

Returns the details of a specific stream. If streamName is left blank, the API returns all available streams.

curl --request GET \ --url http://<Ingress API for the Manager Server>/api/v1/mserver/events/streams \ -u "pulse:<basic credentials>" \ --header 'Content-Type: application/json' \ --data '{ "streamName": "" }'

View Streams

Returns the complete message stored at the specified offset within a stream.

curl --request GET \ --url http://<Ingress API for the Manager Server>/api/v1/mserver/events/streams/view \ -u "pulse:<basic credentials>" \ --header 'Content-Type: application/json' \ --data '{ "streamName": <streamName>, "offset": <offset_number> }'

Delete Stream

Deletes the specified stream.

curl --request DELETE \ --url http://<Ingress API for the Manager Server>/api/v1/mserver/events/streams/delete \ -u "pulse:<basic credentials>" \ --header 'Content-Type: application/json' \ --data '{ "streamName": <streamName> }'

List Consumers of a Stream

Returns all consumers associated with the specified stream.

curl --request GET \ --url http://<Ingress API for the Manager Server>/api/v1/mserver/events/streams/consumer \ -u "pulse:<basic credentials>" \ --header 'Content-Type: application/json' \ --data '{ "streamName": <streamName> }'

View Consumer Info

Returns detailed information about the specified consumer in a stream.

curl --request GET \ --url http://<Ingress API for the Manager Server>/api/v1/mserver/events/streams/consumer/info \ -u "pulse:<basic credentials>" \ --header 'Content-Type: application/json' \ --data '{ "streamName": <streamName>, "consumerName": <consumerName> }'

Delete Consumer

Deletes the specified consumer from a stream.

curl --request DELETE \ --url http://<Ingress API for the Manager Server>/api/v1/mserver/events/streams/consumer/delete \ -u "pulse:<basic credentials>" \ --header 'Content-Type: application/json' \ --data '{ "streamName": <streamName>, "consumerName": <consumerName> }'

Update Stream Config

Updates the configuration of the specified stream, including the maximum number of messages it can retain.

curl --request PUT \ --url http://<Ingress API for the Manager Server>/api/v1/mserver/events/streams/update \ -u "<basic credentials>" \ --header 'Content-Type: application/json' \ --data '{ "streamName": <streamName>, "maxMsg": 100000 }'

Encrypt Text API

Encrypts a plaintext value and returns an encrypted string that can be used for sensitive notification credentials in Alert configurations.

curl --request POST \ --url http://<Ingress API for the Manager Server>/api/v1/mserver/admin/encrypt \ -u "<basic credentials>" \ --header 'Content-Type: application/json' \ --data '{ "encrypt": "<PLAINTEXT>" }'

Parameter

encrypt - The plaintext value to be encrypted.

Use the encrypted output when configuring sensitive notification credentials such as:

jira.jiraToken
bigpanda.appKey
bigpanda.authToken
quantum.subscriptionKey
xmatters.userGroups[].userKey

Info

The encrypted value returned by this API must be used in the Alert Notification configuration. Providing these credentials in plain text may result in notification delivery failures.

Last updated on

Was this page helpful?