Deploy Pulse Database

You can deploy Pulse databases on virtual machines (VMs) or bare metal nodes using the Pulse Manager Server. The Pulse Manager Agent, installed on each VM, applies the database deployment plan and sends the status of the databases to the Pulse Manager Server.

Pulse databases store the data collected from Hadoop nodes and processed by Pulse Core and Add-on services. The Pulse UI retrieves this data from the databases and displays it in the user interface.

Deploy Databases on VMs

As part of the Pulse setup, deploy the following databases on Virtual Machines (VMs):

ElasticSearch: ad-elastic and logstash
VictoriaMetrics:
- ad-vmstorage
- ad-vmselect
- ad-vminsert
MongoDB: ad-db
PostgreSQL:
- ad-pg
- ad-pg-ui
Message Queue
- Ad-events

Prerequisites

Ensure the Pulse Manager Agent is already installed on VMs or Bare Metal Nodes. For more information, see Deploy Pulse Manager Agent.
Have the cacerts file is ready for the ElasticSearch deployment. This is needed to enable secure communication between Elasticsearch nodes.

Deployment Steps

To begin the database (DB) deployment, follow these steps:

Request the /init API of the Manager Server, providing the image tag and other necessary information.

curl --request POST \ --url http://<Ingress API for Manager Server>/api/v1/mserver/init \ --header '<basic credentials>' --header 'Content-Type: application/json' \ --data '{ "pulse_version": "<Pulse Version>", "registry_url": "191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata", "hdfs_retention_days": 31, "mongo_retention_days": 32, "logstore_retention": 32, "metrics_retention": 32, "registry_secret": "<registry secret name>" }'

Before running the command, ensure you have updated all placeholder values enclosed in <> with the appropriate information.

The curl command sends a POST request to the init endpoint of the Pulse Manager Server via its Ingress API URL.
The Authorization provides basic authentication to authorize the request.
This JSON payload provides the initial configuration:

Key	Description
`pulse_version`	Specifies the Pulse version to use.
`registry_url`	The URL of the container registry (e.g., AWS ECR) where Pulse images are stored.
`registry_secret`	The Kubernetes secret used to authenticate with the registry.
`hdfs_retention_days`	How long to retain HDFS-related data in the system (in days).
`mongo_retention_days`	How long to retain MongoDB data.
`logstore_retention`	Log storage retention duration (e.g., ElasticSearch).
`metrics_retention`	Retention period for metrics data (e.g., in VictoriaMetrics).

Upload the CACerts for Elastic Deployment.

curl --request PUT \ --url http://<Ingress API for Manager Server>/api/v1/mserver/upload/certs/es \ --header '<basic credentials>' --header 'Content-Type: multipart/form-data' \ --form es-certs.p12=@cacerts

Execute the Deploy DB API call to initiate the database deployment.

curl --request PUT \ --url http://<Ingress API for Manager Server>/api/v1/mserver/deploy/db \ --header '<basic credentials>'

Verify the database status.

Check whether all the database instances are up and running by querying their status using the appropriate commands or APIs.

[root@dbservers101 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES dbe34ae6973e 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-vminsert:3.8.0-260525 "/run.sh" 35 minutes ago Up 35 minutes 8480/tcp, 8489/tcp, 0.0.0.0:19043->8427/tcp, :::19043->8427/tcp ad-vminsert_default 86ec997eab55 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-vmselect:3.8.0-260525 "/run.sh" 35 minutes ago Up 35 minutes 0.0.0.0:19042->8481/tcp, :::19042->8481/tcp ad-vmselect_default e042a5d6aa39 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-vmstorage:3.8.0-260525 "/run.sh" 35 minutes ago Up 35 minutes 8401/tcp, 8482/tcp, 0.0.0.0:19122->8400/tcp, :::19122->8400/tcp ad-vmstorage_default cf3df5c95450 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-pg:3.8.0-260525 "docker-entrypoint.s…" 35 minutes ago Up 35 minutes 0.0.0.0:19067->5432/tcp, :::19067->5432/tcp ad-pg_default c84a007846da 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-gauntlet:3.8.0-260525 "/bin/sh -c /gauntle…" 35 minutes ago Up 35 minutes ad-gauntlet_default 3e2ccb043891 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-events:3.8.0-260525 "nats-server --confi…" 35 minutes ago Up 35 minutes 9222/tcp, 0.0.0.0:19009->4222/tcp, :::19009->4222/tcp, 0.0.0.0:19008->6222/tcp, :::19008->6222/tcp ad-events_default ffd9cc0155a1 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-logstash:3.8.0-260525 "/usr/local/bin/dock…" 35 minutes ago Up 35 minutes 0.0.0.0:19012->5044/tcp, :::19012->5044/tcp, 0.0.0.0:19051->19015/tcp, :::19051->19015/tcp ad-logstash_default 3e1fd10cc0b1 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-pg-ui:3.8.0-beta "docker-entrypoint.s…" 57 minutes ago Up 57 minutes 0.0.0.0:19068->5432/tcp, :::19068->5432/tcp ad-pg-ui_default 85b9a59c90a7 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-elastic:3.8.0-260525 "/bin/tini -- /usr/l…" 35 minutes ago Up 35 minutes 0.0.0.0:19013->9200/tcp, :::19013->9200/tcp, 0.0.0.0:19014->9300/tcp, :::19014->9300/tcp ad-elastic_default ce04af165637 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-database:3.8.0-260525 "/run.sh" 35 minutes ago Up 35 minutes 0.0.0.0:27017->27017/tcp, :::27017->27017/tcp ad-db-router_default 4c060e87412a 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-database:3.8.0-260525 "/run.sh" 35 minutes ago Up 35 minutes 0.0.0.0:27018->27017/tcp, :::27018->27017/tcp ad-db-shard_default e904d1d26224 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-database:3.8.0-260525 "/run.sh" 35 minutes ago Up 35 minutes 0.0.0.0:27019->27017/tcp, :::27019->27017/tcp ad-db-configs_default

Last updated on

Was this page helpful?