Deploy Pulse Database
You can deploy Pulse databases on virtual machines (VMs) or bare metal nodes using the Pulse Manager Server. The Pulse Manager Agent, installed on each VM, applies the database deployment plan and sends the status of the databases to the Pulse Manager Server.
Pulse databases store the data collected from Hadoop nodes and processed by Pulse Core and Add-on services. The Pulse UI retrieves this data from the databases and displays it in the user interface.
Deploy Databases on VMs
As part of the Pulse setup, deploy the following databases on Virtual Machines (VMs):
ElasticSearch:
ad-elasticandlogstashVictoriaMetrics:
ad-vmstoragead-vmselectad-vminsert
MongoDB:
ad-dbPostgreSQL:
ad-pgad-pg-ui
Message Queue
Ad-events
Prerequisites
- Ensure the Pulse Manager Agent is already installed on VMs or Bare Metal Nodes. For more information, see Deploy Pulse Manager Agent.
- Have the
cacertsfile ready for the ElasticSearch deployment. This is needed to enable secure communication between ElasticSearch nodes.
Deployment Steps
To begin the database (DB) deployment, follow these steps:
- Request the
/initAPI of the Manager Server, providing the image tag and other necessary information.
curl --request POST \ --url http://<Ingress API for Manager Server>/api/v1/mserver/init \ --header '<basic credentials>' --header 'Content-Type: application/json' \ --data '{ "pulse_version": "<Pulse Version>", "registry_url": "191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata", "hdfs_retention_days": 31, "mongo_retention_days": 32, "logstore_retention": 32, "metrics_retention": 32, "registry_secret": "<registry secret name>"}'Before running the command, ensure you have updated all placeholder values enclosed in <> with the appropriate information.
- The curl command sends a POST request to the
initendpoint of the Pulse Manager Server via its Ingress API URL. - The Authorization provides basic authentication to authorize the request.
- This JSON payload provides the initial configuration:
| Key | Description |
|---|---|
pulse_version | Specifies the Pulse version to use. |
registry_url | The URL of the container registry (e.g., AWS ECR) where Pulse images are stored. |
registry_secret | The Kubernetes secret used to authenticate with the registry. |
hdfs_retention_days | How long to retain HDFS-related data in the system (in days). |
mongo_retention_days | How long to retain MongoDB data. |
logstore_retention | Log storage retention duration (e.g., ElasticSearch). |
metrics_retention | Retention period for metrics data (e.g., in VictoriaMetrics). |
- Upload the CACerts for Elastic Deployment.
curl --request PUT \ --url http://<Ingress API for Manager Server>/api/v1/mserver/upload/certs/es \ --header '<basic credentials>' --header 'Content-Type: multipart/form-data' \ --form es-certs.p12=@cacerts- Execute the Deploy DB API call to initiate the database deployment.
curl --request PUT \ --url http://<Ingress API for Manager Server>/api/v1/mserver/deploy/db \ --header '<basic credentials>'- Verify the database status.
Check whether all the database instances are up and running by querying their status using the appropriate commands or APIs.
x
[root@dbservers101 ~]# docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESdbe34ae6973e 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-vminsert:3.8.0-260525 "/run.sh" 35 minutes ago Up 35 minutes 8480/tcp, 8489/tcp, 0.0.0.0:19043->8427/tcp, :::19043->8427/tcp ad-vminsert_default86ec997eab55 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-vmselect:3.8.0-260525 "/run.sh" 35 minutes ago Up 35 minutes 0.0.0.0:19042->8481/tcp, :::19042->8481/tcp ad-vmselect_defaulte042a5d6aa39 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-vmstorage:3.8.0-260525 "/run.sh" 35 minutes ago Up 35 minutes 8401/tcp, 8482/tcp, 0.0.0.0:19122->8400/tcp, :::19122->8400/tcp ad-vmstorage_defaultcf3df5c95450 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-pg:3.8.0-260525 "docker-entrypoint.s…" 35 minutes ago Up 35 minutes 0.0.0.0:19067->5432/tcp, :::19067->5432/tcp ad-pg_defaultc84a007846da 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-gauntlet:3.8.0-260525 "/bin/sh -c /gauntle…" 35 minutes ago Up 35 minutes ad-gauntlet_default3e2ccb043891 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-events:3.8.0-260525 "nats-server --confi…" 35 minutes ago Up 35 minutes 9222/tcp, 0.0.0.0:19009->4222/tcp, :::19009->4222/tcp, 0.0.0.0:19008->6222/tcp, :::19008->6222/tcp ad-events_defaultffd9cc0155a1 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-logstash:3.8.0-260525 "/usr/local/bin/dock…" 35 minutes ago Up 35 minutes 0.0.0.0:19012->5044/tcp, :::19012->5044/tcp, 0.0.0.0:19051->19015/tcp, :::19051->19015/tcp ad-logstash_default3e1fd10cc0b1 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-pg-ui:3.8.0-beta "docker-entrypoint.s…" 57 minutes ago Up 57 minutes 0.0.0.0:19068->5432/tcp, :::19068->5432/tcp ad-pg-ui_default85b9a59c90a7 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-elastic:3.8.0-260525 "/bin/tini -- /usr/l…" 35 minutes ago Up 35 minutes 0.0.0.0:19013->9200/tcp, :::19013->9200/tcp, 0.0.0.0:19014->9300/tcp, :::19014->9300/tcp ad-elastic_defaultce04af165637 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-database:3.8.0-260525 "/run.sh" 35 minutes ago Up 35 minutes 0.0.0.0:27017->27017/tcp, :::27017->27017/tcp ad-db-router_default4c060e87412a 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-database:3.8.0-260525 "/run.sh" 35 minutes ago Up 35 minutes 0.0.0.0:27018->27017/tcp, :::27018->27017/tcp ad-db-shard_defaulte904d1d26224 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-database:3.8.0-260525 "/run.sh" 35 minutes ago Up 35 minutes 0.0.0.0:27019->27017/tcp, :::27019->27017/tcp ad-db-configs_defaultWas this page helpful?