Acceldata
 Pulse
Search documentation… ⌘K

Last updated: Sep 27, 2026 17:39 UTC

Disaster Recovery Strategy for MongoDB and VictoriaDB

Prerequisites

  • A minimum of three nodes to enable High Availability are required
  • MongoDB Version 6 is required for this setup

Steps to Deploy Pulse on New Node

  • Set the MongoDB URI:
export MONGO_URI="<paste previous command output here>"
  • Enable MongoDB Encryption:
export MONGO_ENCRYPTED=true
  • Configure Pulse as a Standalone Node:
export PULSE_SA_NODE=true
  • Create necessary directory structure:
mkdir -p /data01/acceldata/
  • Move the Pulse executable to the designated directory:
mv accelo.linux /data01/acceldata/accelo
  • Navigate to the Pulse directory:
cd /data01/acceldata/
  • Initialize Pulse:
./accelo init
  • Source the environment variables:
source /etc/profile.d/ad.sh
  • Reinitialize Pulse:
accelo init
  • Retrieve Pulse information:
accelo info
  • Log in to Pulse Docker:
accelo login docker

Add Host Entries

Add the following entries to the /etc/hosts file of all the MongoDB nodes, including the Pulse server:

<Pulse-Server-IP> ad-db
<Second-Node-IP> ad-db2
<Third-Node-IP> ad-db3

Set New MongoDB Nodes

  • Retrieve the accelo binary and initiate it with the same version tag as the Pulse version on the primary node.
accelo init
  • Login to the Pulse Registry:
docker network create ad-default
  • Create a new Docker network:
docker network create ad-default
  • Manually download the Mongo image:
docker pull 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-database:<PULSE-VERSION>

Configuring the Primary Node (Pulse Server)

  • Edit Configuration File: In the config/docker/ad-core.yaml file, add the following new fields to the ad-db service section:
  • Add the following fields to the environment block:
ENABLE_MONGODB_RS=true
MONGODB_RS_NAME=addb0
IS_MONGODB_PRIMARY=true
  • Add the following fields to the volumes block:
/etc/localtime:/etc/localtime:ro
/root/acceldata/data/db:/data/db
/root/acceldata/config/db/mongokey:/mongokey
Add the following fields to the extra_hosts block:
  • Add the following fields to the extra_hosts block:
ad-db2:<Second-Node-IP>
ad-db3:<Third-Node-IP>
  • Run Secondary Data Replica: Replace <Pulse-Server-IP>, <Third-Node-IP>, and <PULSE-VERSION> with the appropriate values and execute:
docker run -d --name ad-db3_default -p 27017:27017 -e MONGO_DATA_DIR=/data/db -e MONGO_LOG_DIR=/dev/null -e ENABLE_MONGODB_RS=true -e MONGODB_RS_NAME=addb0 -v /data01/acceldata/config/db/mongokey:/mongokey -v /etc/localtime:/etc/localtime:ro -v /data01/acceldata/data/db:/data/db --restart=unless-stopped --net=ad-default --network-alias=ad-db3 --add-host ad-db:<Pulse-Server-IP> --add-host ad-db2:<Second-Node-IP> 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-database:<PULSE-VERSION>
  • Run Third Data Replica: Replace <Pulse-Server-IP>, <Second-Node-IP>, and <PULSE-VERSION> with the appropriate values and execute:
docker run -d --name ad-db3_default -p 27017:27017 -e MONGO_DATA_DIR=/data/db -e MONGO_LOG_DIR=/dev/null -e ENABLE_MONGODB_RS=true -e MONGODB_RS_NAME=addb0 -v /data01/acceldata/config/db/mongokey:/mongokey -v /etc/localtime:/etc/localtime:ro -v /data01/acceldata/data/db:/data/db --restart=unless-stopped --net=ad-default --network-alias=ad-db3 --add-host ad-db:<Pulse-Server-IP> --add-host ad-db2:<Second-Node-IP> 191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-database:<PULSE-VERSION>
  • Login to Primary Node's MongoDB Shell & Add Replica Members:
chown 1000:1000 /data01/acceldata/config/db/mongokey
docker exec -it ad-db_default bash
mongosh mongodb://admin:ACCELROOT_01082018@localhost:27017/admin
db.adminCommand({"replSetReconfig":{_id: "addb0",version: 2,members: [{ _id : 0, host : "ad-db:27017" },{ _id : 1, host : "ad-db2:27017" },{ _id : 2, host : "ad-db3:27017" }]}})
  • Check Replica Status: Run rs.status() to check the status of the replica.
  • Wait for Stable State: Wait for the MongoDB cluster to stabilize.
  • Update Mongodb Encrypted URI: Replace the MongoDB encrypted URI in all YAML files under config/docker/ and acceldata_acceluster.conf file with the provided value.

Unencrypted string for reference: 

mongodb://accel:ACCELUSER_01082018@ad-db:27017,ad-db2:27017,ad-db3:27017/admin?replicaSet=addb0

VictoriaDB Setup for Backup and Restoration

Info

This is specifically aimed at addressing disk failures; hence, the destination path will reside on a distinct mounted disk.

To setup Victoria database for backup and restore, perform the following:

  • Navigate to the Acceldata directory.
  • Fetch the binaries you want to backup and restore using the below command:
wget "https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.97.1/vmutils-linux-amd64-v1.97.1.tar.gz "
  • Extract the contents of the above tar file:
tar -xvf vmutils-linux-amd64-v1.97.1.tar.gz
  • Remove unnecessary binaries, except vmbackup and vmrestore.
  • Execute the shell script:
source /etc/profile.d/ad.sh
  • Create a new directory for vmdbbackup as below:
mkdir $AcceloHome/data/vmdbbackup
chown 1000:1000 $AcceloHome/data/vmdbbackup
su - $(id -un 1000)
  • Open up access to the ad-vmstorage port by modifying the ad-core.yml file as shown below.
ad-vmstorage:
 image: ad-vmstorage
 container_name: ""
 environment:
 - ARGS=--storageDataPath=/storage
 volumes:
 - /etc/localtime:/etc/localtime:ro
 - /data01/acceldata/data/vmdb:/storage
 ulimits: {}
 ports:
 - 19099:8482
 depends_on: []
 opts: {}
 restart: always
 extra_hosts: []
 network_alias: []
  • Restart vmstorage container using:
accelo restart ad-vmstorage
  • Execute the following command. This command initiates an incremental backup process. Incremental backups are conducted when the destination (-dst) parameter refers to an existing backup directory. After the initial backup is completed, subsequent backups will always target this existing backup directory.
vmbackup-prod -storageDataPath=/data01/acceldata/data/vmdb -snapshot.createURL=http://localhost:19099/snapshot/create -dst=fs:///data01/acceldata/data/vmdbbackup
  • Configure the crontab to execute this task once daily, preferably in the morning:
crontab -u user -e 
0 1 * * * /data01/acceldata/vmbackup-prod -storageDataPath=/data01/acceldata/data/vmdb -snapshot.createURL=http://localhost:19099/snapshot/create -dst=fs:///data01/acceldata/data/vmdbbackup

Steps to Restore

To restore from backup, perform the following:

  • Stop the vmstorage container, if it already running, and run the restore command:
docker stop ad-vmstorage_default 
vmrestore-prod -src=fs:///data01/acceldata/data/vmdbbackup -storageDataPath=/data01/acceldata/data/vmdb
  • Run accelo deploy core
  • Check the logs of the container using docker logs -f ad-vmstorage_default

Troubleshooting

  • If the backup process is slow, consider adjusting the value of the -concurrency flag to a higher setting. This will boost the number of simultaneous workers responsible for uploading data to the backup storage.
  • If vmrestore or vmbackup encounters an interruption due to a temporary error, simply restart it using the same arguments. The process will resume from where it left off, continuing the restore or backup operation.