Manage Data Planes

After you have successfully installed and deployed a Data Plane, you can monitor, configure, and maintain it from the Data Planes tab in Settings. This page describes the key management tasks available from the ADOC interface.

Note Maintenance Mode (Pause and Resume) is available for Data Plane V3 only from ADOC V26.5.0 onwards. All other capabilities on this page apply to all supported Data Plane versions unless stated otherwise.

Data Plane List

The Data Planes tab lists all registered Data Planes in your ADOC environment. Each row provides a snapshot of the Data Plane's current state and configuration.

Select the three-dot menu on any Data Plane row to access available actions for that Data Plane.

View Data Plane Logs

You can view activity and diagnostic logs for a Data Plane directly from the Data Planes list.

Data Plane Activity Logs

1. In the Data Planes list, locate the Data Plane you want to inspect.
2. Select the three-dot menu for that Data Plane.
3. Select View Logs.

Crawler Logs

Crawler logs are generated per data source by the analysis service on the Data Plane.

4. Navigate to the relevant data source.
5. Select the three-dot menu next to the data source.
6. Select View Crawler Logs to monitor crawler activity in real time.

Note Spark job logs are only available if the Data Plane is deployed with Spark on Kubernetes. Spark jobs running on Databricks, Dataproc, or Hadoop are not supported for log viewing in ADOC.

Maintenance Mode — Pause and Resume

Available in: Data Plane V3 only.

Maintenance Mode lets a data plane administrator pause a Data Plane V3 before an upgrade or maintenance window. When paused, no new executions start. This prevents job failures caused by submitting work to a Data Plane that is mid-upgrade, and removes the need to manually disable and re-enable individual policies before and after maintenance.

Execution Behavior When Paused

The following table describes how each execution type is handled when a Data Plane is paused:

Pausing a Data Plane

7. In the left navigation, select Settings.
8. Select the Data Planes tab.
9. On the Data Plane you want to pause, select the three-dot menu and select Pause Data Plane.
10. Review the confirmation message, then confirm the action.

A success notification confirms that the Data Plane is paused. The Status column updates to Paused. If jobs were in flight at the time of the pause, a badge next to the Paused status shows the current in-flight job count. The count decrements as each in-flight job completes.

Tip Wait until the in-flight job count reaches zero before starting your upgrade or maintenance activity to ensure all running executions have completed cleanly.

Error Behavior During Pause

Any attempt to run a manual execution or profiling job against a paused Data Plane returns an error. The error specifies the Data Plane name and states that execution requests are not accepted while the Data Plane is paused.

Resuming a Data Plane

11. In the Data Planes list, locate the Data Plane in Paused status.
12. Select the three-dot menu and select Resume Data Plane.
13. The Data plane resumed dialog confirms the Data Plane is active and that scheduled and manual runs can proceed per each job's configuration.
14. Review the Summary section:

15. Optionally, select Click here to copy the full JSON for all skipped scheduled executions to your clipboard (see Skipped Executions JSON below).
16. Select Done to close the dialog.

Skipped Executions JSON

When you select Click here on the resume dialog, the skipped executions summary is copied to your clipboard as JSON. Use this to audit or selectively re-trigger work after the maintenance window.

xxxxxxxxxx

{

  "analyticsPipelineId": <id>,

  "status": "ACTIVE",

  "pauseDuration": "<duration>",

  "skippedSchedulesSummary": {

    "policies": [],

    "profiles": [],

    "crawlers": []

  }

}

Data Plane Health Monitor

The Data Plane Health Monitor provides real-time visibility into the operational health of your deployed Data Planes. Select Data Plane Observability from the three-dot menu on any Data Plane to open a live dashboard showing Spark History Server activity (if Spark is enabled), client analysis performance, and job activity and resource usage.

Use this dashboard to monitor Data Plane performance, identify slowdowns or failures quickly, and confirm that resources are being used efficiently.

Note Kubernetes node-level metrics (CPU, memory, storage, OS) are available if the Kubernetes monitoring service is running and you have the required RBAC permissions to view node data.

Data Plane Configuration Utility

The Data Plane Configuration Utility lets you reassign a data source from one Data Plane to another without manual reconfiguration. Use this for disaster recovery, planned maintenance, or moving data sources between primary and standby Data Planes.

How It Works

The utility accepts a JSON input file defining the data source, source and target Data Plane names, and updated connection and observability settings. On execution, the data source is moved to the target Data Plane and its connection settings are updated in a single operation. If any step fails, automatic rollback returns the data source to its previous Data Plane.

Input JSON Structure

xxxxxxxxxx

{

  "datasources": {

    "<dataSourceName>": {

      "dataplane_name_from": "<from_dataplane>",

      "dataplane_name_to": "<to_dataplane>",

      "config": {

        "connectionConfig": {

          "properties": [

            {"key": "jdbc.url",      "value": "<new_jdbc_url>"},

            {"key": "jdbc.user",     "value": "<username>"},

            {"key": "jdbc.password", "value": "<password>"}

          ]

        },

        "dataObservabilityConfig": {

          "properties": [

            {"key": "data.freshness.monitoring", "value": true},

            {"key": "schema.drift.monitoring",   "value": true}

          ]

        }

      }

    }

  }

}

Tip Validate your JSON input file before running the utility to avoid runtime errors.

Running the Utility

Execute the switch script with the following command:

python datasource_dataplane_switch.py <url> <access_key> <secret_key> <input.json path> <continue_on_fail> <connection_check>

Example

xxxxxxxxxx

python datasource_dataplane_switch.py https://example.url.com/ ABCD1234 XYZ7890 /path/to/input.json true true

Logs and Rollback

The utility outputs success or failure messages per data source as it runs. If a switch fails, an automatic rollback is triggered and the data source remains associated with its previous Data Plane.

Example: Azure MSSQL Configuration

xxxxxxxxxx

"connectionConfig": {

  "properties": [

    {"key": "mssql.url",                 "value": ""},

    {"key": "mssql.user",                "value": ""},

    {"key": "mssql.use.msi",             "value": false},

    {"key": "use.service.principals",    "value": false},

    {"key": "enable.secret.manager",     "value": true},

    {"key": "secret.configuration.name", "value": "SECRETS_MANAGER"},

    {"key": "secret.key",                "value": "secretKey"}

  ]

},

"dataObservabilityConfig": {

  "properties": [

    {"key": "mssql.db.0",              "value": "dbName"},

    {"key": "query.analysis.service",  "value": false},

    {"key": "timeZone",                "value": "UTC"},

    {"key": "schema.drift.monitoring", "value": false},

    {"key": "slots.enabled",           "value": false}

  ]

}

The same config structure applies to Kafka, HDFS, MongoDB, BigQuery, Hive, and GCS. Update the key names to match the connector-specific properties for each data source type.

Delete a Data Plane

Deleting a Data Plane in ADOC removes it from the Control Plane only. The underlying cloud infrastructure must be cleaned up separately in your cloud provider.

17. In the left navigation, select Settings, then select the Data Planes tab.
18. Locate the Data Plane you want to delete and select the delete icon next to it.
19. In the confirmation dialog, select Confirm to complete the deletion.

Important After deleting a Data Plane in ADOC, you must manually remove the associated cloud resources to complete cleanup. Leaving cloud resources in place will continue to incur costs.

AWS

• Delete the Load Balancer.
• Delete the CloudFormation Stack.
• Remove the Data Plane YAML and any related files from your S3 bucket.

Azure

• Delete resources from the Resource Group.
• Remove the Data Plane YAML and related files from the Azure Storage container.