YARN Data Retrieval

Acceldata Pulse APIs enable you to programmatically retrieve YARN and application-level data for use in custom dashboards, reports, data pipelines, and operational workflows.

You can use these APIs to:

  • Authenticate with Pulse.

  • Retrieve Application Explorer data.

  • Retrieve YARN queue and capacity data.

  • Monitor resource allocation and cluster utilization.

  • Integrate Pulse data with external systems.

Pulse exposes these capabilities through GraphQL operations available at the /graphql endpoint.


Before You Begin

Before invoking the APIs:

  • Obtain a valid JWT token and role value.

  • Identify the Pulse URL for your environment.

  • Identify the monitor group when required by the GraphQL operation.

  • Ensure that the authenticated user has access to YARN data.

For authentication instructions, see Authenticate with Pulse APIs


Call the API Endpoint

All API requests go through the GraphQL endpoint:

POST https://<your-pulse-url>/graphql

Example:

https://trial.acceldata.dev/graphql

Retrieve Application Explorer Data

What You Can Get

You can fetch the same data you see in the Application Explorer UI, including:

  • Application ID

  • Type

  • State

  • Name

  • User

  • Queue

  • Progress

  • Start Time

  • Elapsed Time

  • Finished Time

  • Priority

  • Cluster Usage (%)

  • Queue Usage (%)

  • Application Tags

  • Allocated Memory (MB)

  • Allocated vCores

  • Memory Seconds

  • vCore Seconds


Use This GraphQL Operation

YarnAppDataForExplorer

Example API Request

curl 'https://<your-pulse-url>/graphql' \ -H 'content-type: application/json' \ -H 'Role: <role-name>' \ -b 'jwt=<your-jwt-token>' \ --data-raw '{ "operationName": "YarnAppDataForExplorer", "variables": { "tableMetadata": { "pageNo": 1, "pageSize": 20, "start": "<start_timestamp>", "end": "<end_timestamp>" } }, "query": "query YarnAppDataForExplorer($tableMetadata: TableMetadata!) { yarnAppDataForExplorer(tableMetadata: $tableMetadata) { id name queue user state applicationType progress startedTime finishedTime elapsedTime priority clusterUsagePercentage queueUsagePercentage applicationTags allocatedMB allocatedVCores memorySeconds vcoreSeconds } }" }'

Replace:

  • <your-jwt-token> with the JWT token obtained during authentication.

  • <role-name> with a role assigned to the authenticated user (for example, ADMIN, SUPER_ADMIN, or another custom role).


Example Response

{ "data": { "yarnAppDataForExplorer": [ { "id": "application_123456", "name": "Spark Job Example", "queue": "default", "user": "hadoop_user", "state": "FINISHED", "applicationType": "SPARK", "progress": 100, "startedTime": 1774809183525, "finishedTime": 1774812783525, "elapsedTime": 3600000, "priority": 1, "clusterUsagePercentage": 12.5, "queueUsagePercentage": 45.2, "applicationTags": ["etl", "daily"], "allocatedMB": 4096, "allocatedVCores": 4, "memorySeconds": 123456789, "vcoreSeconds": 98765432 } ] } }

Explore YARN Data

What You Can Access

You can also retrieve YARN-related insights such as:

Queue List

  • All queues and hierarchy

  • Capacity allocation

Example Request:

curl -sS -X POST '<Pulse_UI_URL>/graphql' \ -H 'Content-Type: application/json' \ -H 'MonitorGroup: name_here'\ -H 'role: encoded role here' \ -H 'Cookie: jwt=PASTE_YOUR_FULL_JWT_COOKIE_HERE' \ -d '{ "operationName": "YarnQueueSnapshotDistinctQueue", "variables": { "start": "1775544117899", "end": "1775630517899" }, "query": "query YarnQueueSnapshotDistinctQueue($start: String, $end: String) {\n data: yarnQueueSnapshotDistinctQueue(start: $start, end: $end)\n}\n" }'

Example Response:

{"data":{"data":["root","root.default"]}}

Queue Details

  • Capacity and usage

Example Request:

BASE='http://10.100.10.18:4000' curl -sS -X POST "$BASE/graphql" \ -H 'Content-Type: application/json' \ -H 'MonitorGroup: odp_usopp' \ -H 'role: eyJuYW1lIjoiYWRtaW4iLCJjbHVzdGVycyI6W119' \ -H 'Cookie: jwt=PASTE_YOUR_FULL_SIGNED_JWT_COOKIE_VALUE' \ -d '{ "operationName": "YarnQueueSnapshotTimeHistogram", "query": "query YarnQueueSnapshotTimeHistogram($qNames: [String]!, $start: String!, $end: String!, $metrics: [String]!, $accumulators: [String]!, $nodeLabel: String!, $step: Int) { yarnQueueSnapshotTimeHistogram(qNames: $qNames, start: $start, end: $end, metrics: $metrics, accumulators: $accumulators, nodeLabel: $nodeLabel, step: $step) { _id vcoresUsed_avg memoryUsed_avg usedCapacity_avg absoluteUsedCapacity_avg } }", "variables": { "qNames": ["root.default"], "start": "1775544117899", "end": "1775630517899", "metrics": ["vcoresUsed", "memoryUsed", "usedCapacity"], "accumulators": ["avg", "max"], "nodeLabel": "", "step": 1800 } }'

Example Response:

{ "data": { "yarnQueueSnapshotTimeHistogram": [ { "_id": "1775543400000", "vcoresUsed_avg": "1", "memoryUsed_avg": "1024", "usedCapacity_avg": "3.1746032", "absoluteUsedCapacity_avg": null }, { "_id": "1775545200000", "vcoresUsed_avg": "1", "memoryUsed_avg": "1024", "usedCapacity_avg": "3.1746032", "absoluteUsedCapacity_avg": null }, { "_id": "1775547000000", "vcoresUsed_avg": "1", "memoryUsed_avg": "1024", "usedCapacity_avg": "3.1746032", "absoluteUsedCapacity_avg": null }, ], }, }
  • Running and pending applications

  • Resource allocation

YARN Summary

  • Total cluster resources

  • Used vs available memory and vCores

  • Node and application counts


How You Discover These APIs

YARN APIs are powered by GraphQL queries used internally by the UI. To access them:

  1. Open Pulse in your browser

  2. Go to the YARN tab

  3. Open Developer Tools → Network tab

  4. Filter for GraphQL requests

  5. Click on relevant requests (queues, summary, etc.)

  6. Copy:

    • Query

    • Variables

    • Response

  7. Reuse the same query in your API calls

Example:

BASE='<Pulse_UI_URL>' # or https / other host curl -sS -X POST "$BASE/graphql" \ -H 'Content-Type: application/json' \ -H 'MonitorGroup: odp_usopp' \ -H 'role: eyJuYW1lIjoiYWRtaW4iLCJjbHVzdGVycyI6W119' \ -H 'Cookie: jwt=PASTE_SIGNED_JWT_COOKIE' \ d @ <<'EOF' { "operationName": "OPTIONAL_OR_OMIT", "query": "PASTE_OR_INLINE_QUERY", "variables": { } }

What You Can Build

With these APIs, you can:

  • Build your own custom dashboards

  • Export data to BI tools or data lakes

  • Automate reports and pipelines

  • Monitor cluster utilization programmatically

  • Optimize resource usage and costs