Airflow Dashboard
You can use the following Airflow metrics displayed on the Airflow > Dashboard page to analyze them.
| Metric | Submetric | Description |
|---|---|---|
| Airflow Dag Processing Stats | Average Parse Time | The average amount of time taken to parse each file. |
| Files to be Scanned | The total number of files that need to be processed. | |
| Filesystem Updates | Any changes or updates made to the file system during the parsing process. | |
| Parse Errors | The number of errors encountered while parsing files. | |
| Airflow Dag Run Stats | Average Dag Dependency Check | The average time taken to check dependencies between tasks in a DAG. |
| Average Dag Success Rate | The average percentage of DAGs that are completed successfully. | |
| Average Dag Failed State | The average number of DAGs that enter a failed state. | |
| Average Dag Schedule Delay | The average delay experienced in starting DAGs according to their scheduled time. | |
| Airflow Dataset Stats | Orphaned Datasets | The datasets that are no longer associated with any Active DAGs. |
| Dags Triggered | Tracks the number of times a DAG has been initiated for execution, indicating the frequency of workflow runs. | |
| Dataset Updates | The dataset updates are changes to datasets that DAGs can depend on. | |
| Airflow Pool Stats | Deferred Pool Stats | Number of tasks waiting for external conditions to proceed. |
| Open Pool Stats | Available capacity in a pool for additional tasks. | |
| Queued Pool Stats | Number of tasks waiting in the queue for execution. | |
| Running Pool Stats | Number of tasks currently executing in the pool. | |
| Starving Pool Stats | Number of tasks unable to run due to a lack of available resources. | |
| Airflow Task Finish Instance Stats | Failure | The Task execution failed. |
| Success | The task was completed successfully. | |
| Deferred | The task is waiting for an external trigger to continue. | |
| Running | The task is currently executing. | |
| Queued | The task is waiting in line for execution. | |
| Removed | The task was removed before execution. | |
| Scheduled | The task is planned for execution at a later time. | |
| Skipped | The task was intentionally not executed. | |
| Rescheduled | The task was postponed to run at a different time. | |
| Upstream | The task is dependent on upstream tasks before it can run. | |
| Rescheduled | The task was postponed to run at a different time. | |
| Upstream | The task is dependent on upstream tasks before it can run. | |
| Airflow Executor Stats | DAGs | Directed Acyclic Graphs that define workflows in Airflow. |
| Executor Open Slots | Available slots in the executor for running new tasks. | |
| Airflow Dag Time Stats | Average Dag Queue Time | The average time a DAG spends waiting in the queue before starting. |
| Average Dag Run Time | The average time it takes for a DAG to complete its execution. | |
| Airflow Scheduler Stats | Time Spent on Critical Section | Time spent by the scheduler in critical sections of code. |
| Orphaned Tasks Adopted | Number of orphaned tasks that the scheduler has taken over. | |
| Orphaned Tasks Cleared | Number of orphaned tasks that have been removed by the scheduler. | |
| Loop Duration | Time taken for one complete cycle of the scheduler's loop. | |
| Task Executable | Number of tasks determined to be ready for execution by the scheduler. | |
| Task Starving | Number of tasks that are unable to execute due to resource constraints. |
Was this page helpful?