Airflow

Apache Airflow is an open-source platform designed to author, schedule, and monitor workflows programmatically.

Pulse provides comprehensive observability for Airflow deployments running on Kubernetes or in a Hadoop cluster. It enables administrators and platform teams to monitor DAG execution performance, scheduler and executor health, task states, resource utilization, and runtime trends in near real time.

With Pulse, you can:

• Monitor the Airflow scheduler, executor, and triggerer health
• Track DAG success/failure trends
• Analyze task execution timelines
• Observe queue delays and scheduling latency
• Monitor Kubernetes-based executor load
• Drill down from DAG-level to task-level execution details

Explore In Detail

For details, access the following pages.

• Monitor the Airflow Logs and Service details
• Monitor the Airflow with Alerts
• Airflow Dashboard
• Airflow DagRuns