Flink

Apache Flink is a distributed processing engine for both streaming and batch data. It enables stateful, real-time computations over continuous data streams as well as processing of bounded datasets.

Observability with Pulse

Pulse provides Flink observability by providing a unified platform to monitor query performance, system health, and resource utilization across clusters.

This capability enables you to:

• Monitor Flink component health (such as the Job History Server) in real time.
• Analyze logs to troubleshoot issues and correlate events.
• Detect and resolve performance bottlenecks using Pulse custom alerts and charts.
• Track the Flink application summary and resource usage.
• Track jobs associated with each application to understand workload distribution.
• Track vertex-level data to analyze task and operator performance.
• Build custom dashboards and visualizations for continuous observability and optimization.

Before You Begin

To view Flink data in the Pulse UI, make sure you:

• Configure your Hadoop cluster to send Flink metrics to Pulse.
• Configure Pulse to process and display the collected data.

For detailed setup instructions, see the following pages:

• Configure ODP Flink
• Configure Pulse to Monitor Flink.

Explore in Detail

• Monitor Flink Health and Logs

• Monitor ClickHouse with Pulse Alerts

• Visualize Flink Application Metrics

• Monitor Flink Applications

• Monitor Flink Application Details

  • Analyze Flink Job Details