Release Notes 3.5.X

Disclaimer All versions of Pulse are supported for maintenance for a period of 18 months following their release. This maintenance support includes updates for security vulnerabilities, bug fixes, Common Vulnerabilities and Exposures (CVEs), and the introduction of new features, among other improvements. After the 18-month support period, we strongly recommend transitioning to the most recent version or to another version that continues to receive support. This ensures an enhanced user experience.

Version 3.5.3

Date: 30th January, 2025

• New Flag Added in Yarn Optimizer: The utiliseFQDNYarnOptimise flag has been introduced in the Configuration File of Yarn App Metrics. This flag allows the Yarn optimizer to toggle between using the hostname and hostname -f for resolving the node's hostname where the NodeManager or ResourceManager is running.

• Corrected the log rotation file to clear the log file generated by the Pulseaxn Agent.

Version 3.5.2

Date: 14 January, 2025

• Upgraded the Ubuntu Base Image: Upgraded the Ubuntu Base Image of the containers (Logstash, Elastic, and MongoDB) from version 23.04 to 24.04, as the older version has reached End-of-Life (EOL), ensuring continued security updates, stability, and support.
• Overriding the default-endpoints.yml File Details: Pulse now allows you to override specific information in the default-endpoints.yml file. For instance, if you need to modify the URL of a single component within a service, you must copy the entire section of that service into the override file and update the required URL. For example, to override the URL of Druid’s coordinator component, you need to copy the entire Druid Service section into the override file and modify only the coordinator URL. For detailed steps on overriding the file details, see Overriding the default-endpoints.yml File Details.

• Configuration Update on Yarn Optimizer: The Pulse Yarn Optimizer capability now allows you to configure IsRestEventSupported in ad-yarn-optimizer.yml to enable or disable the REST API call for overcommitment of node memory. For more details, see Enabling the REST API call for Overcommitment of Node Memory.
• (Beta) Pulse New Facelift UI : The Pulse UI Home and all other features have undergone a refreshed design to enhance the overall user experience. For detailed information on the Pulse UI Home Landing Page, refer to the Getting Started with the Pulse NextGen UI.

Version 3.5.1

Date: 22nd November, 2024

• Security Enhancement on the Pulse Agent Side: This release strengthens security on the Pulse Agent by enforcing strict file ownership and permissions within the directory. All the files have an owner defined by Acceldata and there are no file permissions given to other users.
• Security Enhancement for the Template Files: This release enhances security by removing the default plain text passwords from the template or configuration files to avoid access to unauthorized users.
• Support for Yarn Optimizer on the CDP distribution: Pulse now provides support for Yarn Optimizer on the CDP distribution by allowing you to configure the port details. For details about configuring the port, see Configure the OverCommit Timeout Value and Node ID Port.
• Enhancement in Setting the Yarn Optimisation OverCommit Timeout Value: Pulse now improves the Yarn Optimization workflow by allowing you to configure the OverCommit Connection Timeout Value to the pipeline. If the optimization is not done within the timeout value due to the NATS connection issue or any other issues, the memory automatically gets reverted to the original value without any optimization to ensure system stability. For details about the configuration, see Configure the OverCommit Timeout Value and Node ID Port.
• Upgraded the Postgres Version: This release upgraded the Postgres version from 12.X to 16.4, as the Postgres version 12.X has reached End-Of-Life now.
• Removed the Agent Status Trend Chart: This release removes the Agent Status Trend chart from the Pulse Audit UI page as normally users are interested in knowing only the current status of Agents rather than tracking status trends over time.
• Split the Agents Summary: This release splits the Agents summary into individual Agents Summary. The individual Agents Summary now shows the count of hosts for each Agent where the status is currently down.

Version 3.5.0

Date: 31st October, 2024

• Introduced the Airflow Observability: Pulse now provides the observability capability for Apache Airflow including real-time monitoring of Airflow from StatsD metrics, detailed scheduler logs for easier debugging and analysis, and comprehensive tracking of DAG runs and related events. These observability capabilities enable efficient monitoring and management of Airflow workflows, ensuring better performance and reliability. For more information, see Airflow.

• Introduced the YARN Optimizer Capability: The Pulse Yarn Optimizer capability is designed to enhance the performance of Yarn Clusters by optimizing the allocation of memory for yarn jobs, leading to a significant performance improvement. The Beta Yarn Optimizer capability is now generally available with improved configurations and dashboards. For more information, see Yarn Optimizer.

• Introduced the Pulse Auditing Capability: Pulse now introduces an Auditing capability to show the Agent and Pulse statistics such as hosts where the Agent status is down, page login activity, the most viewed pages of Pulse, etc. For more information, see Pulse Audit.

• Introduced the Kafka Connect Observability: The Pulse observability platform now observes Kafka Connect and its components such as Workers, Connectors, Tasks, etc. Also, monitors the entire data pipeline between Kafka and other systems, providing detailed insights into the nature and location of failures. These capabilities allow for quick corrective actions to resolve issues in the Kafka Connect. For more information, see Kafka Connect.

• Introduced Kafka Cruise Control Observability: Pulse provides comprehensive observability of Cruise Control by monitoring key performance metrics to ensure optimal operation and resource utilization. These observability capabilities enable efficient monitoring and management of Kafka Cruise Control workflows. For more information, see Kafka Cruise Control.

• Introduced the Schema Registry Observability: Pulse now provides observability capabilities such as monitoring the overall health and performance of the Schema Registry, providing insights into the percentage of successful and failed requests processed, the total number of errors encountered, etc. These observability capabilities enable you to ensure that the Schema Registry is scaled appropriately to handle the data load. For more information, see Schema Registry.

• Introduced the NiFi Registry Observability: Pulse now provides the observability capability for NiFi Registry and raises an alert when the system hosting NiFi Registry is not active. For more information, see NiFi Registry.

• Enhanced the NiFi Observability: Pulse now provides NiFi service details such as issues with the service, non-response, shutdown or startup view, etc. For more information, see NiFi.

• Introduced the Impala Coordinator metrics: Pulse now provides the Impala coordinator metrics to create dashboards and alerts. For details about the available Impala Coordinator metrics, see Impala Metrics. For details about the newly added Impala alerts and the complete list, see Impala Alerts.

• Introduced the New Action Service Runbooks: Pulse now introduces these three action runbooks to automate tasks: Perform HDFS Non Admin Operations, HDFS Top 10 Directories, and Kerberos. Also, the Actions Service workflow has been improved:

  • Drag and drop multiple action runbooks on the Action Workflow box to create a single action plan.
  • Send notifications to Email, JIRA, and Slack channels when the execution is completed.
  • Schedule and Save action plans.
  • In essence, when an alert is generated, Pulse automatically performs certain tasks and sends notifications to specific channels without requiring manual intervention. For more information, see Actions Service.

• Support for Big Panda Notification Channels: Pulse now supports notifications via the Big Panda notification channel. For more information, see Big Panda Notifications.

• Enhanced the Impala service: The Impala agent now collects Impala coordinator metrics. For more information about enabling the Impala data collection see Impala Connector and Configuring Pulse Agents.

• Enhanced the HDFS Filtering Records Capability: Pulse now provides the following filter options on the HDFS > File Explorer page to filter files and folders:

  • File Replication: Displays the file folders based on the selected file replication value for non-standard replication, that is, greater or less than 3.
  • Using the Last Modified and Last Accessed options, you can now select the custom date range to fetch the older records.

• Update in Deploying Playbooks: The steps used to deploy Playbooks using the command accelo deploy playbooks are no longer supported. For details about the new steps, see Deploy Playbooks.

• Deprecated the accelo CLI commands: The following CLI commands have been now removed from the list of supported commands from the Pulse version 3.5.0:

  • accelo admin database purge-db
  • accelo admin database init-tsdb
  • accelo deploy ad-sa-logsearch
  • accelo deploy ad-sa-db
  • accelo deploy fsa-es
  • accelo admin recreate

• Removed the flags from accelo set: The following accelo set flags have been removed effective this release:
  • -c, --cluster string Cluster name to set
  • -m, --multi-cluster Enable multicluster

• Pulse Installation
• Upgrade to Version 3.5.0