Enhancements

This section consists of the enhancements by services introduced in this release.

Ambari

Bug IDApache JIRADescription
ODP-1336Added the cluster admin permission to allow user management access and fix python3 warnings.
ODP-1629Changed the Ambari as per knox log4j2 format in ODP-3.2.2.0-2.
ODP-1690_python3Added the Knox Homepage UI to Knox via quicklinks.json (#102)
ODP-1627Added the key admin ldap user.
ODP-1873AMBARI-26118 AMBARI-26119 AMBARI-26122 AMBARI-25107 AMBARI-26130 AMBARI-25956 AMBARI-26093Fixed a few security issues and component getting restarted in the maintenance mode with restart all.
ODP-1816Implemented the Ranger Kafka plugin for both Kafka and Kafka 3 (#120).
ODP-2359Fixed the webhdfs url handling for SSL enabled hdfs clusters.
ODP-1926Hive now populates the Ranger service cache for Impala fix.
ODP-2375Added the Druid SSL Properties.
ODP-1353Added the ODP-3.3 the Ambari stack.
ODP-1913Upgraded the Java library versions for CVE reconciliation.
ODP-2577Added the Kafka Mirrormaker 2 observability for JMX port. (#154)
ODP-2519Added the Kafka connect observability for JMX.
ODP-2491Migrated from init.d scripts to an alternative system for Ambari server and agent (#157).
ODP-2428Added the support for Ubuntu22 and RHEL 9 (#156)
ODP-2617Updated stack 3.3 and added stack 3.4.
ODP-2527Removed the init.d script for ranger-kms.
ODP-2428Added the support for Ubuntu 22 and RHEL 9 for new stacks.
ODP-2425Enabled the Solr auditing for Kudu (Python 3).
ODP-2822Added the default Ozone admin user for Ranger KMS service (#174).
ODP-2805Fixed the Python 3 candidates and version check for ambari-python-wrap.
ODP-2719Updated the script shebangs to use ambari-python-wrap (#168).
ODP-2752Removed the stack ODPWIN.
ODP-2747Removed the stacks older than 3.0 (#164)
ODP-2977Handled the task Name Node restart timeout during the rolling upgrade.
ODP-3206Fixed the kerberos deprecated encryption types (#186).
ODP-787Fixed the Oozie directory permissions to support custom Oozie user (#183).
ODP-3198Updated the Python version to 3.11.
ODP-3432Made Ambari Python 3 compatible by replacing iteritems() with items() (#200).
ODP-3530Added the Systemd support for Ambari and fixed the executable issue (#208).
ODP-3540Applied multiple checks before the Ranger setup for impala to fix the Hive Client Installation failures.

Hadoop

Bug IDApache JIRADescription
ODP-2583HADOOP-11616Upgraded Curator to version 5.2.0. #33
ODP-2635HADOOP-16167Fixed the Hadoop shell script for Ubuntu 18. (#44)
ODP-2634YARN-10352Skips scheduling on not heartbeated nodes in Multi Node Placement.(#43)
ODP-2636MAPREDUCE-7431Refactored ShuffleHandler and fixes issues observed after Netty4 upgrade. (apache#5311). (#47)
ODP-2638MAPREDUCE-7433Removed the unused mapred or LoggingHttpResponseEncoder.java (apache#5388) (#48).
ODP-2640HADOOP-16647Supports OpenSSL 1.1.1 LTS - patch-0.
ODP-2831MAPREDUCE-7373This release now uses CMAKECXX standard instead of -std=c++11 directly.
ODP-2768Refactored shebangs for python scripts to ambari-python-wrap.
ODP-3570Provides the NVIDIA GPU Support with MIG Compatibility for YARN 3.2.3.

Airflow

Bug IDApache JIRADescription
ODP-3406Automates quick links in case of SSL enablement.
ODP-3317Limits the Airflow logs creation.
ODP-2757Supports Airflow installation through tarball.

Druid

Bug IDApache JIRADescription
ODP-2564 ODP-2412DRUID-17451:Added support for using Dropwizard and parameterized emitters together in the Druid component.
ODP-2718 ODP-2759Refactored shebangs for Python scripts to ambari-python-wrap (#20).
Bug IDApache JIRADescription
ODP-2676 Fixed the flink-shaded-zookeeper-3 version.
ODP-2774Refactored shebangs for Python scripts to ambari-python-wrap.

HBase

Bug IDApache JIRADescription
ODP-2583Upgraded Curator to 5.2.0.
ODP-2641Downgraded Jackson for Hadoop compatibility.
ODP-2761Refactored shebangs for Python scripts to ambari-python-wrap.

Hive

Bug IDApache JIRADescription
ODP-3080Apache HIVE#4887Added the cookie signature verification to HiveServer2.
HIVE-27952Uses SslContextFactory.Server() instead of SslContextFactory (apache#4947) (#62).

ODP-2718|

ODP-2733

Refactored shebangs for Python scripts to ambari-python-wrap(#61).

Hue

Bug IDApache JIRADescription
ODP-3632HUE-9390Removed the MySQL-python library and added the Mysqlclient library.

Impala

Bug IDApache JIRADescription
ODP-3004Defined IMPALA_SYSTEM_PYTHON {2,3} once before checking its value.
ODP-2739Refactored to use ambari-python-wrap.

Kafka

Bug IDApache JIRADescription
ODP-1921Updated the jackson,netty and snappy versions in dependencies.gradle.
ODP-2577Introduced Kafka Mirrormaker 2 observability for jmx port.
ODP-1924Introduced the Kafka connect observability for jmx port.
ODP-2654 ODP-2480Refactored the Kafka-connect jar names based on the JDK version.
ODP-2772Refactored shebangs for Python scripts to ambari-python-wrap.

Kafka 3

Bug IDApache JIRADescription
ODP-2428Added support for RHEL 9 and Ubuntu 22.
ODP-3602 ODP-2577Added the JMX ports for Kafka Connect 3 and Mirrior Maker 3.

Cruise Control

Bug IDApache JIRADescription
ODP-3451Added log rotation for Cruise-Control 2 and 3.

Knox

Bug IDApache JIRADescription
ODP-1550KNOX-1462Added commons-text as an undeclared dependency and fixed Knox Log4j vulnerability (CVE-2019-17571) detected after ODP upgrade to 3.2.2.0-2.
ODP-2583Upgraded Curator to 5.2.0.
ODP-2718 ODP-2773Refactored shebangs for Python scripts to ambari-python-wrap.

Livy

Bug IDApache JIRADescription
ODP-2050Upgraded bootstrap to 4.6.2
ODP-2050Upgraded jQuery to 3.7.1.
ODP-2050LIVY-903Addressed Netty upgrades for version 0.8.0.
ODP-2050Upgraded jQuery from 1.10.15 to 1.13.11.
ODP-2795Refactored Livy to use ambari-python-wrap.

NiFi

Bug IDApache JIRADescription
ODP-1880Enabled the endpoint override option for S3FileResourceService to allow internal object store custom URLs
ODP-2306NIFI-12219 NIFI-12858 NIFI-13532 NIFI-12524Added the Xodus Flow History Storage with migration.
ODP-2058NIFI-12969 NIFI-12232

Ensured that if a Process Group lacks a Versioned Component ID, we use ComponentIdLookup to generate one based on its Instance ID, aligning with the approach used during flow serialization.

This guarantees consistent IDs when synchronizing flows across the cluster.

Additionally, performed code cleanup, improving failure handling during startup.

ODP-2771Refactored shebangs for Python scripts to ambari-python-wrap.

Oozie

Bug IDApache JIRADescription
ODP-2711Updated the curator version to 5.2.0 in oozie.
ODP-2707Removed the ZKLockService class to make Oozie compatible with curator 5.2.0.
ODP-3117Migrated jquery-1.8.3 to jquery-3.7.1.
ODP-2070Fixed the Log4j 1.* issue in Oozie.
ODP-2760Refactored Oozie to use ambari-python-wrap for all scripts.

Ozone

Bug IDApache JIRADescription
ODP-2583Updated the curator version to 5.2.0 in Ozone.
ODP-2147Fixed the log4j properties file names for all components to match the ODP stack.
ODP-2767Refactored shebangs for Python scripts to ambari-python-wrap.

Ranger

Bug IDApache JIRADescription
ODP-2583Updated the curator version to 5.2.0 in Oozie.
ODP-1816Added the Kafka Ranger environment for KAFKA3 compatibility.
ODP-2523Updated the ODP stack project dependencies in ranger as target version.
ODP-1394Updated Ozone Service Check User Policy to delegate service check user as admin only within the policy.
ODP-1456Enabled the Kudu plugin in Ranger.
ODP-2274Fixed the jackson-core version in kms
ODP-3190Added a message to Kudu ranger plugin connection test.
ODP-3062Updated try-catch block to catch IOException in EagerKeyGeneratorKeyProviderCryptoExtension.java.
ODP-3395RANGER-4915Fixed the default SSL ciphers for the User Sync service to enhance security.
ODP-2781Refactored shebangs for Python scripts to ambari-python-wrap.
ODP-3326RANGER-4775Fixes an issue where Ranger KMS fails when using the Oracle 23 database.
ODP-3039Added the inclusion of bcprov-jdk15on and the versioned bcpkix-jdk15on jar to the Ranger webapp Ozone libraries.

Spark 2

Bug IDApache JIRADescription
ODP-2667Added the scala-reflect dependency to core/pom.xml.
ODP-2061Replaced log4j with reload4j in Spark2.
ODP-2656Upgraded Jetty, Jackson, and netty-all to match Hadoop.
ODP-2583Upgraded Apache Curator to version 5.2.0

Schema Registry

Bug IDApache JIRADescription
ODP-2505Integrated Schema Registry observability JMX ports into the ODP stack.
ODP-2765Refactored Schema Registry to use ambari-python-wrap.
ODP-2105Fixed Scala 2.13 and jackson 2.14.2 compatibility.

Sqoop

Bug IDApache JIRADescription
ODP-1987This release uses asciidoctor tool to compile docs instead of asciidoc and xmlto (#6).
ODP-2764Refactored Sqoop to use ambari-python-wrap for all scripts.
ODP-2848Fixed the Sqoop compilation for RHEL 9.

Tez

Bug IDApache JIRADescription
ODP-2642TEZ-4580Addressed slow preemption of new containers when re-use is enabled (unmerged).
ODP-2643TEZ-4488This release ensured TaskSchedulerManager is initialized when the first DAG arrives.
ODP-2645TEZ-4460Improved Shuffle Handler reliability by correcting the usage of EMPTY_LAST_CONTENT and channel write to prevent read timeouts.
ODP-2646TEZ-4455Improved debuggability by adding LoggingHandler to the ShuffleHandler pipeline.
ODP-2647TEZ-4450Improved shuffle data fetching reliability when using CompositeRoutedDataMovementEvent for data transfer.
ODP-2648TEZ-4383Upgraded to Mockito 4.3.1.
ODP-2649TEZ-4394Optimized Netty4 ShuffleHandler by configuring it to use a single boss thread.
ODP-2650TEZ-4334Resolved potential deadlock in ShuffleScheduler between ShuffleScheduler.close() and the ShufflePenaltyReferee thread.
ODP-2651TEZ-4336Improved ShuffleScheduler to report the original exception when shuffle becomes unhealthy.
ODP-2718 ODP-2763Refactored shebangs for Python scripts to ambari-python-wrap.

Zeppelin

Bug IDApache JIRADescription
ODP-3111Uses reload4j instead of log4j.
ODP-2782Refactored Zeppelin to use ambari-python-wrap.

Zookeeper

Bug IDApache JIRADescription
ODP-2632Downgraded the Jetty and Jackson version to match Hadoop.
ODP-2769Refactored shebangs for python scripts to ambari-python-wrap.

Mpack Enhancements

Hue (Mpack)

Bug IDApache JIRADescription
ODP-2728Enabled the SSL configuration for Hue through the Ambari UI.
ODP-2535Added the HDFS automatic configurations to Hue.
ODP-3057Added the missing package definition for Hue's MPack.
ODP-2815Removed the compat-openssl dependency.
ODP-2826Enabled the automatic configuration for Kerberos in Hue.
ODP-2476Enabled the automatic configuration of Impala when it is enabled in Hue.
Type to search, ESC to discard
Type to search, ESC to discard
Type to search, ESC to discard
  Last updated