Enhancements
This section consists of the enhancements by services introduced in this release.
Ambari
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-1336 | Added the cluster admin permission to allow user management access and fix python3 warnings. | |
ODP-1629 | Changed the Ambari as per knox log4j2 format in ODP-3.2.2.0-2. | |
ODP-1690_python3 | Added the Knox Homepage UI to Knox via quicklinks.json (#102) | |
ODP-1627 | Added the key admin ldap user. | |
ODP-1873 | AMBARI-26118 AMBARI-26119 AMBARI-26122 AMBARI-25107 AMBARI-26130 AMBARI-25956 AMBARI-26093 | Fixed a few security issues and component getting restarted in the maintenance mode with restart all. |
ODP-1816 | Implemented the Ranger Kafka plugin for both Kafka and Kafka 3 (#120). | |
ODP-2359 | Fixed the webhdfs url handling for SSL enabled hdfs clusters. | |
ODP-1926 | Hive now populates the Ranger service cache for Impala fix. | |
ODP-2375 | Added the Druid SSL Properties. | |
ODP-1353 | Added the ODP-3.3 the Ambari stack. | |
ODP-1913 | Upgraded the Java library versions for CVE reconciliation. | |
ODP-2577 | Added the Kafka Mirrormaker 2 observability for JMX port. (#154) | |
ODP-2519 | Added the Kafka connect observability for JMX. | |
ODP-2491 | Migrated from init.d scripts to an alternative system for Ambari server and agent (#157). | |
ODP-2428 | Added the support for Ubuntu22 and RHEL 9 (#156) | |
ODP-2617 | Updated stack 3.3 and added stack 3.4. | |
ODP-2527 | Removed the init.d script for ranger-kms. | |
ODP-2428 | Added the support for Ubuntu 22 and RHEL 9 for new stacks. | |
ODP-2425 | Enabled the Solr auditing for Kudu (Python 3). | |
ODP-2822 | Added the default Ozone admin user for Ranger KMS service (#174). | |
ODP-2805 | Fixed the Python 3 candidates and version check for ambari-python-wrap. | |
ODP-2719 | Updated the script shebangs to use ambari-python-wrap (#168). | |
ODP-2752 | Removed the stack ODPWIN. | |
ODP-2747 | Removed the stacks older than 3.0 (#164) | |
ODP-2977 | Handled the task Name Node restart timeout during the rolling upgrade. | |
ODP-3206 | Fixed the kerberos deprecated encryption types (#186). | |
ODP-787 | Fixed the Oozie directory permissions to support custom Oozie user (#183). | |
ODP-3198 | Updated the Python version to 3.11. | |
ODP-3432 | Made Ambari Python 3 compatible by replacing iteritems() with items() (#200). | |
ODP-3530 | Added the Systemd support for Ambari and fixed the executable issue (#208). | |
ODP-3540 | Applied multiple checks before the Ranger setup for impala to fix the Hive Client Installation failures. |
Hadoop
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-2583 | HADOOP-11616 | Upgraded Curator to version 5.2.0. #33 |
ODP-2635 | HADOOP-16167 | Fixed the Hadoop shell script for Ubuntu 18. (#44) |
ODP-2634 | YARN-10352 | Skips scheduling on not heartbeated nodes in Multi Node Placement.(#43) |
ODP-2636 | MAPREDUCE-7431 | Refactored ShuffleHandler and fixes issues observed after Netty4 upgrade. (apache#5311). (#47) |
ODP-2638 | MAPREDUCE-7433 | Removed the unused mapred or LoggingHttpResponseEncoder.java (apache#5388) (#48). |
ODP-2640 | HADOOP-16647 | Supports OpenSSL 1.1.1 LTS - patch-0. |
ODP-2831 | MAPREDUCE-7373 | This release now uses CMAKE CXX standard instead of -std=c++11 directly. |
ODP-2768 | Refactored shebangs for python scripts to ambari-python-wrap. | |
ODP-3570 | Provides the NVIDIA GPU Support with MIG Compatibility for YARN 3.2.3. |
Airflow
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-3406 | Automates quick links in case of SSL enablement. | |
ODP-3317 | Limits the Airflow logs creation. | |
ODP-2757 | Supports Airflow installation through tarball. |
Druid
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-2564 ODP-2412 | DRUID-17451: | Added support for using Dropwizard and parameterized emitters together in the Druid component. |
ODP-2718 ODP-2759 | Refactored shebangs for Python scripts to ambari-python-wrap (#20). |
Flink
HBase
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-2583 | Upgraded Curator to 5.2.0. | |
ODP-2641 | Downgraded Jackson for Hadoop compatibility. | |
ODP-2761 | Refactored shebangs for Python scripts to ambari-python-wrap . |
Hive
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-3080 | Apache HIVE#4887 | Added the cookie signature verification to HiveServer2. |
HIVE-27952 | Uses SslContextFactory.Server() instead of SslContextFactory (apache#4947) (#62). | |
ODP-2718| ODP-2733 | Refactored shebangs for Python scripts to ambari-python-wrap (#61). |
Hue
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-3632 | HUE-9390 | Removed the MySQL-python library and added the Mysqlclient library. |
Impala
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-3004 | Defined IMPALA_SYSTEM_PYTHON {2,3} once before checking its value. | |
ODP-2739 | Refactored to use ambari-python-wrap. |
Kafka
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-1921 | Updated the jackson,netty and snappy versions in dependencies.gradle. | |
ODP-2577 | Introduced Kafka Mirrormaker 2 observability for jmx port. | |
ODP-1924 | Introduced the Kafka connect observability for jmx port. | |
ODP-2654 ODP-2480 | Refactored the Kafka-connect jar names based on the JDK version. | |
ODP-2772 | Refactored shebangs for Python scripts to ambari-python-wrap. |
Kafka 3
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-2428 | Added support for RHEL 9 and Ubuntu 22. | |
ODP-3602 ODP-2577 | Added the JMX ports for Kafka Connect 3 and Mirrior Maker 3. |
Cruise Control
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-3451 | Added log rotation for Cruise-Control 2 and 3. |
Knox
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-1550 | KNOX-1462 | Added commons-text as an undeclared dependency and fixed Knox Log4j vulnerability (CVE-2019-17571) detected after ODP upgrade to 3.2.2.0-2. |
ODP-2583 | Upgraded Curator to 5.2.0. | |
ODP-2718 ODP-2773 | Refactored shebangs for Python scripts to ambari-python-wrap. |
Livy
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-2050 | Upgraded bootstrap to 4.6.2 | |
ODP-2050 | Upgraded jQuery to 3.7.1. | |
ODP-2050 | LIVY-903 | Addressed Netty upgrades for version 0.8.0. |
ODP-2050 | Upgraded jQuery from 1.10.15 to 1.13.11. | |
ODP-2795 | Refactored Livy to use ambari-python-wrap. |
NiFi
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-1880 | Enabled the endpoint override option for S3FileResourceService to allow internal object store custom URLs | |
ODP-2306 | NIFI-12219 NIFI-12858 NIFI-13532 NIFI-12524 | Added the Xodus Flow History Storage with migration. |
ODP-2058 | NIFI-12969 NIFI-12232 | Ensured that if a Process Group lacks a Versioned Component ID, we use ComponentIdLookup to generate one based on its Instance ID, aligning with the approach used during flow serialization. This guarantees consistent IDs when synchronizing flows across the cluster. Additionally, performed code cleanup, improving failure handling during startup. |
ODP-2771 | Refactored shebangs for Python scripts to ambari-python-wrap. |
Oozie
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-2711 | Updated the curator version to 5.2.0 in oozie. | |
ODP-2707 | Removed the ZKLockService class to make Oozie compatible with curator 5.2.0. | |
ODP-3117 | Migrated jquery-1.8.3 to jquery-3.7.1. | |
ODP-2070 | Fixed the Log4j 1.* issue in Oozie. | |
ODP-2760 | Refactored Oozie to use ambari-python-wrap for all scripts. |
Ozone
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-2583 | Updated the curator version to 5.2.0 in Ozone. | |
ODP-2147 | Fixed the log4j properties file names for all components to match the ODP stack. | |
ODP-2767 | Refactored shebangs for Python scripts to ambari-python-wrap. |
Ranger
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-2583 | Updated the curator version to 5.2.0 in Oozie. | |
ODP-1816 | Added the Kafka Ranger environment for KAFKA3 compatibility. | |
ODP-2523 | Updated the ODP stack project dependencies in ranger as target version. | |
ODP-1394 | Updated Ozone Service Check User Policy to delegate service check user as admin only within the policy. | |
ODP-1456 | Enabled the Kudu plugin in Ranger. | |
ODP-2274 | Fixed the jackson-core version in kms | |
ODP-3190 | Added a message to Kudu ranger plugin connection test. | |
ODP-3062 | Updated try-catch block to catch IOException in EagerKeyGeneratorKeyProviderCryptoExtension.java. | |
ODP-3395 | RANGER-4915 | Fixed the default SSL ciphers for the User Sync service to enhance security. |
ODP-2781 | Refactored shebangs for Python scripts to ambari-python-wrap. | |
ODP-3326 | RANGER-4775 | Fixes an issue where Ranger KMS fails when using the Oracle 23 database. |
ODP-3039 | Added the inclusion of bcprov-jdk15on and the versioned bcpkix-jdk15on jar to the Ranger webapp Ozone libraries. |
Spark 2
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-2667 | Added the scala-reflect dependency to core/pom.xml. | |
ODP-2061 | Replaced log4j with reload4j in Spark2. | |
ODP-2656 | Upgraded Jetty, Jackson, and netty-all to match Hadoop. | |
ODP-2583 | Upgraded Apache Curator to version 5.2.0 |
Schema Registry
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-2505 | Integrated Schema Registry observability JMX ports into the ODP stack. | |
ODP-2765 | Refactored Schema Registry to use ambari-python-wrap. | |
ODP-2105 | Fixed Scala 2.13 and jackson 2.14.2 compatibility. |
Sqoop
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-1987 | This release uses asciidoctor tool to compile docs instead of asciidoc and xmlto (#6). | |
ODP-2764 | Refactored Sqoop to use ambari-python-wrap for all scripts. | |
ODP-2848 | Fixed the Sqoop compilation for RHEL 9. |
Tez
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-2642 | TEZ-4580 | Addressed slow preemption of new containers when re-use is enabled (unmerged). |
ODP-2643 | TEZ-4488 | This release ensured TaskSchedulerManager is initialized when the first DAG arrives. |
ODP-2645 | TEZ-4460 | Improved Shuffle Handler reliability by correcting the usage of EMPTY_LAST_CONTENT and channel write to prevent read timeouts. |
ODP-2646 | TEZ-4455 | Improved debuggability by adding LoggingHandler to the ShuffleHandler pipeline. |
ODP-2647 | TEZ-4450 | Improved shuffle data fetching reliability when using CompositeRoutedDataMovementEvent for data transfer. |
ODP-2648 | TEZ-4383 | Upgraded to Mockito 4.3.1. |
ODP-2649 | TEZ-4394 | Optimized Netty4 ShuffleHandler by configuring it to use a single boss thread. |
ODP-2650 | TEZ-4334 | Resolved potential deadlock in ShuffleScheduler between ShuffleScheduler.close() and the ShufflePenaltyReferee thread. |
ODP-2651 | TEZ-4336 | Improved ShuffleScheduler to report the original exception when shuffle becomes unhealthy. |
ODP-2718 ODP-2763 | Refactored shebangs for Python scripts to ambari-python-wrap. |
Zeppelin
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-3111 | Uses reload4j instead of log4j. | |
ODP-2782 | Refactored Zeppelin to use ambari-python-wrap . |
Zookeeper
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-2632 | Downgraded the Jetty and Jackson version to match Hadoop. | |
ODP-2769 | Refactored shebangs for python scripts to ambari-python-wrap. |
Mpack Enhancements
Hue (Mpack)
Bug ID | Apache JIRA | Description |
---|---|---|
ODP-2728 | Enabled the SSL configuration for Hue through the Ambari UI. | |
ODP-2535 | Added the HDFS automatic configurations to Hue. | |
ODP-3057 | Added the missing package definition for Hue's MPack. | |
ODP-2815 | Removed the compat-openssl dependency. | |
ODP-2826 | Enabled the automatic configuration for Kerberos in Hue. | |
ODP-2476 | Enabled the automatic configuration of Impala when it is enabled in Hue. |
Was this page helpful?