Enhancements

What's New

This section consists of the enhancements by services introduced in this release.

Ambari

Bug ID

Apache JIRA

Description

ODP-1336


Added the cluster admin permission to allow user management access and fix python3 warnings.

ODP-1629


Changed the Ambari as per knox log4j2 format in ODP-3.2.2.0-2.

ODP-1690_python3


Added the Knox Homepage UI to Knox via quicklinks.json (#102)

ODP-1627


Added the key admin ldap user.

ODP-1873

AMBARI-26118 AMBARI-26119 AMBARI-26122 AMBARI-25107 AMBARI-26130 AMBARI-25956 AMBARI-26093

Fixed a few security issues and component getting restarted in the maintenance mode with restart all.

ODP-1816


Implemented the Ranger Kafka plugin for both Kafka and Kafka 3 (#120).

ODP-2359


Fixed the webhdfs url handling for SSL enabled hdfs clusters.

ODP-1926


Hive now populates the Ranger service cache for Impala fix.

ODP-2375


Added the Druid SSL Properties.

ODP-1353


Added the ODP-3.3 the Ambari stack.

ODP-1913


Upgraded the Java library versions for CVE reconciliation.

ODP-2577


Added the Kafka Mirrormaker 2 observability for JMX port. (#154)

ODP-2519


Added the Kafka connect observability for JMX.

ODP-2491


Migrated from init.d scripts to an alternative system for Ambari server and agent (#157).

ODP-2428


Added the support for Ubuntu22 and RHEL 9 (#156)

ODP-2617


Updated stack 3.3 and added stack 3.4.

ODP-2527


Removed the init.d script for ranger-kms.

ODP-2428


Added the support for Ubuntu 22 and RHEL 9 for new stacks.

ODP-2425


Enabled the Solr auditing for Kudu (Python 3).

ODP-2822


Added the default Ozone admin user for Ranger KMS service (#174).

ODP-2805


Fixed the Python 3 candidates and version check for ambari-python-wrap.

ODP-2719


Updated the script shebangs to use ambari-python-wrap (#168).

ODP-2752


Removed the stack ODPWIN.

ODP-2747


Removed the stacks older than 3.0 (#164)

ODP-2977


Handled the task Name Node restart timeout during the rolling upgrade.

ODP-3206


Fixed the kerberos deprecated encryption types (#186).

ODP-787


Fixed the Oozie directory permissions to support custom Oozie user (#183).

ODP-3198


Updated the Python version to 3.11.

ODP-3432


Made Ambari Python 3 compatible by replacing iteritems() with items() (#200).

ODP-3530


Added the Systemd support for Ambari and fixed the executable issue (#208).

ODP-3540


Applied multiple checks before the Ranger setup for impala to fix the Hive Client Installation failures.

Hadoop

Bug ID

Apache JIRA

Description

ODP-2583

HADOOP-11616

Upgraded Curator to version 5.2.0. #33

ODP-2635

HADOOP-16167

Fixed the Hadoop shell script for Ubuntu 18. (#44)

ODP-2634

YARN-10352

Skips scheduling on not heartbeated nodes in Multi Node Placement.(#43)

ODP-2636

MAPREDUCE-7431

Refactored ShuffleHandler and fixes issues observed after Netty4 upgrade. (apache#5311). (#47)

ODP-2638

MAPREDUCE-7433

Removed the unused mapred or LoggingHttpResponseEncoder.java (apache#5388) (#48).

ODP-2640

HADOOP-16647

Supports OpenSSL 1.1.1 LTS - patch-0.

ODP-2831

MAPREDUCE-7373

This release now uses CMAKECXX standard instead of -std=c++11 directly.

ODP-2768


Refactored shebangs for python scripts to ambari-python-wrap.

ODP-3570


Provides the NVIDIA GPU Support with MIG Compatibility for YARN 3.2.3.

Airflow

Bug ID

Apache JIRA

Description

ODP-3406


Automates quick links in case of SSL enablement.

ODP-3317


Limits the Airflow logs creation.

ODP-2757


Supports Airflow installation through tarball.

Bug ID

Apache JIRA

Description

ODP-2564 ODP-2412

DRUID-17451:

Added support for using Dropwizard and parameterized emitters together in the Druid component.

ODP-2718 ODP-2759


Refactored shebangs for Python scripts to ambari-python-wrap (#20).

Flink

Bug ID

Apache JIRA

Description

ODP-2676


Fixed the flink-shaded-zookeeper-3 version.

ODP-2774


Refactored shebangs for Python scripts to ambari-python-wrap.

HBase

Bug ID

Apache JIRA

Description

ODP-2583


Upgraded Curator to 5.2.0.

ODP-2641


Downgraded Jackson for Hadoop compatibility.

ODP-2761


Refactored shebangs for Python scripts to ambari-python-wrap.

Hive

Bug ID

Apache JIRA

Description

ODP-3080

Apache HIVE#4887

Added the cookie signature verification to HiveServer2.


HIVE-27952

Uses SslContextFactory.Server() instead of SslContextFactory (apache#4947) (#62).

ODP-2718|

ODP-2733


Refactored shebangs for Python scripts to ambari-python-wrap(#61).

Hue

Bug ID

Apache JIRA

Description

ODP-3632

HUE-9390

Removed the MySQL-python library and added the Mysqlclient library.

Impala

Bug ID

Apache JIRA

Description

ODP-3004


Defined IMPALA_SYSTEM_PYTHON {2,3} once before checking its value.

ODP-2739


Refactored to use ambari-python-wrap.

Kafka

Bug ID

Apache JIRA

Description

ODP-1921


Updated the jackson,netty and snappy versions in dependencies.gradle.

ODP-2577


Introduced Kafka Mirrormaker 2 observability for jmx port.

ODP-1924


Introduced the Kafka connect observability for jmx port.

ODP-2654 ODP-2480


Refactored the Kafka-connect jar names based on the JDK version.

ODP-2772


Refactored shebangs for Python scripts to ambari-python-wrap.

Kafka 3

Bug ID

Apache JIRA

Description

ODP-2428


Added support for RHEL 9 and Ubuntu 22.

ODP-3602 ODP-2577


Added the JMX ports for Kafka Connect 3 and Mirrior Maker 3.

Cruise Control

Bug ID

Apache JIRA

Description

ODP-3451


Added log rotation for Cruise-Control 2 and 3.

Knox

Bug ID

Apache JIRA

Description

ODP-1550

KNOX-1462

Added commons-text as an undeclared dependency and fixed Knox Log4j vulnerability (CVE-2019-17571) detected after ODP upgrade to 3.2.2.0-2.

ODP-2583


Upgraded Curator to 5.2.0.

ODP-2718 ODP-2773


Refactored shebangs for Python scripts to ambari-python-wrap.

Livy

Bug ID

Apache JIRA

Description

ODP-2050


Upgraded bootstrap to 4.6.2

ODP-2050


Upgraded jQuery to 3.7.1.

ODP-2050

LIVY-903

Addressed Netty upgrades for version 0.8.0.

ODP-2050


Upgraded jQuery from 1.10.15 to 1.13.11.

ODP-2795


Refactored Livy to use ambari-python-wrap.

NiFi

Bug ID

Apache JIRA

Description

ODP-1880


Enabled the endpoint override option for S3FileResourceService to allow internal object store custom URLs

ODP-2306

NIFI-12219 NIFI-12858 NIFI-13532 NIFI-12524

Added the Xodus Flow History Storage with migration.

ODP-2058

NIFI-12969 NIFI-12232

Ensured that if a Process Group lacks a Versioned Component ID, we use ComponentIdLookup to generate one based on its Instance ID, aligning with the approach used during flow serialization.

This guarantees consistent IDs when synchronizing flows across the cluster.

Additionally, performed code cleanup, improving failure handling during startup.

ODP-2771


Refactored shebangs for Python scripts to ambari-python-wrap.

Oozie

Bug ID

Apache JIRA

Description

ODP-2711


Updated the curator version to 5.2.0 in oozie.

ODP-2707


Removed the ZKLockService class to make Oozie compatible with curator 5.2.0.

ODP-3117


Migrated jquery-1.8.3 to jquery-3.7.1.

ODP-2070


Fixed the Log4j 1.* issue in Oozie.

ODP-2760


Refactored Oozie to use ambari-python-wrap for all scripts.

Ozone

Bug ID

Apache JIRA

Description

ODP-2583


Updated the curator version to 5.2.0 in Ozone.

ODP-2147


Fixed the log4j properties file names for all components to match the ODP stack.

ODP-2767


Refactored shebangs for Python scripts to ambari-python-wrap.

Ranger

Bug ID

Apache JIRA

Description

ODP-2583


Updated the curator version to 5.2.0 in Oozie.

ODP-1816


Added the Kafka Ranger environment for KAFKA3 compatibility.

ODP-2523


Updated the ODP stack project dependencies in ranger as target version.

ODP-1394


Updated Ozone Service Check User Policy to delegate service check user as admin only within the policy.

ODP-1456


Enabled the Kudu plugin in Ranger.

ODP-2274


Fixed the jackson-core version in kms

ODP-3190


Added a message to Kudu ranger plugin connection test.

ODP-3062


Updated try-catch block to catch IOException in EagerKeyGeneratorKeyProviderCryptoExtension.java.

ODP-3395

RANGER-4915

Fixed the default SSL ciphers for the User Sync service to enhance security.

ODP-2781


Refactored shebangs for Python scripts to ambari-python-wrap.

ODP-3326

RANGER-4775

Fixes an issue where Ranger KMS fails when using the Oracle 23 database.

ODP-3039


Added the inclusion of bcprov-jdk15on and the versioned bcpkix-jdk15on jar to the Ranger webapp Ozone libraries.

Spark 2

Bug ID

Apache JIRA

Description

ODP-2667


Added the scala-reflect dependency to core/pom.xml.

ODP-2061


Replaced log4j with reload4j in Spark2.

ODP-2656


Upgraded Jetty, Jackson, and netty-all to match Hadoop.

ODP-2583


Upgraded Apache Curator to version 5.2.0

Schema Registry

Bug ID

Apache JIRA

Description

ODP-2505


Integrated Schema Registry observability JMX ports into the ODP stack.

ODP-2765


Refactored Schema Registry to use ambari-python-wrap.

ODP-2105


Fixed Scala 2.13 and jackson 2.14.2 compatibility.

Sqoop

Bug ID

Apache JIRA

Description

ODP-1987


This release uses asciidoctor tool to compile docs instead of asciidoc and xmlto (#6).

ODP-2764


Refactored Sqoop to use ambari-python-wrap for all scripts.

ODP-2848


Fixed the Sqoop compilation for RHEL 9.

Tez

Bug ID

Apache JIRA

Description

ODP-2642

TEZ-4580

Addressed slow preemption of new containers when re-use is enabled (unmerged).

ODP-2643

TEZ-4488

This release ensured TaskSchedulerManager is initialized when the first DAG arrives.

ODP-2645

TEZ-4460

Improved Shuffle Handler reliability by correcting the usage of EMPTY_LAST_CONTENT and channel write to prevent read timeouts.

ODP-2646

TEZ-4455

Improved debuggability by adding LoggingHandler to the ShuffleHandler pipeline.

ODP-2647

TEZ-4450

Improved shuffle data fetching reliability when using CompositeRoutedDataMovementEvent for data transfer.

ODP-2648

TEZ-4383

Upgraded to Mockito 4.3.1.

ODP-2649

TEZ-4394

Optimized Netty4 ShuffleHandler by configuring it to use a single boss thread.

ODP-2650

TEZ-4334

Resolved potential deadlock in ShuffleScheduler between ShuffleScheduler.close() and the ShufflePenaltyReferee thread.

ODP-2651

TEZ-4336

Improved ShuffleScheduler to report the original exception when shuffle becomes unhealthy.

ODP-2718 ODP-2763


Refactored shebangs for Python scripts to ambari-python-wrap.

Zeppelin

Bug ID

Apache JIRA

Description

ODP-3111


Uses reload4j instead of log4j.

ODP-2782


Refactored Zeppelin to use ambari-python-wrap.

Zookeeper

Bug ID

Apache JIRA

Description

ODP-2632


Downgraded the Jetty and Jackson version to match Hadoop.

ODP-2769


Refactored shebangs for python scripts to ambari-python-wrap.

Mpack Enhancements

Hue (Mpack)

Bug ID

Apache JIRA

Description

ODP-2728


Enabled the SSL configuration for Hue through the Ambari UI.

ODP-2535


Added the HDFS automatic configurations to Hue.

ODP-3057


Added the missing package definition for Hue's MPack.

ODP-2815


Removed the compat-openssl dependency.

ODP-2826


Enabled the automatic configuration for Kerberos in Hue.

ODP-2476


Enabled the automatic configuration of Impala when it is enabled in Hue.


  Last updated