ODP-Specific Additions Compared to Apache SeaTunnel

This section describes the ODP-specific changes introduced on top of Apache SeaTunnel 2.3.13 in the nightly/ODP-3.3.6.6 and nightly/ODP-3.4.3.0 branches.

It covers execution engine integrations, version alignment, build fixes, deployment changes, excluded components, and Ambari management pack (mpack) customizations.

Each change includes the associated commit or ticket and a reference to the relevant documentation section.

Reference: Run the following command in theAcceldata SeaTunnel fork to identify changes introduced after the upstream 2.3.13 tag:

git log --oneline 2.3.13..HEAD

1. Execution Engine and Translation Modules

ODP extends Apache SeaTunnel 2.3.13 with additional translation and starter modules for Flink and Spark.

The supported integrations include Flink 1.19 and 2.2, Spark 3.5, and Spark 4.1.

Engine integration changes

Spark 4.1 integration across ODP versions

Spark 4.1.x requires JDK 17.

ODP 3.4.3.0

Spark 4.1.2 is the default Spark 4 target. The following properties define the version and Scala dependency:

spark.4.1.2.version=4.1.2.${odp.release.version}
scala.spark4.version=2.13.17

The following modules are built and included in the distribution:

• seatunnel-spark-4.1-starter
• seatunnel-translation-spark-4.1

After the Scala-independent changes introduced in ODP-8153, these modules produce binaries identical to those in the ODP 3.4.3.0 branch.

To use Spark 4.1 on ODP 3.3.6.6, operators must:

1. Deploy JDK 17.
2. Enable the enable_spark4 mpack configuration option.

Flink 2.2.1 integration

ODP 3.4.3.0 includes Flink 2.2.1 with the following modules:

• seatunnel-flink-2-starter
• seatunnel-translation-flink-2

The -Pflink-2 build profile is enabled by default. The -Pflink-1.19 profile remains available as an optional fallback.

The version is defined by:

flink.2.2.1.version=2.2.1.${odp.release.version}

• FLINK20 enumeration
• seatunnel-translation-flink-20 module

A complete Flink 2.2.1 backport to ODP 3.3.6.6 requires a separate engineering ticket.

ODP-8128: SeaTunnel and Flink launcher fixes

The launcher changes include:

• Updated the Zeta launcher to honor the $SEATUNNEL_CLASSPATH environment variable.
• Updated the Flink launcher to change to $APP_DIR when accessing runtime.tar.gz.
• Updated AbstractFlinkStarter to pass --config <basename> in YARN application mode.
• Fixed the -t short-option binding in FlinkCommandArgs.

ODP-8129: Kerberos UGI integration

The Kerberos integration changes include:

• Added _HOST substitution for the File, JDBC, and Kudu connectors.
• Added Kerberos authentication support for HBase.
• Wrapped Iceberg Catalog operations in a UGI proxy.
• Added UGI doAs execution for Hudi.

ODP-8142: Flink 2 and Hadoop 3.4 compatibility fixes

ODP-8142 includes three changes:

1. FTP connector compatibility: Added a local SeaTunnelFTPInputStream implementation to support the relocation of Apache Commons Net classes to org.apache.hadoop.shaded.* in Hadoop 3.4.
2. Flink 2 starter packaging: Updated the seatunnel-dist assembly to include seatunnel-flink-2-starter.
3. Flink 2 job submission: Updated AbstractFlinkStarter to use flink run for FLINK2 because Flink 2.x removed the run-application subcommand.

The FTP connector compatibility change is not required for ODP 3.3.6.6 because Hadoop 3.3.6 does not relocate Commons Net classes.

The Flink 2 starter packaging and job submission changes depend on the Flink 2.2.1 backport.

ODP-8153: Stability and compatibility improvements

ODP-8153 and ODP-8153-336 include four changes.

1. Zeta REST request handling (F-17)

Updated BaseServlet to parse query strings manually.

This change prevents form-encoded requests to /submit-job from losing the HOCON request body.

2. Spark plugin archive handling (F-21)

The original implementation updated SparkStarter to write plugin archives to java.io.tmpdir and remove them by using deleteOnExit.

However, this approach introduced a lifecycle race between two JVM processes.

The subsequent F-21 regression fix replaces this approach with a reference to Common.pluginTarball(), which uses the mpack-managed archive at:
$SEATUNNEL_HOME/plugins.tar.gz

Updated seatunnel-common to exclude unshaded io.netty dependencies introduced transitively by seatunnel-arrow.

This change prevents Netty version conflicts with the host execution engine.

4. Scala-independent translation modules

Refactored translation modules so that the same compiled starter and translation JARs work with:

• Spark 3.5 and Scala 2.12.18
• Spark 4.1 and Scala 2.13.17

The change eliminates the need for separate Scala-specific build passes.

Shade exclusions also prevent scala-library and scala-reflect from being included in SeaTunnel fat JARs. The host execution engine provides these dependencies at runtime.

F-21 regression follow-up

Staged — Not included in a nightly build

The F-21 regression follow-up replaces the java.io.tmpdir and deleteOnExit approach with Common.pluginTarball().

The updated implementation references the plugin archive created by the Ambari mpack:

$SEATUNNEL_HOME/plugins.tar.gz

common-services/SEATUNNEL/2.3.13/package/scripts/setup.py

• The original AccessDenied error.
• The deleteOnExit lifecycle race between the two JVM processes.

Validation: The changes were validated end-to-end on October 9, 2026, using a Spark 4 YARN workload.

Workload: Spark 4 YARN W1
Application ID: application_1791554762409_0019
Final status: FINISHED/SUCCEEDED

Integration with the Ambari mpack

The starter modules are packaged in the starter/ directory within the mpack tarball.

During installation, the install() method in master.py invokes:
stack_select.select("seatunnel-server", version)

/usr/odp/current/seatunnel

2. Version Alignment with the ODP Stack

The SeaTunnel fork includes dependency and version updates to maintain compatibility with the ODP software stack.

3. JDK and Build Fixes

The following changes address Java compatibility, module compilation, and code formatting.

Integration: These changes apply only during compilation. The released SeaTunnel tarball includes the resulting fixes.

4. Deployment Path

The SeaTunnel fork includes changes to support artifact publishing and deployment through the ODP build system.

Integration with the ODP build system

The do-component-build process in odp-bigtop invokes Maven deployment when the following property is enabled:

deploy_flag=true

5. Components Deferred or Removed Compared to Upstream

The ODP distribution excludes certain upstream connectors and execution engine modules because they are incompatible with or not required by the supported ODP stacks.

Druid connector

• Status: Excluded from the ODP 3.4.3.0 build.
• Commit: cec24c6d7
• Reason: The connector is incompatible with the Druid version included in ODP 3.4.3.0.
• Alternative: Use the HdfsFile sink followed by Druid batch ingestion.

Huawei OBS connector

• Status: Excluded from the ODP 3.4.3.0 build.
• Commit: cec24c6d7
• Reason: The connector is not applicable to the current ODP customer base.

Legacy Spark and Flink modules

• Status: Removed.
• Commit: d7bae486f
• Affected versions:
  • Spark 2.x
  • Spark 3.3
  • Flink versions earlier than 1.19
  • Flink 2.0
• Reason: These execution engine versions are not supported by the ODP stacks.

6. ODP-Specific Ambari Mpack Customizations

The following capabilities are implemented entirely in the ODP packaging and Ambari mpack repositories rather than in the Apache SeaTunnel fork.

These changes are maintained in:

odp-bigtop
ambari-mpacks/seatunnel-2.3.13/

Systemd service lifecycle management

The mpack generates systemd service units for SeaTunnel master and worker components by using the _build_unit_file() method in setup.py.

Service operations, including start, stop, and status, are delegated to SystemdManager from systemd_dbus.

A subprocess fallback is available for Kubernetes pods.

Documentation references:

• 01-overview.md, §2.1
• 05-troubleshooting.md, §§5, 6, 9, and 10

odp-select alias registration

During installation, the mpack registers the following SeaTunnel components through stack_select.select():
stack_select.select("seatunnel-server", version)stack_select.select("seatunnel-client", version)

Documentation reference: 02-usage.md, §3

Log4j2 file-only logging

The mpack configures Log4j2 to write logs to files without using a console appender.

This prevents duplicate log messages from being written to .out files through standard output.

Documentation reference: 05-troubleshooting.md, §11

Bundled connector tarball

The ODP build process packages 11 connectors in the SeaTunnel distribution.

The do-component-build process uses an allowlist to include the required connector dependencies.

For implementation details, see:

odp-bigtop/bigtop-packages/src/common/seatunnel/

Ambari Kerberos integration

The mpack integrates SeaTunnel with Ambari Kerberos configuration through kerberos.json.

This configuration provisions:

• The SeaTunnel keytab.
• The hadoop.proxyuser.seatunnel.* properties.

Documentation reference: 06-security.md, §1

Knox SEATUNNEL role

The mpack includes a Knox SEATUNNEL role through:

• configuration/plugin-config.xml
• The Knox service definition

Documentation reference: 06-security.md, §3

SeaTunnel Web UI quicklink

The mpack provides a single seatunnel_zeta_ui quicklink for accessing the SeaTunnel Web UI.

The quicklink automatically selects the protocol and port based on the enable_https configuration.

Documentation reference: 04-monitoring.md, §2

7. Verify That the Checkout Uses the Latest ODP Commit

Verify that the SeaTunnel source checkout includes the expected ODP-specific commits before rebuilding the distribution.

Step 1: Check the latest commits

Navigate to the SeaTunnel source directory:

cd /path/to/seatunnel

git log --oneline 2.3.13..HEAD | head -5

The expected top-of-log commits are:

If the top-of-log commit does not match the expected commit for the corresponding ODP version, the checkout might be outdated.

Pull the latest changes before rebuilding.

Step 2: Verify the installed SeaTunnel version

Run the following command:

odp-select status seatunnel

3.3.6.6-1004