Acceldata

Last updated: Oct 16, 2026 04:27 UTC

Connector Reference: Validated Examples

All 10 connectors were validated on the ODP 3.3.6.6-1004 lab environment, which included three pods, Zeta cluster mode, Kerberos (ADSRE.COM), and Ranger. Each validated example includes output captured from an actual lab run.

The configuration templates are available in docs/examples/.

To submit a job, run the following command:

sudo -u seatunnel bash -c '
  kinit -kt /etc/security/keytabs/seatunnel.service.keytab seatunnel/$(hostname -f)@<REALM>
  source /etc/seatunnel/conf/seatunnel-env.sh
  /usr/odp/current/seatunnel/bin/seatunnel.sh --config /path/to/job.conf
'

Validation status

Status

Description

Validated

The job ran successfully in cluster mode on the ODP lab, and the output was captured.

Requires

A prerequisite must be met before running the job.

ACL/Policy

An authorization step is required, such as configuring Kafka ACLs, HDFS ACLs, or Hive permissions.

Doc-only

A configuration template is available, but the connector was not validated in the lab.

1. Fake to Console

Status: Validated

Use this example as a baseline smoke test to verify a new installation. This test does not require external dependencies.

Configuration

File: examples/fake-to-console.conf

env {
  parallelism = 1
  job.mode = "BATCH"
}
source {
  FakeSource {
    result_table_name = "src"
    row.num = 5
    schema = {
      fields {
        id = "int"
        name = "string"
      }
    }
  }
}
sink { Console { source_table_name = "src" } }

Validation results

The job ran on seatunnel-0 in Zeta local mode and completed in approximately 2 seconds.

Total Read Count   : 5
Total Write Count  : 5
Total Failed Count : 0
Job status         : FINISHED

2. Fake to Kafka

Status: Validated

Prerequisites

  • Ensure that the Kafka broker is accessible through the SASL_PLAINTEXT port. This ODP environment uses port 6669, not 6667.
  • Create the Kafka topic before running the job, or grant the seatunnel user the CREATE ACL.

Configure Kafka ACLs

This cluster uses the native AclAuthorizer rather than RangerKafkaAuthorizer.

Run the following commands as a Kafka superuser:

BOOTSTRAP=<broker>:6669

# Run as a Kafka superuser.
# See 06-security.md, Kafka admin JAAS section.

kafka-acls.sh --bootstrap-server $BOOTSTRAP --command-config /tmp/kafka_client.properties \
    --add --allow-principal User:seatunnel --producer --topic st_smoke

kafka-acls.sh --bootstrap-server $BOOTSTRAP --command-config /tmp/kafka_client.properties \
    --add --allow-principal User:seatunnel --operation IdempotentWrite --cluster

Configuration

File: examples/fake-to-kafka.conf

env { parallelism = 1; job.mode = "BATCH" }
source {
  FakeSource {
    result_table_name = "src"
    row.num = 10
    schema = { fields { id = "int"; name = "string"; ts = "timestamp" } }
  }
}
sink {
  Kafka {
    source_table_name = "src"
    bootstrap.servers = "seatunnel-1.<domain>:6669"
    topic = "st_smoke"
    format = "json"
    kafka.config = {
      security.protocol = "SASL_PLAINTEXT"
      sasl.mechanism = "GSSAPI"
      sasl.kerberos.service.name = "kafka"
      # On SeaTunnel Zeta with the ODP mpack, omit sasl.jaas.config.
      # The mpack-deployed /etc/seatunnel/conf/seatunnel-jaas.conf
      # provides a per-host KafkaClient entry, and the systemd unit
      # passes -Djava.security.auth.login.config= to the JVM.
      #
      # Set inline sasl.jaas.config only for Flink/Spark YARN engines
      # or non-ODP deployments.
      # See 06-security.md, sections 5.1 and 5.1a.
    }
  }
}

Submit the job

Run the following command:

sudo -u seatunnel /usr/odp/current/seatunnel/bin/seatunnel.sh \
 --config fake-to-kafka.conf -m cluster -n kafka_smoke

Validation results

The job successfully produced 10 rows, which were verified in the Kafka topic.

Total Read Count=10, Total Write Count=10, Total Failed Count=0, FINISHED
# kafka-get-offsets.sh --topic st_smoke
st_smoke:0:0
st_smoke:1:10
st_smoke:2:3

# kafka-console-consumer sample
{"id":1729004799,"name":"sVdnx","ts":"2026-04-22T03:41:06"}
{"id":446002914,"name":"YFdzQ","ts":"2026-12-16T17:06:00"}
{"id":1546010146,"name":"vuIHK","ts":"2026-04-26T11:47:43"}

3. Fake to HDFS (file-hadoop)

Status: Validated

Prerequisites

Identify the HDFS high-availability (HA) nameservice before running the job.

The lab environment uses ODP-Titan.

To identify the nameservice, check the dfs.nameservices property in:

/etc/hadoop/conf/hdfs-site.xml

Configure HDFS permissions

Run the following commands:

sudo -u hdfs kinit -kt /etc/security/keytabs/hdfs.headless.keytab hdfs-<realm>
sudo -u hdfs hdfs dfs -mkdir -p /tmp/seatunnel-smoke
sudo -u hdfs hdfs dfs -chown seatunnel:hadoop /tmp/seatunnel-smoke

Configuration

File: examples/fake-to-hdfs.conf

env { parallelism = 3; job.mode = "BATCH" }
source {
  FakeSource {
    result_table_name = "src"
    row.num = 200
    schema = { fields { id = "int"; name = "string"; ts = "timestamp" } }
  }
}
sink {
  HdfsFile {
    source_table_name = "src"
    fs.defaultFS = "hdfs://ODP-Titan"               # nameservice, not host
    path = "/tmp/seatunnel-smoke/hdfs_cluster_mode"
    file_format_type = "json"
    kerberos_principal = "seatunnel/_HOST@ADSRE.COM"   # ODP _HOST-substitution patch required
    kerberos_keytab_path = "/etc/security/keytabs/seatunnel.service.keytab"
    hdfs_site_path = "/etc/hadoop/conf/hdfs-site.xml"
  }
}

Validation results

The job successfully wrote 600 rows across three output files.

The total row count was calculated as:

parallelism × row.num = 3 × 200 = 600
Total Read Count=600, Total Write Count=600, Total Failed Count=0
/tmp/seatunnel-smoke/hdfs_cluster_mode/T_1157831614978326552_69dc634e29_1_1_0.json
/tmp/seatunnel-smoke/hdfs_cluster_mode/T_1157831614978326552_84af672769_0_1_0.json
/tmp/seatunnel-smoke/hdfs_cluster_mode/T_1157831614978326552_de22811b78_2_1_0.json
{"id":240185971,"name":"GVavl","ts":"2026-12-25T10:25:05"}

4. Fake to Kudu

Status: Validated

Prerequisites

Complete the following prerequisites before running the job.

1. Distribute the Kudu client JAR file.

Place kudu-client-*-all.jar at the same path on every Zeta cluster member.

The bundled connector-kudu-*.jar does not include the Kudu protobuf-generated org.apache.kudu.Common$* enum classes. Therefore, copy the ODP-provided Kudu client JAR to:

/usr/odp/current/seatunnel/plugins/kudu/lib/

SeaTunnel uses a distributed classpath model. If the JAR file exists only on the submission host, the job fails with the following error:

ClassLoaderException: [NOT-FOUND-JAR] - The jar file
file:/usr/odp/current/seatunnel/plugins/kudu/lib/kudu-client-*-all.jar
can not be found in node <other-host>, please ensure that the
deployment paths of SeaTunnel on different nodes are consistent.

For more information about the connector packaging, see section F-13 in 10-odp343-lab-findings.md.

2. Create the Kudu table.

Create the target table before running the job. The SeaTunnel Kudu sink does not automatically create tables.

3. Configure Ranger permissions, if applicable.

If the Ranger Kudu plugin is enabled, configure a policy that grants the seatunnel user access to the target tables.

Without the required policy, kudu-master.INFO logs the following error:

User seatunnel is not authorized to access METADATA on <table>
NonRecoverableException: Unauthorized action
{
 "service": "<cluster>_kudu",
 "name": "seatunnel - kudu st_tables",
 "resources": {
 "database": {
 "values": ["default"]
 },
 "table": {
 "values": ["st_*"]
 },
 "column": {
 "values": ["*"]
 }
 },
 "policyItems": [
 {
 "users": ["seatunnel"],
 "accesses": [
 {"type": "all"},
 {"type": "insert"},
 {"type": "metadata"},
 {"type": "create"},
 {"type": "select"}
 ]
 }
 ]
}

Distribute the Kudu client JAR file

Run the following commands on every host that runs seatunnel-master or seatunnel-worker:

# Select the ODP-provided JAR from a host that runs KUDU_MASTER.
SRC=/usr/odp/current/kudu/jars/kudu-client-1.18.1.3.4.3.0-1015-all.jar

# Repeat on every host that runs seatunnel-master or seatunnel-worker.
sudo install -o seatunnel -g hadoop -m 0755 -d /usr/odp/current/seatunnel/plugins/kudu/lib

sudo install -o seatunnel -g hadoop -m 0644 "$SRC" \
 /usr/odp/current/seatunnel/plugins/kudu/lib/

Create the Kudu table

Run the following commands:

sudo -u kudu kinit -kt /etc/security/keytabs/kudu.keytab kudu/<fqdn>@<REALM>

JSON='{"table_name":"st_smoke","schema":{"columns":[{"column_name":"id","column_type":"INT64","is_nullable":false},{"column_name":"val","column_type":"STRING","is_nullable":true}],"key_column_names":["id"]},"partition":{"hash_partitions":[{"columns":["id"],"num_buckets":2}]},"num_replicas":1}'

sudo -u kudu /usr/odp/current/kudu/bin/kudu table create <kudu-master>:7051 "$JSON"

Configuration

File: examples/fake-to-kudu.conf

env { parallelism = 2; job.mode = "BATCH" }
source {
  FakeSource {
    result_table_name = "src"
    row.num = 30
    schema = { fields { id = "bigint"; val = "string" } }
  }
}
sink {
  Kudu {
    source_table_name = "src"
    kudu_masters = "seatunnel-2.<domain>:7051"
    table_name = "st_smoke"
    save_mode = "APPEND"
    enable_kerberos = true
    kerberos_principal = "seatunnel/_HOST@ADSRE.COM"
    # Kudu uses kerberos_keytab, not the _path suffix.
    kerberos_keytab = "/etc/security/keytabs/seatunnel.service.keytab"
  }
}

Validation results

The job successfully wrote 60 rows, which were verified by using the Kudu CLI.

Total Read Count=60, Total Write Count=60, Total Failed Count=0
$ kudu table scan <master> st_smoke
(int64 id=111156334176727096, string val="mvoca")
(int64 id=538649398984367255, string val="mxsMT")
(int64 id=1719746177831400681, string val="MoIno")
...

5. MySQL to Console (JDBC source)

Status: Validated

Prerequisites

  • Ensure that the MySQL server is accessible.
  • Place the JDBC driver at the same path on every SeaTunnel worker.
  • Ensure that the driver filename is identical across all pods.
  • Restart seatunnel-master and seatunnel-worker after adding the driver.

SeaTunnel uses a distributed classpath model. If the JDBC driver filenames differ across pods, the job fails with a NOT-FOUND-JAR error.

Install the JDBC driver

Run the following commands on every SeaTunnel pod:

mkdir -p /usr/odp/current/seatunnel/plugins/jdbc/lib

cp mysql-connector-j-8.0.32.jar /usr/odp/current/seatunnel/plugins/jdbc/lib/

chown seatunnel:hadoop /usr/odp/current/seatunnel/plugins/jdbc/lib/*.jar

Configuration

File: examples/mysql-to-console.conf

env { parallelism = 2; job.mode = "BATCH" }
source {
  Jdbc {
    result_table_name = "orders"
    url = "jdbc:mysql://mysql-host:3306/st_test?useSSL=false&allowPublicKeyRetrieval=true"
    driver = "com.mysql.cj.jdbc.Driver"
    user = "st_user"
    password = "<pwd>"
    query = "SELECT id, customer, amount, ts FROM orders"
    partition_column = "id"
    partition_num = 2
  }
}
sink { Console { source_table_name = "orders" } }

Validation results

The job successfully read five rows from st_test.orders and wrote them to the console in approximately 2 seconds.

6. Fake to HBase

Status: Validated with the round-2 patch

Prerequisites

Use the ODP connector-hbase-*-2.12.18.jar built from the round-2 ODP branch.

The round-2 patch adds the following functionality:

  • kerberos_principal and kerberos_keytab_path configuration options.
  • UGI.loginUserFromKeytabAndReturnUGI for Kerberos authentication.
  • ugi.doAs(ConnectionFactory.createConnection) for authenticated HBase connections.

The upstream HBase connector does not include Kerberos login functionality.

Configure HBase permissions

Run the following commands:

sudo -u hbase hbase shell <<EOF
grant 'seatunnel', 'RWXCA', '@default'
create 'st_smoke2', 'cf'
EOF

Configuration

File: examples/fake-to-hbase.conf

env { parallelism = 2; job.mode = "BATCH" }
source {
  FakeSource {
    result_table_name = "src"
    row.num = 50
    schema = { fields { id = "string"; name = "string"; score = "int" } }
  }
}
sink {
  Hbase {
    source_table_name = "src"
    zookeeper_quorum = "seatunnel-0.<domain>:2181,seatunnel-1.<domain>:2181,seatunnel-2.<domain>:2181"
    table = "st_smoke2"
    rowkey_column = ["id"]
    family_name = { name = "cf", score = "cf" }
    kerberos_principal = "seatunnel/_HOST@ADSRE.COM"
    kerberos_keytab_path = "/etc/security/keytabs/seatunnel.service.keytab"
    hbase_extra_config = {
      "zookeeper.znode.parent"                = "/hbase-secure"
      "hbase.security.authentication"         = "kerberos"
      "hbase.master.kerberos.principal"       = "hbase/_HOST@ADSRE.COM"
      "hbase.regionserver.kerberos.principal" = "hbase/_HOST@ADSRE.COM"
      "hbase.rpc.protection"                  = "authentication"
    }
  }
}

Validation results

The job successfully wrote 100 rows in cluster mode.

The total row count was calculated as:

parallelism × row.num = 2 × 50 = 100
Total Read Count=100, Total Write Count=100, Total Failed Count=0
hbase> count 'st_smoke2'
100 row(s)

7. Fake to Hive

Status: Validated by using the SEATUNNEL_CLASSPATH configuration

Prerequisites

Complete the following prerequisites before running the job.

1. Configure the SeaTunnel classpath.

Ensure that SEATUNNEL_CLASSPATH in /etc/seatunnel/conf/seatunnel-env.sh includes:

/usr/odp/current/hive-client/lib/*

This path provides the required Hive dependencies, including:

  • hive-exec
  • hive-metastore
  • libthrift
  • libfb303

The ODP mpack configuration template handles the classpath configuration.

The ODP launcher scripts, seatunnel.sh and seatunnel-cluster.sh, append ${SEATUNNEL_CLASSPATH} to the Java -cp argument.

2. Create an external Hive table.

Create the target table by using CREATE EXTERNAL TABLE.

ODP Hive 4.1 uses ACID transactional format by default for managed tables. These tables do not recognize the plain ORC files written by SeaTunnel.

As a result, the data might be written to HDFS, but the following query returns zero rows for managed tables:

SELECT COUNT(*)

Set the following properties in core-site.xml:

hadoop.proxyuser.hive.hosts=*
hadoop.proxyuser.hive.groups=*

4. Configure HDFS permissions.

Create an HDFS Ranger policy or ACL that grants both the seatunnel and hive users write access to the target table location.

Create the external Hive table

Run the following SQL statements:

DROP TABLE IF EXISTS default.st_hive_ext;

CREATE EXTERNAL TABLE default.st_hive_ext (
  id INT, name STRING, ts STRING
) STORED AS ORC
LOCATION 'hdfs://ODP-Titan/tmp/seatunnel-smoke/hive_ext';

Configuration

File: examples/fake-to-hive.conf

env { parallelism = 2; job.mode = "BATCH" }
source {
  FakeSource {
    result_table_name = "src"
    row.num = 40
    schema = { fields { id = "int"; name = "string"; ts = "string" } }
  }
}
sink {
  Hive {
    source_table_name = "src"
    table_name = "default.st_hive_ext"
    metastore_uri = "thrift://seatunnel-1.<domain>:9083,thrift://seatunnel-2.<domain>:9083"
    kerberos_principal = "seatunnel/_HOST@ADSRE.COM"
    kerberos_keytab_path = "/etc/security/keytabs/seatunnel.service.keytab"
    hdfs_site_path = "/etc/hadoop/conf/hdfs-site.xml"
    hive_site_path = "/etc/hive/conf/hive-site.xml"
  }
}

Validation results

The job successfully wrote 80 ORC rows to the external table location.

Total Read Count=80, Total Write Count=80, Total Failed Count=0
$ hdfs dfs -ls /tmp/seatunnel-smoke/hive_ext
-rw-r--r-- 1042 B T_1157853878146826334_05370340c1_1_1_0.orc
-rw-r--r-- 1042 B T_1157853878146826334_560d444fd2_0_1_0.orc

8. Fake to Hudi

Status: Validated with the round-3 CatalogFactory patch

Prerequisites

Use the ODP connector-hudi-*-2.12.18.jar built from the round-3 ODP branch.

The round-3 patch adds the following functionality:

  • KERBEROS_PRINCIPAL, KERBEROS_KEYTAB_PATH, and KRB5_CONF_PATH configuration options.
  • Kerberos initialization through HudiCatalogFactory before opening the catalog.
  • Authentication context handling by wrapping HudiRecordWriter.executeWrite/Delete operations in HudiUtil.doAsLoginUser.

Configuration

File: examples/fake-to-hudi.conf

env { parallelism = 2; job.mode = "BATCH" }
source {
  FakeSource {
    result_table_name = "src"
    row.num = 50
    schema = { fields { id = "int"; name = "string"; ts = "timestamp" } }
  }
}
sink {
  Hudi {
    source_table_name = "src"
    table_dfs_path = "hdfs://ODP-Titan/warehouse/hudi"
    # conf_files_path accepts semicolon-separated file paths, not a directory.
    conf_files_path = "/etc/hadoop/conf/core-site.xml;/etc/hadoop/conf/hdfs-site.xml"
    kerberos_principal = "seatunnel/_HOST@ADSRE.COM"
    kerberos_keytab_path = "/etc/security/keytabs/seatunnel.service.keytab"
    krb5_conf_path = "/etc/krb5.conf"
    table_list = [
      {
        table_name = "st_hudi"
        database = "default"
        table_type = "COPY_ON_WRITE"
        record_key_fields = "id"
        op_type = "INSERT"
        index_type = "INMEMORY"
      }
    ]
  }
}

Validation results

The job successfully wrote 100 rows and created the Hudi .hoodie metadata structure.

Total Read Count=100, Total Write Count=100, Total Failed Count=0
/warehouse/hudi/default/st_hudi/.hoodie
/warehouse/hudi/default/st_hudi/.hoodie/.aux/.bootstrap/.fileids
/warehouse/hudi/default/st_hudi/.hoodie/.aux/.bootstrap/.partitions
/warehouse/hudi/default/st_hudi/.hoodie/.schema

9. Fake to Iceberg (HadoopCatalog)

Status: Validated with the round-3 Proxy patch

Prerequisites

Complete the following prerequisites before running the job.

1. Use the patched Iceberg connector.

Use the ODP connector-iceberg-*-2.12.18.jar built from the round-3 ODP branch.

The patch wraps the Iceberg source in a Java Proxy so that every catalog method executes under ugi.doAs(...).

The patch requires:

  • IcebergCatalogLoader to retain the UserGroupInformation (UGI) context.
  • IcebergCatalog.open() to wrap the underlying catalog in a proxy.

2. Configure Kerberos authentication.

Specify all three Kerberos configuration properties:

  • kerberos_principal
  • kerberos_keytab_path
  • krb5_path

The IcebergCommonConfig.doKerberosLogin() method requires all three properties.

If any property is missing, the connector logs the following warning:

Kerberos authentication is not configured, it will skip
AccessControlException: SIMPLE authentication is not enabled

Set iceberg.hadoop-conf-path to:

/etc/hadoop/conf

This configuration allows Iceberg to load core-site.xml and hdfs-site.xml, which are required to resolve the HDFS HA nameservice.

Configuration

File: examples/fake-to-iceberg.conf

env { parallelism = 1; job.mode = "BATCH" }
source {
  FakeSource {
    result_table_name = "src"
    row.num = 50
    schema = { fields { id = "int"; name = "string"; ts = "timestamp" } }
  }
}
sink {
  Iceberg {
    source_table_name = "src"
    catalog_name = "hadoop_prod"
    iceberg.catalog.config = {
      "type"      = "hadoop"
      "warehouse" = "hdfs://ODP-Titan/warehouse/iceberg"
    }
    iceberg.hadoop-conf-path = "/etc/hadoop/conf"
    namespace = "seatunnel_smoke"
    table = "st_iceberg_sink"
    kerberos_principal   = "seatunnel/_HOST@ADSRE.COM"
    kerberos_keytab_path = "/etc/security/keytabs/seatunnel.service.keytab"
    krb5_path            = "/etc/krb5.conf"
  }
}

Validation results

The job successfully wrote 50 rows and created the Iceberg table structure, including data files, manifests, snapshots, and metadata.

Total Read Count=50, Total Write Count=50, Total Failed Count=0
/warehouse/iceberg/seatunnel_smoke/st_iceberg_sink/data/
    00001-1790824433672-6b67fcff-65e9-449b-b1f1-a6d35f88dcb8-00001.parquet (1786 B)

/warehouse/iceberg/seatunnel_smoke/st_iceberg_sink/metadata/
    09507e33-b38f-4a94-892e-9b23ba09a1a6-m0.avro (manifest, 7164 B)
    snap-7471656282221798844-1-09507e33-...a1a6.avro (snap, 4471 B)
    v1.metadata.json (814 B)
    v2.metadata.json (1733 B)
    version-hint.text (1 B)

10. MySQL-CDC to Kafka

Status: Validated

Prerequisites

Complete the following prerequisites before running the job.

1. Enable MySQL binary logging.

Configure the following MySQL properties:

log_bin=ON
binlog_format=ROW
binlog_row_image=FULL
gtid_mode=ON
enforce_gtid_consistency=ON

Run the following SQL statements:

CREATE USER 'cdc_user'@'%' IDENTIFIED BY '<pwd>';
GRANT REPLICATION SLAVE, REPLICATION CLIENT, SELECT, RELOAD, SHOW DATABASES
ON *.* TO 'cdc_user'@'%';

Ensure that the following JAR files are available in /usr/odp/current/seatunnel/connectors/ on every pod:

  • connector-cdc-mysql-*-2.12.18.jar
  • connector-cdc-base-*.jar

4. Configure streaming mode.

Set job.mode to STREAMING.

The MySQL-CDC source does not support BATCH mode.

5. Configure Kafka permissions.

Create the target Kafka topic and grant the required ACLs to the seatunnel user, as described in section 2.

Configuration

File: examples/mysql-cdc-to-kafka.conf

env {
  parallelism = 1
  job.mode = "STREAMING"
  checkpoint.interval = 2000
}
source {
  MySQL-CDC {
    result_table_name = "orders"
    hostname = "seatunnel-1.<domain>"
    port = 3306
    username = "st_user"
    password = "<pwd>"
    database-names = ["st_test"]
    table-names    = ["st_test.orders"]
    startup.mode = "initial"           # also "latest" | "earliest" | "specific"
    snapshot.split.size = 1024
    url      = "jdbc:mysql://seatunnel-1.<domain>:3306/st_test?useSSL=false&allowPublicKeyRetrieval=true"
    base-url = "jdbc:mysql://seatunnel-1.<domain>:3306?useSSL=false&allowPublicKeyRetrieval=true"
  }
}
sink {
  Kafka {
    source_table_name = "orders"
    bootstrap.servers = "seatunnel-1.<domain>:6669"
    topic = "cdc.orders"
    format = "json"           # or "compatible_debezium_json" for Debezium compatibility
    kafka.config = {
      security.protocol = "SASL_PLAINTEXT"
      sasl.mechanism = "GSSAPI"
      sasl.kerberos.service.name = "kafka"
      # On SeaTunnel Zeta with the ODP mpack, omit sasl.jaas.config.
      # Global JAAS configuration handles authentication.
      #
      # For Flink/Spark YARN engines or non-ODP deployments,
      # use inline sasl.jaas.config with a headless principal.
      # Kafka's Krb5LoginModule does not substitute _HOST.
      #
      # sasl.jaas.config = "com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab=\"/etc/security/keytabs/seatunnel.headless.keytab\" principal=\"seatunnel@<REALM>\" serviceName=\"kafka\";"
      #
      # See 06-security.md, section 5.1.
    }
  }
}

Validation results

The job successfully captured an initial snapshot of five rows and three subsequent inserts for frank, grace, and heidi.

All eight records were written to Kafka as JSON payloads.

$ mysql st_test> INSERT INTO orders (customer, amount) VALUES ("frank",300), ("grace",450.5), ("heidi",777.77);

$ kafka-get-offsets.sh --topic cdc.orders
cdc.orders:0:8 # 5 snapshot + 3 incremental

$ kafka-console-consumer --from-beginning --max-messages 8
{"id":1,"customer":"alice","amount":100.5,"ts":"..."}
{"id":2,"customer":"bob","amount":200.75,"ts":"..."}
{"id":3,"customer":"carol","amount":50,"ts":"..."}
{"id":4,"customer":"dave","amount":999.99,"ts":"..."}
{"id":5,"customer":"eve","amount":15.25,"ts":"..."}
{"id":6,"customer":"frank","amount":300,"ts":"2026-10-01T02:41:38"}
{"id":7,"customer":"grace","amount":450.5,"ts":"2026-10-01T02:41:38"}
{"id":8,"customer":"heidi","amount":777.77,"ts":"2026-10-01T02:41:38"}

Startup modes

Mode

Description

initial

Captures a complete snapshot of the source tables and then automatically transitions to reading the binary log.

latest

Skips the initial snapshot and starts reading new binary log events.

earliest

Replays all available binary log events.

specific

Starts from a specified binary log position by using startup.specific-offset.file and startup.specific-offset.pos.

JDBC driver placement

The ODP distribution includes the JDBC connector and the following CDC connectors:

  • connector-cdc-mysql
  • connector-cdc-postgres
  • connector-cdc-sqlserver
  • connector-cdc-oracle

However, the distribution does not include the JDBC drivers.

JDBC drivers must be installed separately because licensing requirements and driver versions vary across environments.

Place the drivers in the following directory on every SeaTunnel pod:

/usr/odp/current/seatunnel/plugins/jdbc/lib/

Important: SeaTunnel uses a distributed classpath model that sends the exact JAR file path from the submission host to every worker.

If the driver filename differs across pods, the job might fail with a NOT-FOUND-JAR error.

For example, if one pod contains mysql-connector-java.jar and another contains mysql-connector-j-8.0.32.jar, the job might fail.

Use the same filename on every pod. For more information, see section 9 in 05-troubleshooting.md.

Supported JDBC drivers

Database

Driver artifact

Download

MySQL

com.mysql:mysql-connector-j:8.0.32

Download

PostgreSQL

org.postgresql:postgresql:42.7.3

Download

SQL Server

com.microsoft.sqlserver:mssql-jdbc:12.6.1.jre11

Download

Oracle

com.oracle.database.jdbc:ojdbc11:23.5.0.24.07

Oracle Technology Network (account required)

ClickHouse

com.clickhouse:clickhouse-jdbc:0.6.0:all

Download

Deploy the JDBC driver

Complete the following steps on every SeaTunnel pod.

  1. Navigate to the JDBC plugin directory:

  2. cd /usr/odp/current/seatunnel/plugins/jdbc/lib/

  3. Remove alternative MySQL driver filenames and symbolic links:
find . -type f -name 'mysql-connector-*' ! -name 'mysql-connector-j-8.0.32.jar' -delete
  1. find . -type l -delete

  2. Download the MySQL JDBC driver:

  3. curl -sLO https://repo1.maven.org/maven2/com/mysql/mysql-connector-j/8.0.32/mysql-connector-j-8.0.32.jar

  4. Set the ownership and permissions:
chown seatunnel:hadoop ./mysql-connector-j-8.0.32.jar
  1. chmod 644 ./mysql-connector-j-8.0.32.jar

  2. Restart the SeaTunnel worker:

  3. systemctl restart seatunnel-worker

  4. Restart the SeaTunnel masters sequentially by using the Ambari API script:
    docs/examples/rolling_restart.sh

Note: MySQL-CDC, PostgreSQL-CDC, SQL Server-CDC, and Oracle-CDC use the JDBC connector's driver-loading mechanism. Therefore, the same driver filename consistency requirement applies to all these connectors.

11. ClickHouse sink

Status: Doc-only

The ClickHouse sink was not deployed or validated in this lab environment.

The following configuration template is provided for reference.

Configuration

sink {
  Clickhouse {
    source_table_name = "src"
    host = "clickhouse-host:8123"
    database = "default"; table = "st_smoke"
    username = "seatunnel"; password = "***"
    bulk_size = 20000
  }
}

12. MongoDB-CDC source

Status: Doc-only

The MongoDB-CDC source was not deployed or validated in this lab environment.

The configuration requires:

  • A MongoDB replica set with an oplog.
  • readAnyDatabase permissions.
  • Access to local.oplog.rs.

The configuration template is based on the upstream SeaTunnel documentation.

Engine-support matrix

The following matrix describes connector capabilities and the directions validated in the ODP lab.

Connector capability identifies whether the connector supports source operations, sink operations, or both. This information is based on the classes available in the connector implementation under:

seatunnel-connectors-v2/<connector>/src/main/java/.../{source,sink}/

Validated direction identifies the direction tested end to end in the ODP lab.

A connector listed as validated only for sink operations might also support source operations. The validation status indicates what was tested, not necessarily the complete capability of the connector.

Connector

Capability

Validated direction

Zeta

Flink 1.19

Spark 3.5

Spark 4.1

FakeSource / Console

Source / Sink

Source → Sink

✅

✅

✅

✅

JDBC

Source + Sink

Source → Sink

✅

✅

✅

✅

Kafka

Source + Sink

Source → Sink

✅

✅

✅

✅

HdfsFile (file-hadoop)

Source + Sink

Sink (Fake → HDFS)

✅

✅

✅

✅

Hive

Source + Sink

Sink (Fake → Hive)

✅

✅

✅

⚠️

HBase

Source + Sink

Sink (Fake → HBase)

✅

✅

✅

⚠️

Kudu

Source + Sink

Sink (Fake → Kudu)

✅

✅

✅

⚠️

Iceberg (HadoopCatalog)

Source + Sink

Sink (Fake → Iceberg)

✅

✅

✅

✅

Hudi (COW)

Sink only

Sink (Fake → Hudi)

⚠️

⚠️

✅

✅

MySQL-CDC

Source only

Source (MySQL-CDC → Kafka)

✅

✅

❌

❌

MongoDB-CDC

Source only

Not validated (doc-only)

✅

✅

❌

❌

ClickHouse

Source + Sink (two sink implementations)

Not validated (doc-only)

✅

✅

✅

✅

Connector-specific notes

Connector

Notes

FakeSource / Console

Provides a baseline smoke test across supported engines.

JDBC

Requires JDBC driver installation and consistent driver filenames across pods.

Kafka

On Zeta with the ODP mpack, omit sasl.jaas.config because the global JAAS configuration handles authentication. On Flink/Spark YARN, use inline sasl.jaas.config with a headless seatunnel@REALM principal instead of _HOST. See section 5.1 in 06-security.md.

HdfsFile

Requires the ODP _HOST substitution patch.

Hive

Requires an external Hive table and the SEATUNNEL_CLASSPATH configuration.

HBase

Requires the round-2 ODP patch. The source connector is included but was not validated in the lab.

Kudu

Requires the target table to be created before job execution. The source connector is included but was not validated in the lab.

Iceberg

Requires the round-3 Proxy patch. The source connector is included but was not validated in the lab.

Hudi

The upstream SeaTunnel 2.3.x release removed source support. Copy-on-write (COW) works on Zeta, while merge-on-read (MOR) requires Spark.

MySQL-CDC

Supports streaming mode only. Spark is unsupported for both batch and streaming operations. Upstream disables CDC on Spark in MysqlCDCIT.java, lines 28–31.

MongoDB-CDC

The same Spark restriction applies to all CDC connectors, including MySQL, PostgreSQL, SQL Server, Oracle, OpenGauss, MongoDB, and TiDB.

ClickHouse

Includes two sink implementations: Clickhouse (JDBC) and ClickhouseFile (parts upload). The source connector is included but was not validated in the lab.

Validation limitations

Not validated (doc-only) indicates that a configuration template is available in docs/examples/, but no end-to-end validation was captured on the ODP 3.3.6.6-1004 lab environment.

This status does not mean that the connector is unsupported or cannot function. It indicates only that the connector was not exercised as part of the lab validation.

For engine-specific launcher commands, see section 3.2 in 02-usage.md.