Kafka Connect 3
Setup Kafka Connect from Ambari
When installing Kafka, ensure to select the appropriate nodes for installing the Kafka broker and connect services.
Kafka Connect Setup with Kerberos, SSL, and Ranger
Since Kafka is already configured with SSL, Kerberos, and Ranger, the underlying topics maintain these settings during Kafka connect operations. Below are the configurations for the Advanced kafka3-connect-distributed section in Kafka3's CONFIGS.
x
bootstrap.servers=<broker1-host>:6669,<broker2-host>:6669,<brokern-host>:6669config.storage.replication.factor=3config.storage.topic=connect-configsgroup.id=connect-clusterkey.converter=org.apache.kafka.connect.json.JsonConverterkey.converter.schemas.enable=trueoffset.flush.interval.ms=10000offset.storage.replication.factor=3offset.storage.topic=connect-offsetsplugin.path=/usr/odp/current/kafka3-broker/libs/*listeners=localhost:8083rest.advertised.listener=HTTPSstatus.storage.replication.factor=3status.storage.topic=connect-statussun.security.krb5.debug=truevalue.converter=org.apache.kafka.connect.json.JsonConvertervalue.converter.schemas.enable=truesasl.enabled.mechanisms=GSSAPIsasl.kerberos.service.name=kafkasasl.mechanism=GSSAPIsasl.mechanism.inter.broker.protocol=GSSAPIsecurity.inter.broker.protocol=SASL_SSLsecurity.protocol=SASL_SSLssl.key.password=<password>ssl.keystore.location=/opt/security/pki/keystore.jksssl.keystore.password=<password>ssl.truststore.location=/opt/security/pki/truststore.jksssl.truststore.password=<password>java.security.auth.login.config=/usr/odp/current/kafka-broker/config/kafka_jaas.confjava.security.krb5.kdc=<KDC server hostname>java.security.krb5.realm=<KDC server realm name>javax.security.auth.useSubjectCredsOnly=trueconsumer.bootstrap.servers=<broker1-host>:6669,<broker2-host>:6669,<brokern-host>:6669consumer.security.protocol=SASL_SSLconsumer.ssl.truststore.location=/opt/security/pki/truststore.jksconsumer.ssl.truststore.password=<password>producer.bootstrap.servers=<broker1-host>:6669,<broker2-host>:6669,<brokern-host>:6669producer.security.protocol=SASL_SSLproducer.ssl.truststore.location=/opt/security/pki/truststore.jksAfter configuring the above settings, restart the Kafka service (including broker and Connect). Once the restart is completed, it will start to operate in Kerberos and SSL mode.
Kafka Connector Use Cases
Acceldata ODP Kafka Connect offers support for the following connectors, which have been included in this release:
The following table describes the supported connector type and library versions:
| Connector Name | Supported Type | Library Version |
|---|---|---|
| Pubsub | Source & Sink | 1.2.0 |
| MongoDB | Source & Sink | 1.6.1 |
| JDBC | Source & Sink | 6.1.2 |
| S3 | Sink Only | 2.6.0 |
| Kinesis | Sink Only | 0.0.8 |
Was this page helpful?