Upgrade Standalone Kafka

This topic provides steps to start a standalone Kafka cluster in ZooKeeper mode, validate its functionality, and perform a rolling upgrade from Kafka 2.8.2.3.3.6.x-x to Kafka 3.7.2.3.3.6.x-x.

Starting Kafka(2.8.2.3.3.6.x-x) in ZooKeeper Mode

For details about the ODP Standalone Binaries, see Accessing Acceldata Repositories.

1. Start the ZooKeeper Service using the following command.

bin/zookeeper-server-start.sh config/zookeeper.properties

2. Start the Kafka Broker Service. Open another terminal session and run the following command.

bin/kafka-server-start.sh config/server.properties

Once all the services are successfully launched, you have a basic Kafka environment ready for use.

Pre-upgrade Validation

Before proceeding with the upgrade, validate the Kafka setup by performing the following steps:

1. List Topics

bin/kafka-topics.sh --list --bootstrap-server localhost:9092

Example output:

__consumer_offsets

quickstart-events

2. Create a Same Topic

bin/kafka-topics.sh --create --topic Test1 --bootstrap-server localhost:9092

Example output:

Created topic Test1

3. Write Events into the Topic

bin/kafka-console-producer.sh --topic Test1 --bootstrap-server localhost:9092

Input:

>This is my first event

>This is my second event

Terminate with CTRL+C.

4. Read Events from the Topic

bin/kafka-console-consumer.sh --topic Test1 --from-beginning --bootstrap-server localhost:9092

Example output:

This is my first event

This is my second event

Rolling Upgrade Steps

To perform a rolling upgrade from Kafka 2.8.2.3.3.6.x-x to Kafka 3.7.2.3.3.6.x-x, follow these steps:

1. Update server.properties on All Brokers. Edit server.properties and add the following property:

inter.broker.protocol.version=2.8.2.3.3.6.2-1

2. Upgrade Brokers One at a Time. On each broker, perform the following steps:

   1. Shut down the broker.
   2. Update the code by replacing the bin and libs directories with kafka_2.12-3.7.1.3.3.6.x-x.
   3. Restart the broker.

3. Verify Cluster Behavior. Once all brokers are upgraded, ensure that the cluster is functioning correctly by monitoring logs and metrics. Also try to read events from the topic which are written in the above steps:

bin/kafka-console-producer.sh --topic Test1 --bootstrap-server localhost:9092

Example output:

This is my first event

This is my second event

4. Update Protocol Version. Once satisfied with the stability, update server.properties:

inter.broker.protocol.version=3.7.2.3.3.6.2-1

5. Restart Brokers One After the Other: Restart each broker sequentially for the new protocol version to take effect.

For more details, see Upgrading Apache Kafka.