Apache Kafka dockerization is less complicated that in the case of Cassandra (take a look at post about Dockerize Cassandra troubleshooting). But even of that, there are some thinks to know, globally of the same type as in the case of Cassandra.
New ebook 🔥
Learn 84 ways to solve common data engineering problems with cloud services.
👉 I want my copy
This post won't follow usual organization in sections. It's because Kafka containerization concerns mostly network issues - and more specifically advertised.listeners property (by supposing that we want to keep ZooKeeper private to container).
The advertised.listeners property defines hostname and port exposed to producers and consumers. By default this value is null. It doesn't make any problem when Kafka is executed in standalone mode, as process of client's machine. The 9092 port is then reachable. The problem occurs when they're not on the same server. For such case, one of solutions is to change advertised.listeners in a way that it exposes broker's IP:
DOCKER_HOST="$(hostname --ip-address)" echo "advertised.listeners=PLAINTEXT://$DOCKER_HOST:9092" >> $kafkaConfig
Beside of that, in configured broker I didn't want to allow automatic topic creation neither block topic deletion. It's the reason why below properties were defined as simply as previous one:
echo "delete.topic.enable=true" >> $kafkaConfig echo "auto.create.topics.enable=false" >> $kafkaConfig
This short post describes some prerequisites to containerize Kafka. It explains which properties need to be configured to make broker available for client application as well some custom entries helping to control better topics manipulation.
