I'm the author of Data Engineering Design Patterns (O'Reilly),
a Databricks MVP, and
a freelance data engineer specializing in Apache Spark and Databricks.
I help teams move from working pipelines to resilient architectures.
I'm currently accepting new projects for Jun 2026. Whether you need a 2-day architectural audit, a hands-on lead for a
complex data engineering problem, or a workshop
let's discuss your project here.
One of the easy solutions is to create a side container that will start after the broker and create the relevant topics. You can express the dependencies with depends_on and healthcheck operations: ...
Over the years Windows was not the good candidate to work with Docker. But since the release of Docker for Windows, you can work with Docker containers on Windows much easier. However, it's not always...
It's simple. You should use the service_completed_successfully condition, as in the snippet below: version: '3.1' services: data_generator: image: waitingforcode/data-generator-blogging-platf...
