Context-based learning:
You've joined a news company. The company publishes news on a website and is just in the beginning of their data journey.
So far they've been relying on batch processing to generate insight. They have been using tools like Apache Spark SQL, Apache Airflow, an object store, and a data warehouse.
However, the project requires near real-time processing capabilities in many places. You're the one who will lead this batch-to-streaming transformation!
Your goal is to go through the course and solve each homework exercise with the elements learned so far. By the end of the course your system will become streaming-first and you'll take a few months off followed by a raise, as promised by your Head of Data Engineering 🙂
Have you written your first streaming pipeline and think you know it all?
Demos and homework exercises implemented with Scala and Python.
Join the waiting list 📨