And it shouldn't be, right? After all, it's "just" about using a Unit Test framework and defining the test cases. Well, that's "just" a theory!
The "Data is the new Oil" is one of popular sentences describing the huge role of data in our world. And as other resources, data must be extracted too. To find these "Oil workers", organizations look for, among others, data engineers. The task is more or less easier and this difficulty depends on various factors. From my 6-years perspective, one of the key starting elements is the job announcement.
Yesterday I shared with you the human part of my Data+AI Summit. It's time now to give you my takeaways from the technical talks.
There will be many "first times" in our lives. For me, the Data+AI Summit 2022 was the first time I've visited the USA, put the 3D dimensions to the pictures of my virtual friends and felt a huge community support in a very troubled moment. Besides, I also enjoyed the talks and walking, even though the latter one wasn't so good for my skin ;)
More and more often in my daily contact with the data world I hear this word "modern". And I couldn't get it. I was doing cloud data engineering with Apache Spark/Apache Beam, so it wasn't modern at all? No idea while I'm writing this introduction. But I hope to know more about this term by the end of the article!
Recently you've discovered the building blocks of a feature store. This time I would like to demonstrate a feature store in action. I've chosen Feast as a playground use case because it's Open Source, has good working examples, and implements an Apache Spark ingestion job!
Feature stores are a more and more common topic in the data landscape and they have been in my backlog for several months already. Finally, I ended up writing the blog post and I really appreciated the learning experience! Even though it's a blog post from a data engineer perspective, so maybe without a deep data science deep dive.
I wrote this blog post a week before passing the GCP Data Engineer exam, hoping it'll help to organize a few things in my head (it did!). I also hope that it'll help you too in understanding ML from a data engineering perspective!
After introducing DataOps concepts, it's a good time to share my feelings on them ?
"DataOps", this term is present in my backlog since a while already and I postponed it multiple times. But I finally found some time to learn more about it and share my thoughts with you.
Few weeks ago I got a comment asking me about the recommended data engineering books. I mentioned few of them in Becoming a data engineer - a feedback of my journey blog post but without explaining why. I will try to complete that in this blog post then.
Recently a reader asked me in a PM about the things to know and to learn before starting to work as a data engineer. Since I think that my point of view may be interesting for more than 1 person (if not, I'm really sorry), I decided to write a few words about it.
