A 3-day bug hunt on a 3-person team costs up to β¬7,200 in lost engineering time. This workshop teaches you to prevent that β unit tests, data tests, and integration tests for PySpark and Databricks Lakeflow, including Spark Declarative Pipelines.
As a former Java engineer, when I have started to work with Scala, I was very early punished for my bad habits about mutable collections. In this post I will show you the story which learned me to prefer the immutability everywhere it's possible.
Scala Stream offers something we have not a habit to see in other languages - lazy computation of the values alongside the memoization. However it's sometimes misleading and some people think about Streams as about iterators, i.e. a data structure computing and forgetting about the results. Such thinking can often lead to memory problems, especially with infinite streams.
Nowadays a lot of SDKs and libraries are written with Java. Fortunately dealing with Java code in Scala is possible and even in case of collections.
Some time ago I covered in this blog the complexity of Scala immutable collections, explaining a little what happened under-the-hood. Now it's a good moment to back to this topic and apply it for mutable collections.
Using the right tool at the right time is important in every domain. It's particularly useful in the case of collections that can have different complexity for writing and reading. And very often this difference can dictate the use of one specific implementation.
