Spark testing articles

Articles tagged with Spark testing. There are 4 article(s) corresponding to the tag Spark testing. If you don't find what you're looking for, please check related tags: access pattern, Ad-hoc polymorphism, Akka Distributed Data, Akka examples, algorithm analysis, algorithm complexity, Apache Beam configuration, Apache Beam internals, Apache Beam partitioning, Apache Beam PCollection.

Check out my new course on Data Engineering!

Are you a data scientist who wants to extend his data engineering skills? Or a software engineer who wants to work with Big Data? If not, maybe a BI developer who wants to evolve to engineering position? My course will help you to achieve your goal! Join the class →

ScalaTest and testing styles

At first glance the wide choice of testing families in Scala can scary. After all in JUnit and other xUnit frameworks, the choice of tests declaration is limited. Hopefully after some digging ScalaTest's testing styles become more obvious to understand and to use. Continue Reading →

Testing Spark applications

It's difficult to contest the importance of testing in programming. Tests help to avoid regressions (a lot of regressions) and also to better understand developed code. Spark (and other data processing frameworks by the way) is not an exception of this rule. But, obviously, testing applications working in distributed mode is more tricky than in the case of standalone programs. Continue Reading →