ScalaTest extra features

Versions: ScalaTest 3.0.3

ScalaTest, as xUnit family testing frameworks, provides a lot of features. Even though several of them are not frequently used, it's always good to know them and the context of their use.

Data Engineering Design Patterns

Looking for a book that defines and solves most common data engineering problems? I'm currently writing one on that topic and the first chapters are already available in πŸ‘‰ Early Release on the O'Reilly platform

I also help solve your data engineering problems πŸ‘‰ contact@waitingforcode.com πŸ“©

This post is divided in 5 parts. Each of them presents one interesting ScalaTest features. The first described one is tagging. It's followed by fixtures, property-based tests, given/when/then mixin and the pending test markers.

Tagging

The role of tags in ScalaTest is similar to their role in xUnit testing frameworks, i.e. the tags are used to define the specificities of test cases. An example of such specificities can be the character (integration vs unit) of the test, its criticality or the reason of adding (e.g. detected after a regression). The tags are the classes extending org.scalatest.Tag class.

Test cases can be annotated with one or several tags. The tagging method depends on the test family (see more in the post about ScalaTest and testing styles). The following code shows a simple implementation for FlatSpec and FunSpec:

object ImportantTest extends Tag("ImportantTest")

class TaggingTest extends FlatSpec {

  "the flat spec test" should "be tagged" taggedAs(ImportantTest) in {
  }

}

class TaggingFunSpecTest extends FunSpec {
  it("the fun spec test should be also tagged as important", ImportantTest) {
  }
  it("the fun spec test should not be tagged as important") {
  }
}

Another writing style uses the annotations to mark whole test class with given tag:

 
import com.waitingforcode.annotations.ImportantTest

@ImportantTest
class TaggingTest extends FlatSpec {

  "the flat spec test" should "be tagged" in {
  }

}

Fixtures

Very often test cases need to share a specific feature, as for instance an in-memory database simulating distributed data store, sockets connections or test configurations. ScalaTest provides a solution to share them throughout the fixtures. They can be introduced with one of these approaches:

Property-based tests

The most common tests declaration method specifies one tested object per use case. It's the first way to ensure that the code meets the functional expectations. But a complementary approach exists and it's called property-based tests. In such tests instead of defining 1 single tested value, we either define a set of the values or let the test framework to generate them in our place. Thanks to that we can check how the implementation behaves on a much wider variety of options.

In ScalaTest such tests can be declared with the use of org.scalatest.prop.PropertyChecks mixin, directly in the test definition. It can be used for both previously quoted declaration types (explicit and implicit), as shown in the following code:

class PropertyTests extends PropSpec  with Matchers with PropertyChecks {

  property("tuples conversion should be made correctly") {
    forAll { (firstName: String, lastName: String, bornYear: Int) => {
        val currentYear = 2018
        whenever(bornYear < currentYear) {
          val person = Converter.convertTupleToPerson(firstName, lastName, bornYear, currentYear)

          // As you can notice, the logic from the converter is repeated here
          person.firstName shouldEqual firstName
          person.lastName shouldEqual lastName
          person.age shouldEqual (currentYear - bornYear)
        }
      }
    }
  }

  private val PropertyTable = Table(
    "test dataset",
    ("a", "b", 2000, 18),
    ("a", "b", 1990, 28),
    ("a", "b", 1980, 38)
  )

  property("converting dataset from property table") {
    val referenceYear = 2018
    forAll(PropertyTable) { testTuple => {
        val (firstName, lastName, bornYear, expectedAge) = testTuple

        val person = Converter.convertTupleToPerson(firstName, lastName, bornYear, referenceYear)

        person.firstName shouldEqual firstName
        person.lastName shouldEqual lastName
        person.age shouldEqual expectedAge
      }
    }
  }

}


object Converter {

  def convertTupleToPerson(firstName: String, lastName: String, bornYear: Int, referenceYear: Int): Person = {
    val age = referenceYear - bornYear
    Person(firstName, lastName, age)
  }

}

case class Person(firstName: String, lastName: String, age: Int)

In the first test case the data is generated automatically by built-in generators. But we can also generate more specific objects by implementing the Gen[+T] trait. You can also notice the use of a whenever method controlling to the input parameters we want to check. Maybe one of the most important drawbacks of the automatic property checks is the logic duplication. As you can see in the presented (pretty simple though) case, we duplicate the logic of computing the age in the assertion person.age shouldEqual (currentYear - bornYear) and in the implementation val age = referenceYear - bornYear.

For the case of the second test there is much less magic since all tested data is defined in an object.

Given/when/then mixin

Sometimes it's not obvious to transparently structure the test cases with given/when/then style. ScalaTest provides a way to define these boundaries explicitly with the help of org.scalatest.GivenWhenThen mixin:

class GivenThenMixinTest extends FlatSpec with GivenWhenThen with Matchers {

  case class Numbers(nr1: Int, nr2: Int, nr3: Int)

  "a case class" should "be copied with changed properties" in {
    Given("a case class")
    val numbers = Numbers(1, 2, 3)

    When("the case class is copied with changed element")
    val changedNumbers = numbers.copy(nr3 = 30)

    Then("copied case class should have one attribute changed")
    changedNumbers.nr1 shouldEqual 1
    changedNumbers.nr2 shouldEqual 2
    changedNumbers.nr3 shouldEqual 30
  }

}

And it outputs:

Given a case class
When the case class is copied with changed element
Then copied case class should have one attribute changed

However, since the text in Given/When/Then methods is printed, sometimes it can reduce the readability of the test results in the CI reports.

Pending tests

Another ScalaTest's feature concerns pending tests. A pending test is the test that we know the existence but we don't know the implementation, for example because it was specified before as a kind of functional guideline.

Pending tests are different than ignored tests from 2 points of view: philosophical and technical. Ignored tests are the tests that already exist but that, because of some reason, e.g. quickfix or significant refactoring, must be ignored temporary and so inevitably fixed later. The pending tests appear even before the functional logic and thus are written as a kind of specification. For the technical difference ignored tests are not executed at all while pending ones are executed until reaching the TestPendingException thrown by org.scalatest.Assertions#pending() method:

class PendingTestFunSuite extends FunSuite {

  test("a new document should be persisted in the local storage")(pending)


  test("a new document should be persisted in the local storage with some initial implementation") {
    println("Before calling pending method...")

    pending
  }

  ignore("this test should be ignored") {
    println("Before calling ignored test case...")
  }
}

The snippet above outputs:

Test Pending
Before calling pending method...

Test Pending

Test Ignored

Of course, the way of defining tests as pending varies with the used family and for FlatSpec it will be:

class PendingTestFlatSpec extends FlatSpec {

  "a new document" should "be persisted in the local storage" in (pending)

}

ScalaTest brings several facility methods helping to define the tests easier and make them more maintainable. We can use custom tags to classify test executions or one of available fixtures methods to avoid the common setup and cleaning code duplication. We can also go further and test our code against a set of dynamically constructed values. The BDD also has its points in the extended features with given/when/then mixin and the pending tag letting to define the acceptance tests before the implementation.