Underscore role in Scala

Versions: Scala 2.12.1

Scala has a lot of syntactic sugar. Despite the fact that most of times it pretty simply understandable, it occurs often that it increases the learning curve. One of symbols widely used in Scala and hard to understand at first contact is the underscore.

Data Engineering Design Patterns

Looking for a book that defines and solves most common data engineering problems? I wrote one on that topic! You can read it online on the O'Reilly platform, or get a print copy on Amazon.

I also help solve your data engineering problems 👉 contact@waitingforcode.com 📩

In this new post about Scala features we'll focus on the underscore. Despite the fact that it's hard to categorize its use, this post is divided in only 3 parts. The first section shows underscore's use in ignoring things. The second section explains its role in the conversions. The final one is devoted to miscellaneous use cases grouped in a common list. Each of the sections contains a short explanation and some learning tests.

Ignoring things

The underscore can be used to ignore things as: not used variables or not used types. Let's start by showing how to ignore the former ones. For instance, if we split some text and we want to get only the first of so constructed entries, we could write the code as it:

it("should ignore not used variables") {
  val text = "a,b"
  val Array(firstLetter, _) = text.split(",")

  firstLetter shouldEqual "a"
}

Second ignore use case can be hiding not used parameters in function execution, as for example:

it("should ignore parameter in the mapping function") {
    val numbers = 1 to 10

    val numberSeq = numbers.map(_ => "Number")

    numberSeq should have size 10
    numberSeq should contain only "Number"
}

Regarding to the previous point, sometimes we may want to use the anonymized parameters in the functions (aka placeholders). It means that we don't assign them a name, somehow like in Java's anonymous classes. With that we gain some "place" but in the other side we make the code less explicit. Thus the parameters anonymization should be used really only in the case of short operations (first case below) and not in the case of long and chained constructions (second case below) that are harder to understand:

it("should anonymize parameter in a short function") {
    val lettersUpperCase = Seq("A", "B", "C")

    val lettersLowerCase = lettersUpperCase.map(_.toLowerCase())

    lettersLowerCase should have size 3
    lettersLowerCase should contain allOf("a", "b", "c")
}
it("should anonymize parameter in a lot of chained functions") {
    val lettersWithNumbers = Seq(("A", 1, true), ("B", 2, true), ("C", 3, true), ("D", 4, false))

    val lettersEvenAndTrue = lettersWithNumbers
      .filter(_._2%2 == 0)
      .filter(_._3)
      .map(_._1)

    lettersEvenAndTrue should have size 1
    lettersEvenAndTrue should contain only "B"
}

The underscore can also be employed to ignore the types in pattern matching. For instance, if we're only interested on matching given type without taking its value into account, we can simply write:

it("should ignore type in the pattern matching") {
    val lettersSet = Set("A", "B", "C")

    val collectionType = lettersSet match {
      case _: Set[_] => "set"
      case _: List[_] => "list"
      case _ => "unknown"
    }

    collectionType shouldEqual "set"
}

Among these all ignoring purposes we can also mention the use of underscore with hidden imports:

import java.util.{HashSet => _, _}

Above will import all members of java.util package except the HashSet.

Conversions

The underscore can also be used in conversion. The first conversion concerns varargs that can be constructed from a sequence with seqName: _* expression:

it("should construct varargs from sequence") {
  val letters = Seq("A", "B", "C", "D")
  def concatenateLetters(letters: String*): String = {
    letters.mkString(",")
  }

  val concatenatedLetters = concatenateLetters(letters: _*)

  concatenatedLetters shouldEqual "A,B,C,D"
}

Another conversion use case concerns methods. Scala's method can be transformed to a function (aka Eta expansion of method into method value) with the help of an underscore symbol:

it("should convert method to a function") {
  def concatenateLetters(letter1: String, letter2: String): String = s"$letter1,$letter2"
  val concatenationFunction = concatenateLetters _

  val aAndBConcatenated = concatenationFunction("A", "B")

  aAndBConcatenated shouldEqual "A,B"
}

The underscore can be also helpful to define partially applied functions in Scala:

it("should be used in partially applied functions") {
  def add(number1: Int, number2: Int, number3: Int): Int = {
    number1 + number2 + number3
  }
  val addTo1: (Int, Int) => Int = add(_, _, 1)

  val add1To1To1Result = addTo1(1, 1)

  add1To1To1Result shouldEqual 3
}

Miscellaneous

This part covers miscellaneous uses of the underscore, listed below for better readability:

• universality - the underscore can be used as universal substitution (we can perceive is also as ignore though), for instance, in the wildcard imports:

      import org.scalatest._
      

  
  In the other side, it can also be found in the pattern matching to catch not matched cases as here for "everything else":

      it("should be used in pattern matching") {
        val text: Any = "a, b, c, d"
  
        val matchedType = text match {
          case nrInt:Int => "int"
          case nrLong:Long => "long"
          case _ => "everything else"
        }
  
        matchedType shouldEqual "everything else"
      }
      

  
  Or to bind the whole matched sequence independently on the type to some variable:

      it("should bind the whole matching sequence to a variable") {
        val letters = Seq("A", "B", "C")
  
        val lowerCasedLetters = letters match {
          case Seq(matchedStrings @ _*) => matchedStrings.map(symbol => symbol.toLowerCase())
          case _ => Seq.empty
        }
  
        lowerCasedLetters should have size 3
        lowerCasedLetters should contain allOf("a", "b", "c")
      }
      

• setters overriding - Scala classes setters are automatically created with public variables. However sometimes we may need to override it in order to add some extra logic. We can do it by suffixing the method named the same as the attribute with _:

      it("should override the setter") {
        class Letter {
          var isInitialized = false
          private var localValue = ""
  
          def value=localValue
          def value_=(newValue: String) {
            isInitialized = true
            localValue = newValue
          }
        }
  
        val a = new Letter()
        a.value = "a"
  
        a.isInitialized shouldBe true
        a.value shouldEqual "a"
      }
      

• types - the underscore can be used to define higher-kinded and existential types. But they will be explained in one of later posts. Now it's enough to know that the higher-kinded type is the way to define a type abstracting over some type that, in its own turn, abstracts over other type. Simply speaking it's Scala's ability to generalize across type constructors:

      it("should construct higher-kinded types") {
        // AnyObjectContainer parameterized by a first-order type, thus it becomes a higher-kinded type
        trait AnyObjectContainer[A[_]] {
          def checkIfEmpty[T](collection: A[T]): Boolean
        }
        object SeqObjectContainer extends AnyObjectContainer[Seq] {
          override def checkIfEmpty[T](collection: Seq[T]): Boolean = !collection.nonEmpty
        }
  
        val seqIsEmpty = SeqObjectContainer.checkIfEmpty(Seq("A"))
  
        seqIsEmpty should be false
      }
      

  Type constructor

  A type in Scala is often denoted as *. The type constructor is then written as * -> * and it means that given the first type, the second one is created. An example of that can be a List that alone is a type (*). But the List accepts type parameters that represent the stored elements. When we supply the type argument to the List (e.g. String), we create completely new type (List[String]), so we deal with type constructor.

  
  The existential type also will be explained further. Here it's only good to know that the existential types help to not specify the type involved in given operation because it's not needed in the context. They can be expressed either with forSome block or with Scala's wildcards syntactic sugar. To take an example, List[List[_]] is the syntactic sugar's version for existential type expressed as List[List[T] forSome { type T }].
• self types - underscore is also related to the self types. They'll be explained more in details in one of further posts. By now we can only tell that the self types help to declare that given trait must be mixed into another trait without any direct inheritance declaration:

  it("should reassign this in self type") {
      trait Programmer {
        def languages: String
      }
      trait TeamLead {
        _: Programmer =>
        def showPreferences: String = s"TeamLead preferences are ${languages}"
      }
      // because of self typed TeamLead, ScalaTeamLead must be mixed with Programmer
      class ScalaTeamLead extends TeamLead with Programmer {
        override def languages: String = "Scala"
      }
      val scalaTeamLead = new ScalaTeamLead()
  
      val scalaTeamLeadPreferences = scalaTeamLead.showPreferences
  
      scalaTeamLeadPreferences shouldEqual "TeamLead preferences are Scala"
  }
  

• defaults - the underscore can be used to initialize a var to the type's default value:

        var isTrue: Boolean = _
        describe("initialization") {
          it("should set the type to its default value") {
  
            isTrue shouldBe false
          }
        }
      

• tuples and getters - the underscore mixed with a number representing n-part of the tuple and be used as the accessor to the nth value of this tuple:

it("should be retrieved with _2") { val letters = ("A", "B", "C") val secondLetter = letters._2 secondLetter shouldEqual "B" }
• identifiers naming - the identifiers (method, variable names) can contain both alphanumerical and not alphanumerical symbols. If both of them are used in given name, they must be separated by an underscore:

      it("should allow to use underscore to separate alphanumerics from symbols on identifiers") {
        val hello_! = "Hello !"
  
        hello_! shouldEqual "Hello !"
      }
      

The underscore symbol and Scala is a long story. As shown in this post, it's used in a lot of places for different purposes. The most of them are reserved to ignoring stuff (imports, types, cases in pattern matching). Beside it, the underscore is also employed in conversions to varargs or partially applied function. The last part shown some remaining use cases of the underscore, such as: in the type system, custom setters or universality.

Consulting

With nearly 16 years of experience, including 8 as data engineer, I offer expert consulting to design and optimize scalable data solutions. As an O’Reilly author, Data+AI Summit speaker, and blogger, I bring cutting-edge insights to modernize infrastructure, build robust pipelines, and drive data-driven decision-making. Let's transform your data challenges into opportunities—reach out to elevate your data engineering game today!

👉 contact@waitingforcode.com
🔗 past projects