Waiting for code

on waitingforcode.com

Syntactic sugar in Scala

When a programming language provides operator overloading, the learning curve increases most of the time because of the syntactic sugar it brings. After all more of operations will be expressed as not meaningful (at least in first approach) symbols. Scala also comes with its own syntactic sugar that can be applied in a lot of places: sequences, functions or conversion. Continue Reading →

Sacks - data parallelization unit in Gnocchi

To facilitate parallel processing Apache Spark and Apache Kafka have their concept of partitions, Apache Beam works with bundles and Gnocchi deals with sacks. Despite the different naming, the sacks are the same for Gnocchi as the partitions for Spark or Kafka - the unit of work parallelization. Continue Reading →

Higher-kinded types in Scala

Types and type-safety in Scala have a special privileged place. But this wide range of techniques to deal with them makes the language discovery more difficult. And at first glance one of difficult type-related concepts are higher-kinded types, covered in this post. Continue Reading →

What Kubernetes can bring to Apache Spark pipelines ?

Commercial version of Apache Spark distributed by Databricks offers a serverless and auto-scalable approach for the applications written in this framework. Among the time some other companies tried to provide similar alternatives, going even to put Apache Spark pipelines into AWS Lambda functions. But with the version 2.3.0 another alternative appears as a solution for scalability and elasticity overhead - Kubernetes. Continue Reading →

Carbonara storage format

Even though carbonara is mostly known as an Italian pasta dish, in the context of Gnocchi it means completely different thing. Carbonara is the name of time points storage format in Gnocchi. Continue Reading →

Horizontal scalability in Gnocchi

One of the reasons behind the choice of Gnocchi as time series database to study was its naturally provided horizontal scalability. At the moment of making that choice I was relying only on the official documentation. Now it's a good moment to come back and analyze the horizontal scalability by myself. Continue Reading →