Spark SQL code generation articles

The why of code generation in Apache Spark SQL

By the end of 2018 I published a post about code generation in Apache Spark SQL where I answered the questions about who, when, how and what. But I omitted the "why" and cozos created an issue on my Github to complete the article. Something I will try to do here.

Continue Reading →

The who, when, how and what of Apache Spark SQL code generation

The code generated by Apache Spark for all the queries defined with higher-level concepts as SQL queries is the key to understand the processing logic performance. This post, started after a discussion on my Github, tries to explain some of the basics of code generation workflow.

Continue Reading →

Generated code in Spark SQL

One of powerful features of Spark SQL is dynamic generation of code. Several different layers are generated and this post explains some of them.

Continue Reading →