Spark memory articles

on waitingforcode.com
Articles tagged with Spark memory. There are 5 article(s) corresponding to the tag Spark memory. If you don't find what you're looking for, please check related tags: access pattern, Ad-hoc polymorphism, Akka Distributed Data, Akka examples, algorithm analysis, algorithm complexity, Apache Beam configuration, Apache Beam internals, Apache Beam partitioning, Apache Beam PCollection.

Memory and Apache Spark classes

In previous posts about memory in Apache Spark, I've been exploring memory behavior of Apache Spark when the input files are much bigger than the allocated memory. After that it's a good moment to sum up that in the post dedicated to classes involved in memory using tasks. Continue Reading →

Apache Spark and off-heap memory

With data-intensive applications as the streaming ones, bad memory management can add long pauses for GC. Luckily, we can reduce this impact by writing memory-optimized code and using the storage outside the heap called off-heap. Continue Reading →

Memory management in Spark

Memory management in Spark went through some changes. In the first versions, the allocation had a fix size. Only the 1.6 release changed it to more dynamic behavior. This change will be the main topic of the post. Continue Reading →