Checkpoint allows Spark to truncate dependencies on previously computed RDDs. In the case of streams processing their role is extended. In additional, they're not a single method to prevent against failures.
Get new posts, recommended reading and other exclusive information every week. SPAM free - no 3rd party ads, only the information about waitingforcode! Curious about the content ? Check some of already sent newsletters
You want to learn data engineering but have no idea where and how to start in this wide domain? Check if Become a Data Engineer can help you 💪