The examples in this repository are support to the Spark in Action, 2nd edition book by Jean-Georges Perrin and published by Manning. Find out more about the book on Manning's website.

Spark in Action, 2nd edition - chapter 16

Welcome to Spark in Action, 2e, chapter 16. This chapter is about checkpointing and caching.

This code is designed to work with Apache Spark v3.0.1.

Datasets

Original datasets can be downloaded from:

• Goodreads books by Soumik and coming from Kaggle.
• Brazilian Cities by Christina Parada and coming from Kaggle.

Labs

Each chapter has one or more labs. Labs are examples used for teaching in the book(https://www.manning.com/books/spark-in-action-second-edition?a_aid=jgp). You are encouraged to take ownership of the code and modify it, experiment with it, hence the use of the term lab.

Lab #100

TBD

Lab #110

TBD

Lab #200

TBD

Lab #900

TBD

Running the labs

Java

For information on running the Java labs, see chapter 16 in Spark in Action, 2nd edition.

As lab #110 is designed to work from the command line, you can run it through Maven.

First install:

mvn install

Then execute:

mvn -q exec:exec -Dargs="5"

Where 5 is the number of iterations. More details can be found in chapter 16.

PySpark

TBD

Scala

TBD

---

Thank you for supporting Spark in Action, 2e, please share the <3! Follow and ping me on Twitter to get updates about the book, Apache Spark, and (opinionated) views about software and data engineering: @jgperrin. Join the book's community on Facebook or in Manning's live book and community site.