Parallel Course improvements

[mbauman]

Post-mortem from the first running of the online parallel course:

JuliaBox challenges

• Getting DArrays working with JuliaBox on multiple processes is tricky. Need to ensure the package/manifest matches what's in the JuliaBox registry.
• There were some challenges with the Flux-on-GPU demo; I think these all largely stem from using an old version of Flux/CuArrays/Adapt. Also note that the notebooks use the new Flux APIs.
• It'd be nice to provide the multithreaded kernel to everyone by default
• JuliaBox does not demonstrate timing differences for branch predictions. I assume it's spectre/meltdown hardened? This might be a bit of a distraction; not sure if it's worth talking about.

Curriculum content

• We should finish with a parallel JuliaDB demo! (Would likely need to delete some material to make this happen — both times we've completely filled the time)
• Need more power-point slide-like visual aids. We're introducing lots of concepts and visuals help lots. Key things to include:
  • Caches/pipelines and prefetching
  • False sharing (L1 vs L3 cache)
  • Differences between GPU and CPU (die shots)
  • GPU execution sketch (e.g., with some units waiting idle while others take a branch)
• I found the whole section talking about specialization and native code generation a little clunky. Not sure how to improve it, but all the @code_*s are getting in the way of seeing the picture clearly.
• Could use a diagram about how @sync for i in 1:10 @async sleep(1) end works.
• I had wanted to emphasize a bit more about how Tasks are great for UI — like webservers. Forgot to say that.
• findpi_gpu_lazy is broken because we can't broadcast curand.().
• It'd be nice to transform some of the demos into exercises (or add exercises for the students to do) — and give them time to complete them live. Good parallel examples that are meaningful and don't require tons of data are hard.
• diff! actually isn't safe to do in-place in the SIMD notebook.
• use @simd instead of @fastmath in the Julia is fast notebook

Course marketing

We may want to more strongly suggest moderate Julia ability. I found it hard to cater to the wide range of experiences of the class.