[DNM] Enable SMP on RISC-V

[nategraff-sifive]

Do not merge. Opening a PR to seek feedback and comments.

A series of changes to enable symmetric multiprocessing on riscv32. I based my work partly on the way that SMP was enabled for Xtensa, including the use of the new _arch_switch API in place of swap. Thanks @andyross for the ground work to make this possible in Zephyr.

There are few areas that I'm concerned about:

Scheduler Changes

During development I found that I needed to make a number of changes to the common scheduler code. Most of these revolved around the fact that SMP removes the currently running threads from the ready queue, and in several places the scheduler made the incorrect assumption that threads were in the ready queue when they were not. This usually manifests in the ready queue being corrupted (threads getting dropped) and/or the scheduler handing back a bad thread pointer and making the CPU jump to nowhere.

Odds are I've made some mistakes, or at least not found the extent of the problems. Especially looking for review here.

Testing in QEMU

I've created a sifive-multicore target board for testing purposes. It's basically a clone of HiFive 1 with the SMP flags turned on. QEMU 3.0.0 doesn't have an embedded multicore RISC-V target, so in order to try this out you'll need to patch QEMU. It's a very simple one-line change. In hw/riscv/sifive_e.c, just change line 203 to

    mc->max_cpus = <some number greater than 1>;

I haven't gotten sanitycheck passing on sifive-multicore yet either. Still tracking down bugs.

Saved Registers During Interrupts

The multicore interrupt handler I created saves both callee and caller-saved registers onto the stack immediately, as opposed to the single-core handler which only saves callee-saved registers when it knows that a reschedule is about to occur. I didn't see a particularly good way to optimize this, but I wanted to point it out and see what people thought.

[codecov-io]

Codecov Report

Merging #11354 into master will increase coverage by <.01%.
The diff coverage is 71.42%.

@@            Coverage Diff             @@
##           master   #11354      +/-   ##
==========================================
+ Coverage   48.37%   48.37%   +<.01%     
==========================================
  Files         265      265              
  Lines       42188    42196       +8     
  Branches    10137    10143       +6     
==========================================
+ Hits        20408    20412       +4     
  Misses      17703    17703              
- Partials     4077     4081       +4

Impacted Files	Coverage Δ
kernel/sched.c	91.36% <71.42%> (-1.13%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 39b2a09...29c5ab7. Read the comment docs.