Create simulator interface which provides thread scheduling + speculative fetching

[derekbruening]

Rather than having each simulator figure out how to schedule traced software threads onto simulated cores in their own ad hoc way, we would like to provide a scheduler service, which should result in several benefits:

• Ease of use: a new simulator use case has one less thing to implement
• Consistency: all simulators can now use the same approach
• Fill in gaps in trace-based simulation:
  • We can re-schedule threads even when simulating the recorded hardware to deflate context switches increased by tracing overhead
  • We can more easily combine multiple single-workload traces
  • We can provide speculative path fetching using various schemes (from heuristics to additional data recorded during tracing) to help bridge the gap with execution-driven simulation

Xref #5694: provide per-core iterator.
That may become subsumed by this new broader-scope feature.

[derekbruening]

There are many design points here; documenting some smaller ones and will probably put the rest in a separate doc:

Lots of little issues with the scheduler -- here is one: the output streams
have to keep their own record counts (b/c they combine multiple inputs).
Yet the inputs sometimes "hide" records like the synthetic headers after a
skip:

<--record#-> <--instr#->: <---tid---> <record details>
------------------------------------------------------------
           0          63:      296231 <marker: timestamp 13319413770947393>
           0          63:      296231 <marker: tid 296231 on core 10>
          90          64:      296231 ifetch       4 byte\(s\) @ 0x0000000000401028 48 83 eb 01          sub    \$0x0000000000000001 %rbx -> %rbx
          91          65:      296231 ifetch       4 byte\(s\) @ 0x000000000040102c 48 83 fb 00          cmp    %rbx \$0x0000000000000000

The scheduler though does the skip and asks the zipfile reader what the new
record# is so it can update its count and is told 0 so it goes from there
and doesn't see the 90 2 entries later (only queries input on a skip):

<--record#-> <--instr#->: <---tid---> <record details>
------------------------------------------------------------
           0          63:      296231 <marker: timestamp 13319413770947393>
           1          63:      296231 <marker: tid 296231 on core 10>
           2          64:      296231 ifetch       4 byte(s) @ 0x0000000000401028 48 83 eb 01          sub    $0x0000000000000001 %rbx -> %rbx
           3          65:      296231 ifetch       4 byte(s) @ 0x000000000040102c 48 83 fb 00          cmp    %rbx $0x0000000000000000

What is the best solution?

Does it have to query both ordinals before and after every input advance?

Have separate "effective" and "presented" ordinals?

Use the same get_last_record_ordinal() proposed for scheduler-inserted
"doesn't count" cpuid markers w/ ords of 0?

Maybe these inserted records should all be reported as the same prior
ordinal and we add a separate flag "inserted" and the view tool looks for
"inserted" and displays 0 in that case. Or abandon the 0 and leave it
blank or as "--" or sthg? But will that be confusing if the view tool
shows one thing and the direct query shows another? I guess it's the 0
that's confusing: anything else seems compatible with the direct query
showing the prior record ordinal.

I seem to recall a prior discussion where we came up with the 0 and liked
it though, for synthetic records, which include the post-skip headers
above plus the scheduler inserting
cpuid markers for synthetic schedules: we decided those would not interrupt
the original record count.

Decison:

• Add to memtrace_stream_t: is_current_record_synthetic()
• Remove reader_t games where record ordinals are 0 for a few records
• Have get_record_ordinal() return the previous record's ordinal for
  synthetic records not present in the original stream
• Have view_t use the new API to display "--" or some other non-numeric
  indicator for synthetic records