Hot Path TCG Direct RAM Read Windows

TCG Direct-RAM Read Windows

Lifecycle: Performance - Unproven · Type: Speculative memory shortcut
State: Source-backed; runtime value is not established.

Status: Architecturally possible, but not yet promoted as an xemu shortcut target; no specific translated region has been measured.

Mechanism class: Speculative guarded shortcut / region-level SoftMMU bypass.

Reviewed source: 322986f4c5502b3fa887a9605ac8cd88e9b2c579

Scope: x86 TCG backend and system-mode guest RAM reads.

Summary

Current TCG system-mode loads already have an important fast path. For each guest memory access, generated code performs a per-vCPU software-TLB lookup/compare; a TLB hit produces a direct host address and the generated code performs the load without leaving translated execution. Slow or specially flagged mappings take the helper path.

The proposed shortcut would go one level further for a measured translated region that performs several reads from one stable RAM window: validate the mapping/window once, then emit multiple direct reads without repeating the normal software-TLB sequence for every access.

This is a real bypass concept, but it is not yet a validated xemu-owned target. It changes a mature QEMU memory contract and should not be placed beside renderer shortcuts until profiling and generated-code inspection identify a concrete region where repeated TLB checks are material.

Current TCG flow

flowchart TD
    A[Translated guest load] --> B[Generated software-TLB load and compare]
    B -->|TLB hit, ordinary allowed mapping| C[Compute direct host address]
    C --> D[Generated host load]
    B -->|Miss or flagged mapping| E[Exit to slow helper]
    E --> F[Resolve RAM/MMIO/fault/watch/dirty behavior]
    F --> D

Loading

prepare_host_addr() explicitly prepares the host address for the fast path and records a slow path when required. This means xemu is not currently calling a generic C memory function for every ordinary RAM read.

Proposed region flow

flowchart TD
    A[Enter measured translated read region] --> B{Plan identity and mapping epoch still valid?}
    B -->|No| R[Execute generic translated region]
    B -->|Yes| C{Entire read window is ordinary readable RAM?}
    C -->|No| R
    C -->|Yes| D[Compute/pin host base for bounded region]
    D --> E[Emit multiple direct reads without per-access TLB checks]
    E --> F[Exit before page/mapping/protection boundary]
    R --> F

Loading

The direct route reads current memory values. It must never cache the loaded values themselves.

Why this was not promoted as a normal xemu shortcut

The proposal currently lacks four pieces of evidence:

1. A concrete hot guest PC/translated region. No xemu trace in this review identifies a multi-read block where per-access TLB checks dominate.
2. Generated-code proof. The expected repeated sequence must be visible in the exact Release binary before assuming it can be removed.
3. An xemu-specific ownership seam. This is primarily QEMU TCG/SoftMMU architecture, not a local renderer/device path.
4. A bounded invalidation design. Mapping, permissions, watchpoints, dirty tracking, self-modifying code, MMIO substitution, and snapshot restoration all participate.

Until those exist, the page records a research gate rather than an implementation plan.

Candidate admission contract

A future prototype should start with reads only and require:

• one exact translated region/TB identity;
• one bounded virtual/physical page window;
• ordinary RAM mapping only;
• stable mapping/protection epoch;
• no page crossing beyond the validated span;
• no watchpoint or instrumentation requiring per-access slow behavior;
• no MMIO, ROM behavior, or dynamically rerouted MemoryRegion;
• no write dirty-tracking or self-modifying-code responsibilities;
• deoptimization/fallback when any condition changes.

Writes should remain out of the first prototype because they add dirty-memory, migration/display, code invalidation, and ordering obligations.

Emulation boundary

The guest observes values, faults, permissions, watchpoints, MMIO side effects, and memory ordering. It does not observe the software-TLB lookup itself.

The exact contract is:

Every admitted direct read must return the same value or fault as the normal TCG/SoftMMU route at that guest instruction, and every mapping or instrumentation change must force fallback before the next affected read.

Negative controls

1. Remap the same guest address to different RAM.
2. Replace the page with MMIO.
3. Cross a page boundary inside the candidate region.
4. Revoke read permission and require the same guest fault.
5. Add a watchpoint or debugger instrumentation.
6. Restore a snapshot with different mappings.
7. Trigger TLB flush and TB invalidation.
8. Modify executable guest code in the region.
9. Use an access requiring byte swapping, alignment handling, or another MemOp difference.
10. Verify the fallback executes the original generic region rather than an approximate result.

Evidence required before implementation

exact executable and DWARF identity
hot guest PC / TB identity
number of reads per region execution
software-TLB instructions per access
TLB hit rate
region executions per frame/work unit
sampled or timed cost of the generated lookup sequence
candidate window eligibility rate
mapping/invalidation frequency

Disassembly must show the intended sequence disappeared in the prototype. Function-level sampling alone is insufficient because much of the cost lives in generated guest code.

Counters

direct_window_candidates
direct_window_admitted
direct_window_reject_mapping
direct_window_reject_boundary
direct_window_reject_instrumentation
direct_window_fallbacks
direct_reads_executed
tlb_checks_avoided
window_invalidations
fault_mismatches
value_mismatches

Risks

• Silently bypassing MMIO or MemoryRegion dispatch.
• Missing permission/fault transitions.
• Bypassing watchpoints or debugging instrumentation.
• Stale mapping after TLB flush, restore, or device remap.
• Crossing a page with different attributes.
• Introducing a fork-specific TCG specialization that is difficult to rebase or upstream.
• Entry validation costing as much as the repeated checks it replaces.
• Optimizing a sequence that the host compiler/backend already handles efficiently.

Qualification path

1. Use Release + matching DWARF and guest-PC attribution to identify a target.
2. Inspect generated machine code and count repeated TLB sequences.
3. Build a throwaway read-only prototype for that one region.
4. Run mapping, MMIO, boundary, permission, watchpoint, restore, and SMC negative controls.
5. Compare fixed-work duration and representative titles, not only microbenchmarks.
6. Evaluate whether the design belongs in upstream QEMU rather than a fork-local shortcut.
7. Promote to an xemu shortcut page only after the region, route, and benefit are concrete.

Disposition

Deferred architecture research. It is applicable to the software stack in principle, but the current source review does not justify calling it an actionable xemu shortcut. The existing per-vCPU TLB/direct-host fast path must be acknowledged as the baseline.

Related pages

• CPU & vCPU Execution
• Memory & DMA
• Debugging Lag & Hot-Path Analysis
• Debugging Tools & Commands
• Assumption-Based Shortcut Validation

Full technical reference

Primary reference: This subject page is the design and research reference for Direct-RAM Read Windows. Pull requests are implementation/test history only.
Lifecycle: Performance - Unproven · Mechanism: Speculative multi-access SoftMMU shortcut
Reviewed source: 322986f4c5502b3fa887a9605ac8cd88e9b2c579 · Acceptance class: intended exact or behavior-preserving for its admitted scope

1. Scope

Validate one stable RAM window for a recognized translated region and use direct host loads for several reads instead of repeating software-TLB preparation for each access.

This entry remains Unproven: the source path and opportunity are concrete, but no accepted xemu runtime result has established that the candidate improves representative workloads. The page defines what must be bypassed, what remains mandatory, and how the hypothesis can be rejected.

2. Source seams

File / function	Role
tcg/i386/tcg-target.c.inc	prepare_host_addr() and generated qemu_ld memory paths
target/i386/tcg/translate.c	translated-region structure and exit boundaries

3. Current execution path

flowchart TD
    N0["Each guest read"]
    N1["Software TLB lookup/tag check"]
    N0 --> N1
    N2["Compute host address or helper slow path"]
    N1 --> N2
    N3["Load value"]
    N2 --> N3
    N4["Repeat for nearby reads"]
    N3 --> N4

Loading

4. Candidate execution path

flowchart TD
    N0["Recognized region entry validates mapping epoch/window"]
    N1["Derive host base once"]
    N0 --> N1
    N2["Several bounded direct reads"]
    N1 --> N2
    N3["Exit on boundary/invalidation"]
    N2 --> N3
    N4["Reference path on mismatch"]
    N3 --> N4

Loading

5. Why it may help

Current TCG already has a direct path after each TLB hit, so the candidate must prove repeated lookup sequences are a measurable cost in a specific hot block. It is not justified by source appearance alone. The first prototype should be read-only and tied to one generated region.

6. Work removed

• Repeated software-TLB tag/address preparation within an admitted region
• Some repeated branch/helper setup

7. Work that remains mandatory

• Current memory values
• Fault/permission behavior
• Watchpoints
• MMIO exclusion
• Mapping invalidation
• Reference fallback

8. Implementation sketch

if (plan.mapping_epoch != cpu->mapping_epoch ||
    !plan.covers(guest_base, access_span)) {
    goto generic_region;
}
host_base = plan.host_base + (guest_base - plan.guest_base);
value0 = load_le32(host_base + off0);
value1 = load_le32(host_base + off1);

The sketch defines the intended boundary; it is not a complete patch and does not replace source review, ownership design, or focused tests.

9. Correctness contract

• All accesses stay in the admitted RAM mapping with required permissions/alignment.
• Mapping/protection/watchpoint/debug changes invalidate the plan.
• Fault ordering and partial execution match the reference scope.
• MMIO/ROM/device aliases are rejected.

10. State, ownership, invalidation, and fallback

A plan belongs to a translated block/region and is keyed by mapping epoch plus guest range. Code invalidation, TLB flush/remap, snapshot restore, watchpoint changes, and page protection invalidate or force generic execution.

The maintained reference path remains the fallback whenever admission, capability, lifetime, or validation is uncertain. A fast result with an unproven dependency is not acceptable as an exact optimization.

11. Advantages

• Can remove repeated generated checks in memory-heavy loops
• Leaves values live/current rather than caching contents
• Narrow read-only prototype possible
• Generated-code inspection gives a clear oracle

12. Costs and failure modes

• Large TCG/SoftMMU engineering commitment
• Existing TLB hit path may already be cheap
• Fault/watchpoint semantics are difficult
• Entry validation may erase savings
• Architecture/backend portability

13. Evidence counters

Counters must be cheap, subsystem-owned, and snapshotted outside the hot loop. Record at least:

• hot_block_execution_count
• guest_reads_in_candidate_regions
• TLB_fast_checks_avoided
• slow_path_entries
• entry_validation_failures
• generated_host_instructions
• host samples per block

A local reduction is not enough. Evidence must connect route reachability and removed work to fixed guest progress, CPU/GPU time where applicable, and p95/p99 behavior.

14. Qualification matrix

• Map/remap same virtual address
• Cross page boundary
• Revoke permission
• Watchpoint
• MMIO substitution
• Snapshot restore
• Read-only hot loop
• Generated-code diff
• Fault/exception progress

Correctness runs may use validation layers, detailed comparisons, and fault injection. Performance acceptance must use optimized Release builds with matching symbols and minimal instrumentation.

15. Acceptance rule

Move from Unproven only after a concrete hot guest PC/block shows significant repeated lookup cost and a narrow prototype preserves memory semantics.

16. Rejection rule

Reject if profiling cannot identify such a block, entry checks dominate, or correctness requires recreating the general SoftMMU path.

17. Related architecture and history

CPU-and-vCPU-Execution · Performance-Unproven · Debugging-Lag-and-Hot-Path-Analysis · Memory-and-DMA

PR pages may be linked as supporting evidence, but they are not substitutes for this reference.