Skip to content

HyperRAM FIFO and Caching Strategies

Jump to bottom
sy2002 edited this page Jul 11, 2026 · 2 revisions

This article assumes you have read HyperRAM for Beginners. That guide handed you a toolbox — avm_fifo to cross a clock, avm_cache to hide latency, an arbiter to share the chip — and showed, at a survey altitude, how the two reference cores bolt them together. Good enough to wire up a working core. Not good enough to design one.

This is the article you read next, when you want to build something the survey did not cover: a client with an unusual access pattern, a cache that has to freeze its consumer, a FIFO you have to size by hand. It opens the two tools up. A FIFO and a cache are the two classic answers to two completely different problems, and most HyperRAM bugs come from reaching for the wrong one, sizing it wrong, or placing it wrong. So we build each from first principles — what is actually inside, and why — and then dissect the real M2M implementations: three asynchronous FIFOs (the framework's avm_fifo, the vendor macro beneath it, and the controller's hand-rolled receive FIFO) and two contrasting caches (avm_cache's streaming line, and the C64 cartridge's fully-associative crt_cacher).

A FIFO decouples two clocks in time; a cache decouples fast logic from slow memory using locality. The beginners guide told you which block to drop in; this article shows you how each one works inside, why M2M built them the way it did, and how you would design your own.

Part 1 — Two problems, two tools

Before any mechanism, get the map straight, because the single most common design mistake is solving one of these problems with the tool built for the other.

1.1 Problem A: rates and clocks that do not line up

Your core runs on its own clock — the C64 core near 31.5 MHz, the Amiga near 28.375 MHz — and HyperRAM lives in the framework's 100 MHz hr_clk domain. Two clocks, unrelated. But even if the two frequencies were identical, you would still have a problem: a producer that emits data in clumps and a consumer that takes it on its own schedule never line up cycle-for-cycle, and behind a shared arbiter (beginners guide §5.3) the consumer — HyperRAM — vanishes for long stretches while the video scaler has the bus.

What you need is a component that lets the two sides run at their own pace without dropping a word and without forcing them into lockstep. That component is a FIFO (Part 2).

1.2 Problem B: memory that is far and slow

HyperRAM is off-chip, slow to first word (~7 to ~11 clocks; beginners guide §2.3), and shared three ways. Touching it for every access is ruinous. But real access patterns are not random — they revisit and cluster addresses, a property called locality (defined in Part 4). What you need is a component that keeps a small, fast, on-chip copy of the hot data and touches the slow chip only when it must. That component is a cache (Part 4 onward).

1.3 They are orthogonal — and usually stacked

The two tools do not overlap, and neither substitutes for the other:

  • A FIFO does not reduce latency. It carries a request across a clock boundary and carries the answer back; the slow round trip is exactly as slow as before. It only decouples timing.
  • A cache does not cross clocks. It answers a hit (the data is already in its fast copy) in its own clock domain and, on a miss (the data is absent), hands the traffic downstream. It only hides latency. (Hit and miss are defined properly in Part 4.)

That is why the canonical stack (beginners guide §6.2) uses both, in a fixed order — cache first to kill the traffic, FIFO second to cross the clock:

flowchart LR
  L["your logic"] --> A["protocol adapter<br/>native → Avalon"]
  A --> C["avm_cache<br/>hide latency"]
  C --> F["avm_fifo<br/>cross the clock"]
  F --> AR["arbiter"]
  AR --> HR["hr_core_*"]
Loading

Part 2 and Part 3 open up the FIFO half; Part 4 through Part 6 open up the cache half; Part 7 helps you choose, size, and place them.

Part 2 — FIFOs from first principles

A FIFO — First In, First Out — is a hardware queue. It is the most under-appreciated block in the toolbox, because it looks trivial (a buffer!) and is quietly subtle (crossing clocks correctly is genuinely hard). This part builds it up so that when you open avm_fifo in Part 3, nothing is mysterious.

2.1 The queue: producer, consumer, and two pointers

Picture a small block of memory used as a ring buffer and two pointers into it: a write pointer where the producer pushes the next item, and a read pointer where the consumer pops the oldest item. Push advances the write pointer; pop advances the read pointer; both wrap around the end of the memory back to the start. Because the consumer always takes the oldest item first, order is preserved end to end — first in, first out.

   ┌────┬────┬────┬────┬────┬────┬────┬────┐
   │ D0 │ D1 │ D2 │ D3 │    │    │    │    │   depth = 8 slots, fill level = 4
   └────┴────┴────┴────┴────┴────┴────┴────┘
     ▲ read ptr                    ▲ write ptr
       oldest, next to pop           first free, next to push

   Pop order is D0, D1, D2, D3 — first in, first out. It is a ring: when the
   write ptr runs off the right-hand end it simply wraps back to slot 0.

2.2 Full, empty, fill level, and backpressure

Two pointers give you everything:

  • Empty: read pointer == write pointer — nothing to pop. The consumer must wait.
  • Full: the write pointer has caught up to the read pointer from behind — no room to push. The producer must wait.
  • Fill level: how many items are currently queued (the gap between the pointers). This is the number a designer actually cares about — it is the live measure of how much slack is left.

"The producer must wait" and "the consumer must wait" are backpressure: the FIFO pushing back on whichever side is running ahead. In hardware this is a ready/valid handshake, the same two-wire contract Avalon dresses up as waitrequest (beginners guide §3.3):

  • valid — the producer says "I have an item this cycle."
  • ready — the consumer says "I can accept one this cycle."
  • A transfer happens only on a clock edge where both are high. Either side can stall the other simply by dropping its wire.

A FIFO is just this handshake on both ends with storage in between: it raises ready to its producer until it is full, and raises valid to its consumer until it is empty.

2.3 Depth is a design decision, not a default

The depth is how many items the FIFO can hold. It is not a number you copy from an example — it is the amount of slack you are buying, and it should be sized to the three things that make producer and consumer drift apart:

  1. Bursts. The producer emits a clump faster than the consumer drains it. Depth absorbs the clump so the producer need not stall mid-burst.
  2. Rate mismatch. One side is sustainedly faster for a while. Depth buys time before backpressure bites (but only transiently — see §2.4).
  3. Arbitration jitter. The consumer (HyperRAM, behind a shared round-robin arbiter) is unavailable for stretches while another tenant is served. Depth covers the outage so your producer keeps running through it.

The reference cores size their core→HyperRAM FIFOs at 16 entries (G_WR_DEPTH => 16, G_RD_DEPTH => 16; Part 3). Sixteen is comfortable for a bursty client sitting behind a shared arbiter: it is deep enough to ride out a normal arbitration outage without backpressure, and shallow enough to cost only a scrap of LUTRAM — small RAM built from the FPGA's lookup tables rather than a Block RAM — or one small BRAM.

2.4 A FIFO decouples in time; it does not add throughput

This is the sentence to tattoo on the inside of your eyelids. A FIFO smooths transient mismatch; it cannot fix a sustained one. If the consumer is chronically slower than the producer, no depth on earth saves you — the FIFO fills, backpressure latches on, and the producer runs at the consumer's rate regardless. A FIFO that is always nearly full is not a buffer, it is a warning light: the real fix is elsewhere — a cache to cut the traffic, longer bursts to raise effective bandwidth, or a faster consumer. Reach for depth to cover a hiccup, never to paper over a throughput deficit.

2.5 Synchronous vs asynchronous FIFOs

There are two species, and the difference is exactly which problem from Part 1 they solve:

  • A synchronous FIFO has both ends on the same clock. It is a pure elastic buffer — it decouples a bursty producer from a paced consumer, but no clocks are crossed. (M2M's axi_fifo_small, §3.4, is one of these.)
  • An asynchronous FIFO has its two ends on different, unrelated clocks. It does everything the synchronous one does and safely carries data across a clock-domain boundary. This is the species that matters for HyperRAM, because your core clock is not hr_clk.

Everything hard about a FIFO is hard only in the asynchronous case, and it is all about those two pointers.

2.6 Why crossing clocks is the hard part

Clocking and Clock Domain Crossing derives this in full; here is the one paragraph you need to follow the mechanism. When a signal born in clock A is sampled by an unrelated clock B, B's edge will eventually land exactly while the signal is switching, and the sampling flip-flop goes metastable — its output hovers between 0 and 1 for an unpredictable time before settling. Worse, for a multi-bit value like a pointer, the bits settle on their own schedules, so for a cycle the receiver can latch a torn value — some bits old, some new — a number that never existed. A binary counter going 0111 → 1000 flips all four bits at once; sample it mid-flight and you might read 1111 or 0000.

So an async FIFO cannot simply hand its binary write pointer over to the read clock (and vice versa) to compute full and empty — the pointer would tear, and the FIFO would conclude it is empty when it is full, or full when it is empty, and corrupt data. This is the whole problem an async FIFO has to solve.

2.7 Gray-code pointers and the two-flop synchronizer

The trick is beautiful and worth internalizing, because you will meet it again in the controller's own FIFO (§3.3). Encode each pointer in Gray code — a counting scheme in which exactly one bit changes between any two consecutive values:

   binary   gray
   000      000
   001      001
   010      011
   011      010     ← only ever ONE bit flips from the row above
   100      110
   101      111
   110      101
   111      100

Now when the other clock samples the pointer mid-increment, only one bit can be in transition, so the sampled value is either the old pointer or the new pointer — never a torn in-between. Whichever it reads is a legal pointer value, merely possibly one step stale, and one step stale only ever makes the FIFO look slightly more full or more empty than it is — conservative, never wrong. Pass that Gray-coded pointer through a two-flop synchronizer (two back-to-back flip-flops in the destination domain, giving any metastable wobble a full cycle to settle) and the receiver has a safe, slightly-conservative view of the other side's pointer. Compare the two pointers to derive full and empty, and the crossing is correct.

   ── write clock domain ──┐                    ┌── read clock domain ──
                           │                    │
   producer ─▶ [ring buffer memory: dual-port ] ─▶ consumer
                           │                    │
   wr_ptr (binary)         │                    │        rd_ptr (binary)
      │ to Gray            │                    │           │ to Gray
      ▼                    │                    │           ▼
   wr_gray ───────▶ [2-flop sync] ─▶ wr_gray_synced   rd_gray ──▶ [2-flop sync] ─▶ rd_gray_synced
                    (in read domain)   │                          (in write domain)  │
                                       ▼                                             ▼
                              compare vs rd_ptr → EMPTY?              compare vs wr_ptr → FULL?

That is the entire secret of asynchronous clock-domain crossing (CDC): Gray-code the thing that crosses, synchronize it with two flops, and never let a raw multi-bit value cross unprotected. Keep it in mind; the three real M2M FIFOs are three different packagings of exactly this idea.

Part 3 — M2M's FIFOs, dissected

Three FIFOs carry HyperRAM traffic in M2M, and each teaches something the others do not. All three are asynchronous; all three ultimately rest on §2.7.

3.1 avm_fifo — packing a whole transaction across a clock

M2M/vhdl/memory/avm_fifo.vhd (~135 lines) is the block that gets your core from its own clock into hr_clk. Its cleverness is what it chooses to carry: not a data word, but a whole Avalon transaction. It has a complete Avalon slave face on your clock (s_clk_i) and a complete Avalon master face on the HyperRAM clock (m_clk_i), and in between it flattens each request into a single FIFO word (avm_fifo.vhd:42-46):

   one packed request word (MSB on the left):
   ┌───────┬────────────┬───────────┬─────────┐
   │ write │ byteenable │ writedata │ address │
   └───────┴────────────┴───────────┴─────────┘
     1 bit     2 bits      16 bits    32 bits

   The read/write direction rides along as the single `write` bit;
   burstcount is NOT in this word — it rides in a side-channel (see below)

Packing the entire request into one word is what makes the crossing safe and simple: the producer pushes one atom, the consumer pops one atom, and the §2.7 machinery only ever has to keep two pointers coherent — never a half-written request.

The two internal FIFOs. avm_fifo is really two FIFOs back to back (avm_fifo.vhd:80, :107): a request FIFO carrying that packed word from your clock into hr_clk, and a response FIFO carrying readdata back from hr_clk into your clock. On the far side it simply replays the stored write bit to re-drive m_avm_write_o / m_avm_read_o (avm_fifo.vhd:73-74) — the transaction pops out exactly as it went in, one clock domain later.

burstcount rides a side-channel. The packed word does not include burstcount; instead it travels in the FIFO's AXI-Stream TUSER field — a general-purpose per-beat side-channel that rides alongside the data (s_wr_fifo_user <= s_avm_burstcount_i, avm_fifo.vhd:69, G_USER_SIZE => 8). (AXI-Stream is the streaming handshake the underlying Xilinx FIFO speaks; TDATA is its payload lane and TUSER a companion lane that crosses in lockstep with it.)

The response split, and why G_DATA_SIZE must be > 8. The response FIFO plays a small trick that explains an odd constraint. It stuffs the high G_DATA_SIZE-8 bits of each returned word into TDATA and the low 8 bits into that same TUSER side-channel (avm_fifo.vhd:119, :122), then reassembles them on the far side (:127, :130). It works for any width above 8; at exactly G_DATA_SIZE = 8 the TDATA slice would be zero bits wide — illegal — which is precisely why the block requires G_DATA_SIZE > 8. HyperRAM's 16-bit word sails past the limit; the rule only bites if you try to reuse avm_fifo for a byte-wide bus.

Flow control and the sizing rule that follows. The request side back-pressures your core normally — s_avm_waitrequest_o <= not s_wr_fifo_ready (avm_fifo.vhd:64) stalls you when the request FIFO is full. The response side, however, cannot back-pressure the HyperRAM controller: the response FIFO's HyperRAM-facing input-ready is left unconnected — s_axis_tready_o => open (avm_fifo.vhd:117) — so the controller's readdatavalid is never throttled and read data is pushed into the FIFO unconditionally. (On its core-facing output the FIFO is likewise hardwired always-ready, m_axis_tready_i => '1', avm_fifo.vhd:125, so your core is always drained — a separate fact.) That is deliberate — you cannot stall data the controller is already returning — but it hands you a sizing obligation:

G_RD_DEPTH must be deep enough to hold a whole in-flight burst. Because nothing throttles the returning read data, the response FIFO must have room for every word a burst can deliver before your core drains any of them, or it overflows. Size the read depth to your largest burstcount, not to a guessed average.

The reference cores instantiate avm_fifo with G_ADDRESS_SIZE => 32, G_DATA_SIZE => 16, G_WR_DEPTH => 16, G_RD_DEPTH => 16 — 16 comfortably covers their 8-word cache-line bursts with headroom.

The reset discipline — reset each side from its own clock. This is the one detail that most often turns a working async FIFO into a mysteriously hung bus, so learn it from the reference. Because the two ends live in different clock domains, each side must be reset from a reset that is synchronous to that side's own clock, and the two must only ever be asserted together. The Amiga core is the model (AExp/CORE/vhdl/mega65.vhd:922):

i_avm_fifo_adf : entity work.avm_fifo
   ...
   s_rst_i => main_reset_m2m_i,   -- request side: reset in the main_clk domain  (mega65.vhd:932)
   m_rst_i => hr_rst_i,           -- response side: reset in the hr_clk domain    (mega65.vhd:943)

The core's own comment (mega65.vhd:919-921) states the failure mode plainly: "resetting only one side desynchronizes the chain." If you flush the request side but not the response side (or drive both from a single reset that is only synchronous to one clock — a CDC violation on the other), commands already forwarded toward HyperRAM still return responses that pile into a just-flushed response FIFO, or a queued command finds no matching drained response. The Avalon handshake then waits forever and the bus deadlocks. Note the subtlety the Amiga makes explicit: s_rst_i is main_reset_m2m_i, the coordinated hard M2M reset — deliberately not the core's own volatile reset (amiga_rst, which also fires on a soft reset or a ~64-cycle keyboard warm boot, AExp/CORE/vhdl/main.vhd:418). "Reset each side from its own domain" also means "and only from the coordinated full reset that asserts both sides together" — never from a short, one-sided core reset. (This is the FIFO-level companion to the framework's don't-decouple-the- HyperRAM-reset rule, beginners guide §5.8.)

3.2 axi_fifo — the vendor macro underneath

Peel avm_fifo open and its two internal FIFOs are each an axi_fifo (M2M/vhdl/memory/axi_fifo.vhd), which is a thin wrapper over Xilinx's hardened xpm_fifo_axis. XPM stands for Xilinx Parameterized Macro — a library of pre-verified, silicon-proven building blocks (FIFOs, memories, CDC primitives) that Xilinx ships and supports, so you neither write nor debug the tricky §2.7 pointer logic yourself. The wrapper just pins the parameters that matter (axi_fifo.vhd:41-58):

  • CLOCKING_MODE => "independent_clock" — the two ports run on unrelated clocks. This is the switch that makes it a true asynchronous, clock-crossing FIFO.
  • CDC_SYNC_STAGES => 2 — the two-flop synchronizer of §2.7, made explicit.
  • FIFO_MEMORY_TYPE => "auto" — let the tool pick LUTRAM for a shallow FIFO or BRAM for a deep one, by depth.
  • USE_ADV_FEATURES => "1404" — enables the data-count outputs that surface as the fill-level ports.

The lesson worth carrying away: "an async CDC FIFO" is not exotic hardware you invent — it is a proven vendor macro you instantiate. Everything in §2.7 is real and worth understanding, but in production you lean on the macro and spend your care on sizing and reset, not on re-deriving Gray code.

3.3 hyperram_fifo — the controller's hand-rolled receive FIFO

The one place M2M does not use the vendor macro is inside the HyperRAM controller itself (M2M/vhdl/controllers/hyperram/hyperram_fifo.vhd), and the reason is a lovely piece of engineering judgment. This is the shallow asynchronous FIFO on the controller's receive path — the one the beginners guide met in §4.4, that catches read data arriving on the chip's non-free-running RWDS strobe and carries it into the steady 100 MHz domain. It is deliberately hand-rolled with the exact §2.7 mechanism — bin2gray/gray2bin functions and a two-flop async_reg synchronizer on the write pointer (hyperram_fifo.vhd:60-88, :132-138) — rather than an xpm_fifo_axis, for two reasons the file states:

  • It wants a set_max_delay, not a set_false_path. The XPM FIFO bundles its own set_false_path timing exception; this receive path instead needs a bounded set_max_delay -datapath_only constraint (the same family of constraint every M2M CDC helper needs, Clocking and Clock Domain Crossing §2.6). Rolling the FIFO by hand keeps that timing contract under the designer's control (hyperram_fifo.vhd:129-131).
  • It uses LUTRAM for a fast write. The storage is distributed LUTRAM (ram_style => "distributed", hyperram_fifo.vhd:37-38) rather than a Block RAM, because the delayed-RWDS write side needs a lower write delay than BRAM can offer. (BRAM has a deeper write path; for this tiny four-word receive FIFO the LUTRAM is both smaller and quicker.)

It is genuinely tiny: C_GRAY_SIZE = 3 gives C_FIFO_SIZE = 2**(3-1) = 4 words (hyperram_fifo.vhd:27, :30). And it is this FIFO that forces the beautiful "read one extra word, discard the first" trick of beginners §4.4: because RWDS stops toggling after the last data word, the FIFO's write side is left one clock pulse short of pushing that final word across the domain, so the controller clocks one extra word out of the chip to flush it through and then throws away the now-bogus first FIFO entry. The odd trick and this odd little FIFO are the same fact seen from two sides.

One naming caution: unlike avm_fifo, hyperram_fifo is not an Avalon FIFO. It is a plain src_valid/src_data → dst_valid/dst_data streaming FIFO (hyperram_fifo.vhd:14-21) — no waitrequest, no addresses — because the receive path only ever streams words one direction. Do not reach for it as a general crossing block; avm_fifo is that block.

3.4 Do not confuse axi_fifo with axi_fifo_small

A trap hiding in the file names. M2M/vhdl/axi_fifo_small.vhd sounds like a smaller sibling of axi_fifo, but it is the opposite species: a single-clock, inferred-RAM FIFO (§2.5) — a plain elastic buffer with no clock crossing at all. It is instantiated in exactly one place in the whole framework, inside avm_increase.vhd (the width up-converter), and nowhere else.

So keep them straight: axi_fifo is the dual-clock CDC FIFO (the workhorse inside avm_fifo); axi_fifo_small is a single-clock buffer used once, deep inside a width converter. Same stem, opposite job.

Part 4 — Caching from first principles

A FIFO moved data across a boundary in time. A cache does something different: it makes a slow, distant memory appear fast by keeping a small copy of the useful part close by. This part builds the vocabulary — locality, line, tag, hit, miss, associativity, replacement, write policy, prefetch — so that when we dissect the two real M2M caches in Parts 5 and 6, every word already means something.

4.1 Locality: the reason caches work at all

A cache is a bet — or, more accurately, a prediction — and what it predicts is locality: the empirical fact that programs do not touch memory at random:

  • Temporal locality — if you touched an address, you will probably touch it again soon (a loop re-reads the same code; a bank stays selected for a while).
  • Spatial locality — if you touched an address, you will probably touch its neighbours soon (a DMA copy walks consecutive addresses; a track is read sector after sector).

Where there is locality, a small fast memory holding the recently and nearby used data will satisfy the vast majority of accesses. Where there is none — truly random access with no reuse — a cache cannot help, and you must not use one (Part 7).

4.2 The memory hierarchy in one sentence

Put a small, fast memory in front of the big, slow one; serve the common case from the fast copy and disturb the slow chip only when you must. On the MEGA65 that is BRAM in front of HyperRAM — exactly the pairing beginners guide §1.2 set up. Every term below is machinery for deciding what to keep in the fast memory and what to do on the rare access that is not there.

4.3 Line, tag, hit, miss, and the number that matters

A cache does not track individual words; it works in lines. A cache line is a contiguous run of words moved and tracked as one unit — sized to exploit spatial locality (fetch the neighbours while you are paying the access cost anyway). Each resident line carries a tag: the identity of which chunk of the big memory currently occupies it.

On each access the cache compares the requested address against the resident tags:

  • Hit — the line is present. Serve it from fast memory, immediately. The slow chip is never touched.
  • Miss — the line is absent. Fetch it from slow memory (the miss penalty — the full slow latency you have been trying to avoid), install it, then serve.

The hit rate — the fraction of accesses that hit — is the whole game, because average access time is hit_rate × fast + miss_rate × slow. Since slow dwarfs fast, a cache is worth having exactly when it makes misses rare. A cache that misses often is worse than no cache at all: you pay the slow access plus the bookkeeping.

4.4 Associativity: where is a line allowed to live?

When a chunk of big memory is pulled in, which slot(s) of the fast memory may hold it? That choice is associativity, and it trades comparator cost against conflict:

  • Direct-mapped — each chunk maps to exactly one slot (by some bits of its address). One comparator, dead cheap. But two hot chunks that happen to map to the same slot evict each other forever — a conflict miss — even while other slots sit empty.
  • Fully associative — a chunk may live in any slot. Zero conflict misses, best hit rate for a given size. The cost: you must compare the address against every slot's tag in parallel, so N slots means N comparators.
  • N-way set-associative — the middle ground: a chunk may live in any of N slots within one set. Most CPU caches live here.

The two M2M caches sit at the two extremes, which is what makes them such a clean teaching pair: avm_cache (Part 5) is a single streaming line; crt_cacher (Part 6) is fully associative with a parallel tag search across all its slots.

4.5 Replacement: which line to evict on a miss

When a miss needs a slot and all candidate slots are full, one resident line must be evicted — it becomes the victim. The policy that picks the victim:

  • LRU (least-recently-used) — evict the line unused longest. Best hit rate, but you must track a recency order across all slots — real hardware cost.
  • FIFO / round-robin — evict in a fixed rotation (a wrapping victim pointer), ignoring usage entirely. Almost free: one small counter.
  • Random — evict a random slot. Cheapest of all, surprisingly decent.

The lesson Part 6 will make concrete: when the working set is a handful of slots holding a tiny hot set, round-robin performs almost as well as LRU for a fraction of the logic, which is exactly why crt_cacher uses it.

4.6 Write policy: what a write does — and the word that means two things

Reads are half the story. When the client writes, the cache must decide how the write reaches the slow backing store. There are two classic policies, and you must know both because one M2M core uses both at once (Part 7):

  • Write-through — every write updates the cached line and is forwarded to the backing store immediately. The two are always in sync, so a later read is always coherent. The cost is write traffic: every write touches the slow memory.
  • Write-back — a write updates the cached line only, marking it dirty; the dirty line is written to the backing store later, lazily (typically on eviction, or on a timer). Far less write traffic and kinder to a slow medium, but it needs dirty-tracking and a flush discipline, and un-flushed data is lost if power drops first.

(A related knob, write-allocate, decides whether a write miss pulls the line in first; it is not central to the M2M caches, so we leave it aside.)

Watch this word. "Write-back" names both a cache write policy (the opposite of write-through) and, informally, the act of persisting data to a backing file. In the Amiga core those are two different layers with opposite policies — its BRAM cache is write-through, while its HyperRAM-to-SD relationship is write-back. Part 7.2 pulls them apart deliberately; keep the ambiguity in mind until then.

4.7 Prefetch: don't wait for the miss you can see coming

A miss costs the full slow latency — unless you fetched the line before it was asked for. Prefetch bets on spatial locality: while the client is chewing through the current line, speculatively pull in the next one, so by the time the client reaches it the data is already resident and the miss never stalls. The simplest and most effective form is sequential prefetch — "they are walking forward, so fetch the address ahead" — and it is the beating heart of avm_cache, which we open up next.

Part 5 — Strategy A: the streaming prefetch cache (avm_cache)

M2M/vhdl/memory/avm_cache.vhd (~227 lines) is the framework's stock cache, and its own header is refreshingly honest: "a very simple read cache with just one cache line" (avm_cache.vhd:1). One line. That sounds too small to matter — until you see how it uses that line, which is the whole lesson. Both streaming clients in the project use it: the C64 REU and the Amiga floppy.

5.1 The shape: one line, two states

The entire cache is a G_CACHE_SIZE-word array (cache_data), a base address for the line (cache_addr), a count of valid words (cache_count), and a two-state machine (avm_cache.vhd:42-55):

  • IDLE_ST — serving the client from the resident line, or launching a fill on a miss.
  • READING_ST — draining a burst from HyperRAM into the line.

G_CACHE_SIZE is literally the number of 16-bit words in that single line — 8 in both reference cores. It runs on one clock (clk_i), which is the point of the whole design: it lives entirely on your core's clock, before the CDC (§5.5).

5.2 The hit, and the sliding-window prefetch that makes one line enough

A hit is a single-word read whose address falls inside the resident line (cache_offset_s < cache_count, avm_cache.vhd:61). The hit is detected by combinational logic (cache_rd_hit_s, :61), so waitrequest drops the same cycle (:67) and the request is accepted at once; the word itself is then returned registered, one core-clock later, straight out of cache_data (:110-112) — no HyperRAM, no wait. A miss issues one G_CACHE_SIZE-word burst to fill the line (:130) and pays the slow latency once for the whole line instead of once per word.

The clever part — the reason a single line suffices for endless sequential streams — is the sliding-window prefetch (avm_cache.vhd:113-124). The moment the client has consumed the first half of the line, the cache reshapes the line under it, in the same cycle:

   line = 8 words                consume first half            after the slide:
   ┌──┬──┬──┬──┬──┬──┬──┬──┐      ┌──┬──┬──┬──┬──┬──┬──┬──┐     ┌──┬──┬──┬──┬──┬──┬──┬──┐
   │w0│w1│w2│w3│w4│w5│w6│w7│      │▓▓│▓▓│▓▓│▓▓│w4│w5│w6│w7│     │w4│w5│w6│w7│N0│N1│N2│N3│
   └──┴──┴──┴──┴──┴──┴──┴──┘      └──┴──┴──┴──┴──┴──┴──┴──┘     └──┴──┴──┴──┴──┴──┴──┴──┘
   cache_addr = A                 first 4 read (▓ = spent)      base advanced by 4;
                                                                second half shifted down;
                                                                next 4 words (N0..N3)
                                                                prefetched in background

Concretely, when cache_offset_s reaches G_CACHE_SIZE/2-1, the cache shifts the second half down to the front (cache_data(0 .. N/2-1) <= cache_data(N/2 .. N-1), avm_cache.vhd:120); advances cache_addr by G_CACHE_SIZE/2 (:121); and issues a background burst for the next G_CACHE_SIZE/2 words from cache_addr + G_CACHE_SIZE (:118-119). It is a double buffer built inside one line: the client keeps hitting on the words already there while the next chunk streams in behind them. A long sequential run — a REU DMA copy, an ADF sector fetch — hits continuously and essentially never stalls after the first fill.

5.3 Write-through, and why the Amiga cannot live without it

avm_cache is write-through (§4.6). A single-word write forwards straight to HyperRAM (avm_cache.vhd:92-98), and if the write address is inside the resident line, the cache patches its own copy in place, byte by byte, honouring byteenable (:99-105, and again in the fill state at :159-165):

if cache_wr_hit_s = '1' then
   for i in 0 to G_DATA_SIZE/8-1 loop
      if s_avm_byteenable_i(i) = '1' then
         cache_data(to_integer(cache_offset_s))(8*i+7 downto 8*i) <= s_avm_writedata_i(8*i+7 downto 8*i);
      end if;
   end loop;
end if;

For the read-only REU this is invisible. For the read-write Amiga floppy it is the linchpin of correctness. The Amiga writes a track and then reads it back; without write-through, the read would be served from a stale cached line and the disk would silently corrupt. Because the write both reaches HyperRAM and refreshes the cached copy, a read-after-write returns the new value — read-after-write coherence — which the Amiga's own header calls out as the reason it uses this block (AExp/CORE/vhdl/main.vhd:581-584). Note that the hit test requires burstcount = "01" (avm_cache.vhd:64); that is exactly why the Amiga engine documents its sector commits as single-word writes — a burst write would not register as a hit and would leave the line stale.

5.4 Its one limitation: strictly sequential

The flip side of the streaming design: avm_cache is only good for sequential access. A random single-word read that misses pays a full G_CACHE_SIZE-word burst to fetch a line of which the client wants one word — all cost, no benefit: 7 of the 8 fetched words are thrown away. Use it where the client walks addresses (DMA, sector streaming); do not use it for random-access patterns (that is what Part 6 is for).

5.5 The placement law: core-side, before the CDC

The load-bearing rule the beginners guide stated (§6.1, Porting Guide S78) now has its full why. avm_cache runs on one clock and answers a hit in one cycle of that clock. Put it on the core side, before the avm_fifo, and a hit never touches the crossing. Put it on the far side of the FIFO and every hit — the common case, the whole point — would have to make the async round trip it exists to avoid, paying the very latency it is there to hide. So: cache first (your clock), FIFO second (cross the clock). In the REU this is explicit — avm_cache_inst sits in main.vhd on clk_main_i (C64MEGA65/CORE/vhdl/main.vhd:1755-1762, G_CACHE_SIZE => 8 at :1757), and the main2hr_avm_fifo that crosses into hr_clk lives one hierarchy level up in mega65.vhd:1148. Cache and crossing are in different files precisely because they are on different sides of the clock boundary.

5.6 Worked example: the REU warms the cache on purpose

The REU (beginners guide §7) is the archetypal streaming client, and reu_mapper.vhd adds a touch worth seeing. Besides the byte-to-word adaptation (dropping the low address bit, using byteenable = "01"/"10" for the byte lane — reu_mapper.vhd:100-102), it contains what its own comment cheerfully labels "a massive hack" (reu_mapper.vhd:75-94): the instant the REU's address jumps (rather than steps sequentially), the mapper fires a speculative read purely to warm the cache, then discards that speculative read's response so it does not reach the REU. By the time the REU issues the real read for the new address, the line is already filling — so even the REU's occasional non-sequential jumps see a warm cache. It is a domain-specific flourish (you would not copy it into an unrelated core), but it shows the mindset: shape your access so the streaming cache stays warm.

Part 6 — Strategy B: the fully-associative bank cache (crt_cacher)

The REU tolerated latency; a C64 cartridge does not. A running program bank-switches and expects each bank to answer at ROM speed now. That rules out avm_cache (a random bank switch is exactly the sequential-only cache's worst case) and forces a real, hand-built cache. The result, C64MEGA65/CORE/vhdl/crt_cacher.vhd (~300 lines), is a genuine CPU-style fully-associative cache built for one specific access pattern — and it is the masterclass of this article. (Beginners guide §8 gives the high-level tour; here is the line-by-line.)

6.1 Where it sits, and the size

One structural fact first, because it is easy to get lost: the cache is two levels down from the wrapper you would grep. sw_cartridge_wrapper.vhd (C_CACHE_SIZE := 3, :72) instantiates crt_loader.vhd (:261), which in turn holds three things — the CRT file parser, an internal 2→1 arbiter, and the cache proper, crt_cacher (crt_loader.vhd:151). C_CACHE_SIZE = 3 gives 2**3 = 8 slots per side. "Per side" because a C64 cartridge exposes two 8 KB windows — ROML at $8000 and ROMH at $A000 — so the cache keeps two independent slot sets, LO and HI.

6.2 Two-level storage: a full directory vs the resident tags

The heart of the design is that it separates two lookups that a naive cache would conflate:

   LEVEL 1 — the full directory (all 128 banks, filled once at load)
   lobanks / hibanks : array(0 to 127) of 23-bit HyperRAM byte address
   ┌──────┬──────┬──────┬──────┬──────┐
   │ bank0│ bank1│ bank2│ .... │bnk127│   which HyperRAM address holds each bank?
   └──────┴──────┴──────┴──────┴──────┘

   LEVEL 2 — the resident tags (only the 8 banks currently in BRAM)
   cache_ram_lo / cache_ram_hi : array(0 to 7) of 7-bit bank number
   ┌─────┬─────┬─────┬─────┬─────┬─────┬─────┬─────┐
   │slot0│slot1│slot2│slot3│slot4│slot5│slot6│slot7│   which bank sits in each BRAM slot?
   └─────┴─────┴─────┴─────┴─────┴─────┴─────┴─────┘
  • Level 1 — the bank→address directory. lobanks and hibanks are array(0 to 127) of 23-bit HyperRAM byte addresses (crt_cacher.vhd:80-81) — a complete table covering all 128 possible banks, not a cache. It is filled once as the parser walks the CRT file (p_banks, :107-127). The ROML/ROMH split lives here: a 16 KB bank stores its ROMH half at +0x2000 after its ROML half (hibanks(...) <= raddr + 0x2000 when cart_bank_size_i > 0x2000, crt_cacher.vhd:119-120).
  • Level 2 — the resident tags. cache_ram_lo/cache_ram_hi are array(0 to 7) of 7-bit bank numbers (crt_cacher.vhd:93-94) — the tags: which bank currently occupies each of the 8 BRAM slots. This is the actual cache metadata; the 8 KB of bank data per slot lives in BRAM outside the module (§6.5).

Splitting them is what keeps the tag search tiny: you never search 128 entries, only the 8 resident tags — and the directory turns any hit-or-miss decision straight into a HyperRAM address.

6.3 The parallel tag search and the round-robin victim

The hit/miss engine (p_crt_load, crt_cacher.vhd:225-316) watches the live bank register the running C64 program writes (via $DExx/$DFxx). When the bank number changes (bank_lo_d /= bank_lo_i, :242):

  • It loops over all 8 tags in parallel — for i in 0 to 2**G_CACHE_SIZE-1, :245-252 — comparing each against the requested bank. Because the comparison is unrolled across every slot, any bank can live in any slot: fully associative (§4.4), made real in eight lines of VHDL.
  • Hit (cache_ram_lo(i) = bank_lo_i): it simply outputs that slot index as the BRAM read address (cache_addr_lo_o <= i, :249) and is done. No HyperRAM access, no fill, no stall — the C64 keeps running at ROM speed. This is the common case, and it is why the whole scheme works.
  • Miss: it hands out the round-robin victim slot (cache_addr_lo_o <= next_cache_addr_lo, :255), overwrites that slot's tag with the new bank number (:256), advances the victim pointer (next_cache_addr_lo <= next_cache_addr_lo + 1, :260 — a plain wrapping counter, the round-robin of §4.5, with the code's own comment admitting it is "very simple"), and requests a fill.

6.4 The miss-fill FSM: chained 256-byte bursts

Filling a slot means pulling a whole 8 KB bank (4096 words) from HyperRAM into BRAM, and the FSM (p_fsm, crt_cacher.vhd:129-223, states IDLE_ST/READ_HI_ST/READ_LO_ST) does it in 128-word (256-byte) bursts (avm_burstcount_o <= x"80", :150/:159), at the bank's word address (hibanks(bank_hi_i)(22 downto 1) — the directory's byte address shifted to a word address, :149/:158). Thirty-two such bursts fill one bank. To keep HyperRAM from idling between them it uses early-refill prefetch: two words before a burst boundary (bram_address_o(6 downto 0) = x"7E", :172/:192) it pre-issues the next burst (avm_address_o + x"80", :178-179), so the fill pipeline never bubbles. And a restart guard (:173/:193) aborts and re-launches the fill from the top if the program flips banks again mid-fill — so the resident data always matches the latest requested bank.

6.5 The miss stall is a CPU freeze — and why that is fine

A half-filled slot must never be read, so during a fill the cache freezes the CPU. The signal is bank_wait_o, asserted whenever a fill is pending or running (crt_cacher.vhd:100-104). It is carried across into the core clock and turned into a DMA bus-hold — the dma_req line of the C64's CPU (a 6502-family 6510), which halts the CPU on a clean cycle boundary (originally so external hardware could borrow the bus). The exact join is main.vhd:1155:

core_dma_v := cartridge_loading_i or crt_bank_wait_i;   -- freeze during load OR bank-cache fill

For the tens of microseconds it takes to pull 8 KB from shared HyperRAM, the 6510 simply stops; then it is released and reads its bank out of BRAM at full speed. Because bank switches are discrete and infrequent, this pause is imperceptible — and that is the entire justification for the caching strategy. Under a shared, contended memory you cannot make any single access reliably fast, so the design instead makes misses rare and eats a brief, invisible freeze when one happens.

6.6 The dual-port BRAM is the clock-domain bridge

The most reusable idea in the module is what it does not contain. crt_cacher runs entirely in the hr_clk domain and holds no bank data at all; its header says so (crt_cacher.vhd:4-9). The 8 KB-per-slot storage is two true dual-port BRAMs (tdp_ram) up in the wrapper (sw_cartridge_wrapper.vhd:466, :486), each ADDR_WIDTH = 12 + C_CACHE_SIZE = 15 bits — the address is {slot}{word-in-bank}. The two ports are deliberately on different clocks:

flowchart LR
  FSM["crt_cacher FSM (hr_clk)<br/>fills word-by-word from<br/>256-byte HyperRAM read bursts"]
  BRAM["true dual-port BRAM<br/>8 slots × 8 KB<br/>(the RAM itself crosses the clock domain)"]
  CPU["the C64 CPU (main_clk)<br/>reads its bank at ROM speed"]
  FSM -->|"Port B: write, hr_clk"| BRAM
  BRAM -->|"Port A: read, main_clk"| CPU
Loading

Because a true dual-port BRAM resolves asynchronous access between its two ports in the silicon itself, it is the clock-domain crossing for the 8 KB of bank payload — no FIFO, no handshake needed for the bulk data. Only the thin control signals (the slot index the search picks, plus bank_wait and a little status) need an explicit crossing, and they ride one shared cdc_stable bundle (sw_cartridge_wrapper.vhd:396-420). This is the pattern to steal: keep the big memory outside the FSM as a dual-clock BRAM, and it doubles as the bridge between the fill clock and the read clock.

Part 7 — Choosing, designing, tuning

You have now seen both strategies in full. This part is the payoff: how to pick, how to avoid the terminology trap, and how to size, simulate, and build your own.

7.1 The decision: match the tool to the access pattern

Your client's access pattern What to build
Sequential, latency-tolerant (DMA copy, sector stream) avm_cache (streaming prefetch), on the core clock, before the FIFO.
Random but local, and you can stall the consumer on a miss (bank switching, tile/sector fetch) A bespoke associative cache like crt_cacher: tag array + victim pointer + BRAM slots + freeze-on-miss.
Truly random, no locality, and the consumer cannot stall Neither cache helps — it must live in BRAM, or the design has to change. Caching a pattern with no locality just adds a slow miss to every access.
Two or more of your own HyperRAM clients Merge them with your own avm_arbit_general before hr_core_* (nested arbitration; beginners guide §6.1).

The single question that routes you through this table is the one from Part 4: does the access pattern have locality, and of what shape — sequential or clustered-random?

7.2 The two "write-backs" — say both out loud

Here is the terminology trap flagged in §4.6, resolved on a real design. The Amiga floppy core stacks two cache-like layers with opposite write policies, and the same word names both:

flowchart LR
  AM["running Amiga"] -->|"read / write sectors"| C["avm_cache<br/>BRAM, 1 line"]
  C -->|"WRITE-THROUGH<br/>coherence · hardware · per access"| HR["HyperRAM<br/>the disk surface"]
  HR -->|"WRITE-BACK<br/>persistence · firmware · lazy"| SD["SD .adf file<br/>the backing store"]
Loading
  • Layer 1 — avm_cache relative to HyperRAM is write-through (§5.3). Every Amiga write hits HyperRAM immediately and refreshes the cached line, so the next read is coherent. This is a hardware mechanism, firing on every access, purely for read correctness. Nothing to do with the SD card.
  • Layer 2 — HyperRAM relative to the SD file is write-back. HyperRAM is volatile; changed tracks must reach the .adf file eventually, but SD writes are far too slow to do on every access. So a dirty bitmap records which tracks changed, an anti-thrashing timer (default 2000 ms) waits for writes to go quiet, and a firmware background flush copies only the dirty tracks (5632 bytes each, not the whole 880 KB image) to SD. That is a textbook write-back cache, with HyperRAM as the cache and the SD file as its backing store: writes deferred and coalesced, dirty state tracked, persistence lazy.

So HyperRAM is, at the same time, the write-through backing store for avm_cache and the write-back cache for the SD file — same word, opposite policy, two different layers. Name both when you describe such a design, or your reader (and your future self) will conflate a per-access coherence mechanism with a lazy persistence one. (The mechanics of the dirty bitmap, the write-1-to-clear register (writing a 1 to a bit clears it) where a hardware "set" beats a concurrent firmware "clear", the HANDLE_CORE_IO callback that runs the flush even while the menu is open, and the yellow drive-LED that means "not yet safe to power off" are all in the Amiga core's own floppy-adf.md; they are the persistence half, downstream of everything this article covers.)

7.3 Design for the worst case: a miss can cross four arbiters

When you size and place your cache, remember what a miss actually costs on the real hardware. It is not just HyperRAM's ~7–~11-clock first-word latency; it is that latency behind up to four levels of arbitration. A cartridge miss traverses all four: the 2→1 arbiter inside crt_loader (the cacher versus the CRT parser, crt_loader.vhd:187), then a second 2→1 arbiter in sw_cartridge_wrapper (the cartridge path versus QNICE's mount/load access, sw_cartridge_wrapper.vhd:312), then the core's own 3→1 avm_arbit_general (mega65.vhd:534), then the framework's 3→1 tenant arbiter (beginners guide §5.3) — and every one of them can be busy with someone else, especially the video scaler hammering the bus during active display. The controller is single-transaction (it finishes one read before accepting the next; beginners guide §3.4), so these do not overlap — they add up. Design your cache so misses are rare (high hit rate) and so the consumer can genuinely tolerate the stall when one lands, because under contention a miss is not "a few clocks" — it is a queue.

7.4 Tuning the two knobs

  • G_CACHE_SIZE (avm_cache). The words per line. Both reference cores use 8, which pairs a 4-word working half with a 4-word prefetch half (§5.2). Larger lines amortise the fill over more hits on long runs but waste more bandwidth on a short or random access, and enlarge the burst the read FIFO must hold. Start at 8; only change it with a measured access pattern.
  • G_WR_DEPTH / G_RD_DEPTH (avm_fifo). The FIFO depths of §2.3. 16/16 is the proven default. The hard rule from §3.1: G_RD_DEPTH must hold a full in-flight burst, because nothing throttles the returning read data — so if you enlarge G_CACHE_SIZE, revisit G_RD_DEPTH.

7.5 Prove it in simulation before Vivado

A HyperRAM client's bugs — a mis-sized FIFO, a cache that stalls, a protocol slip — are far cheaper to find in a testbench than on a board after a 20-minute synthesis. Drive your cache and FIFO against the framework's sim-only avm_memory_pause model (beginners guide §6.1), which answers Avalon requests with injected wait cycles so you exercise exactly the stalls and variable latency the real shared chip will throw at you. If it survives avm_memory_pause with random pauses, it will survive the arbiter.

7.6 A recipe for your own bespoke cache

When the decision table (§7.1) sends you to build something like crt_cacher, this is the shape, distilled from Part 6:

  1. Pick the unit of locality as your line = BRAM slot. A cartridge bank, an arcade tile page, a disk sector — the chunk your client reuses. Size the slot to it.
  2. Keep a small tag array and a victim pointer. One tag (the chunk id) per slot; a wrapping round-robin pointer for eviction (§4.5) — LRU is almost never worth the logic for a handful of slots.
  3. On access, search the tags in parallel. Hit → re-point the BRAM read address, zero stall. Miss → allocate the victim slot, rewrite its tag, and burst the unit in from HyperRAM (with early-refill prefetch so the fill does not bubble).
  4. Stall the consumer during the miss. A DMA bus-hold, a waitrequest, a clock-enable gate — whatever freezes your client cleanly. This is legitimate precisely because a good design keeps misses rare.
  5. Put the BRAM outside the cache FSM, as a dual-clock BRAM. It then doubles as the clock-domain bridge (§6.6): the FSM fills on hr_clk, the consumer reads on the core clock, and only the thin control signals need an explicit CDC.

That shape fits a large banked ROM, an arcade tile ROM, or a CD/disk sector buffer as readily as it fits a C64 cartridge. It is the same five moves every time.

Where to go next

  • The survey this article deepened: HyperRAM for Beginners — the Avalon protocol, the controller internals, the arbiter and the C_HMAP_* map, and the toolbox these blocks come from.
  • The clock-domain half of the job: Clocking and Clock Domain Crossing — metastability, torn buses, the two-flop synchronizer, and the set_max_delay constraint every CDC (including every FIFO here) needs.
  • The exact wiring steps: The Ultimate MiSTer2MEGA65 Porting Guide — §2.6 Phase 6 (S77–S78) for the hr_core_* master and the avm_fifo + avm_cache recipe, §3.I.8 for the Avalon reference, and §3.F for CDC and the XDC constraints.
  • The QNICE device bus that loads a file into HyperRAM before your cache serves it: Devices.
  • The persistence half this article stops before: how a written HyperRAM image is flushed back to the SD .adf file — the dirty-bitmap, anti-thrashing timer, and HANDLE_CORE_IO background-flush discipline. The Amiga core's floppy-adf.md design note documents it end to end.
  • The controller and its FIFO, in depth: MJoergen's HyperRAM documentation — the receive-path timing, the gray-code FIFO, and the full Avalon and HyperBus references.

Clone this wiki locally