GPU AtomSpace: implement the real data structures (guided by Linas)

[plankatron]

Context

Previous issues (#1-#6) and PR #3 were closed as LLM slop — they described custom language-learning GPU pools that have no relationship to the actual AtomSpace data structures. Starting fresh based on direct guidance from Linas (Feb 12 2026 Discord).

Goal

Put a hypergraph structure in GPU RAM that resembles the AtomSpace. Not a language-learning engine — a general-purpose atom store.

The 4 Steps (from Linas)

Step 1: Study opencog/atomspace/ (~3KLOC)

How the AtomSpace keeps track of atoms:

• TypeIndex: vector of AtomSet (8 hash buckets per Type)
• Each bucket: unordered_set<Handle> + shared_mutex
• Insert/find/remove by content hash, O(1) average
• Bucket selection: (atom_hash % 8) + (type * POOL_SIZE)

A GPU version needs: a hash table per type (or a flat table with type discrimination), atomic insert/find/remove.

Step 2: Study opencog/atoms/base/ (~2KLOC)

What an atom actually is:

• Atom (144 bytes): Type (uint16), ContentHash (uint64), flags (uint8), values (key→value map), incoming set (type→set map)
• Node adds: std::string _name
• Link adds: HandleSeq _outgoing (vector of atom references, any arity)
• Links can point to other links (not just nodes)
• Content hash is Merkle-tree: nodes hash type+name, links hash type+outgoing hashes

GPU version needs: a way to represent nodes and links with their hashes, names (for nodes), outgoing sets (for links), and incoming sets. Values are attached to atoms via keys.

Step 3: Implement BackingStore.h on CPU side

This is the API for moving data to/from GPU RAM. Required methods:

• storeAtom(Handle) — CPU → GPU
• getAtom(Handle) — GPU → CPU
• fetchIncomingSet(AtomSpace*, Handle) — get all links pointing at an atom
• fetchIncomingByType(AtomSpace*, Handle, Type) — filtered incoming
• loadType(AtomSpace*, Type) — bulk load all atoms of a type
• loadAtomSpace(AtomSpace*) / storeAtomSpace(AtomSpace*) — bulk
• barrier() — sync fence

Step 4: GPU kernels access Values

The actual parallel computation happens on Values (FloatValue, etc.) attached to atoms — not on the atom graph structure itself. Kernels read/write Values in bulk while the graph structure is just the index.

What NOT to do

• Do NOT implement specific atom types in C++/CUDA (ignore the type hierarchy beyond base Node/Link)
• Do NOT build domain-specific pools (no WordPool, PairPool, SectionPool)
• Do NOT design JIT compilers, LLM comparisons, or grand architectures
• Keep it small, keep it testable, verify against the real AtomSpace at each step

Approach

Small steps. Each step verified before moving on. Ask Linas before building.

[linas]

Review:

• Skip step 1.
• Ignore the merkel-tree hash in step 2. That is, ignore ContentHash (uint64), flags (uint8), values (key→value map), incoming set (type→set map) these are not yet needed. (not obviously useful on the GPU)
• Step 3: do only storeAtom() and getAtom() for now; stub out the other methods.

[linas]

The hard part is what to do about Values. I think these will have a completely different design on the GPU as compared to the CPU, and yet they have to be mapped to one-another.

First let me explain what happens on the CPU side. There, every Atom has a key-value map. Keys are always some Atom. Values are one of 4 things: a vector of floats, a vector of strings, a vector of bools, or a vector of Values. Given an Atom plus a key, the Atom::getValue(key) method returns the Value at that (atom,key) pair.

On the GPU side, I am thinking the slicing needs to go the other way. You have a table. For each key in the table, there are two vectors: the first vector is a vector of atoms. The second is a vector of Values. These must be in correspondence to what is on the CPU side of things.

The reason for this slicing is that it makes paralleliing loops easy, On the CPU side, loops often have the form:

for all atoms A in set S do {
   get value Vx at key X;
   get value Vy at key Y;
   add Vx and Vy to get Vz;
   store Vz at key Z;
}

On the GPU side, the the kernel becomes add Vx and Vy to get Vz; and you skip the part that does get value Vx at key X; because that' has been pre-vectorized by the slice. You used key X to look up a vector in the slice table; you just hand it to the GPU's.

There's an unresolved issue here: suppose the vector of Atoms at key X is not exactly identical to the vector of Atoms at key Y? Then what? There are several ways to solve this, but I leave it open as an unresolved issue, for now.

[plankatron]

Step 2 complete: nodes and links in GPU RAM

AoS layout — one 12-byte GpuAtom struct per slot (type, is_node, data_offset, data_len), with shared pools for variable-length names and outgoing sets. No hashes, flags, values, or incoming sets.

AoS because graph access is per-atom (store one, fetch one). SoA reserved for Step 4 (Values slice-table).

CUDA + OpenCL both compile in. gpu_table_alloc() tries CUDA first, falls back to OpenCL.

~92 MB GPU RAM at defaults (1M atoms, 64MB names, 4M outgoing slots).

All old code removed (GpuStorageNode, kernels, benchmarks, gpu-types, guile bindings).

Branch: step2-atom-table

---

where we are

• Step 1 — skipped per review
• Step 2 — nodes + links in GPU RAM (this branch)
• Step 3 — storeAtom() / getAtom() via BackingStore.h
• Step 4 — Values slice-table (the hard part)

[plankatron]

Step 3+4 proposal

Step 3: GpuStorageNode — BackingStore wrapper around GpuAtomTable. Implements storeAtom() and getAtom(), stubs everything else. CPU-side ContentHash → slot map handles atom identity. URI: gpu://.

Step 4: Values slice-table — per-key columnar storage on GPU. For each key K, two contiguous device arrays: atom_slots[] and values[] (doubles). CPU resolves keys to device pointers, hands them to kernels. Only FloatValue for now.

The slice-table is SoA applied to Values (not to atom structure). This matches your description in the earlier comment — "for each key in the table, there are two vectors." Want to confirm this is the right direction before building it.

Starting with Step 3. Will post Step 4 code after your review.

[linas]

First, the easy parts:

• the 64-bit content hash is not a unique UUID; you have some small chance of collisions (due birthday paradox) ... a 128-bit or 256-bit would be "better" but is maybe overkill for just right now.
• You need to keep an inverse table somewhere: given a (quasi-)uid, what is the corresponding atom? If this table is kept on the CPU, then I can nominate the opencog/persist/uuid blob to do that -- its simple, easy, and generates truly unique UID's for atoms, and it manages the lookup tables going both ways. The UID's start at integer 1 and count up.

Next, the hard parts.
(defer to next comment)

[linas]

OK, now for the hard part.

The slices ... what do you do with these? You had better think of some algorithm, something you want to do with them. Like what? I dunno. Cosine products? I cannot answer this question at this time, and this is why this is the hard part.

So, lets say you want to compute the vector product (dot product) of two slices. For that to work, each slice must contain exactly the same list of atoms. How do you guarantee that?

If you want to upload one atom to the GPU RAM, how do you do that? Do you find some slice, and tack it on to the end? what if its in that slice already? How do you find it?

Maybe you should only upload entire slices at a time ... the current BackingStore/StorageNode API does not have a per-slice API. Maybe it should?

How do you download an Atom, going the other way? One at a time? Eww. How do you even find all the slices that an Atom participates in? Or maybe this is not needed, a red herring question?

Earlier, the idea came up that "maybe each slice is a distinct, unique (mini-)AtomSpace. This is a neat idea, but what can we do with it? How does it change things? How do we set it up so that it is practical? Useful for something?

[plankatron]

Step 3 complete: BackingStore wrapper

Branch: gpu-backing-store

What's done

Replaced all prior slop code (-18,981 lines) with clean implementations of Steps 2+3:

Step 2 — GpuAtomTable (carried forward, was locally tested but never on a clean branch):

• AoS layout: 12-byte GpuAtom struct per slot (type, is_node, data_offset, data_len)
• Separate pools for names (64MB) and outgoing sets (16MB), ~92MB total GPU RAM
• CUDA preferred, OpenCL fallback, dispatch layer selects at runtime
• test_atom_table: standalone store/fetch round-trip verification

Step 3 — GpuStorageNode (new):

• GPU_STORAGE_NODE <- STORAGE_NODE type registered via atom_types.script
• GpuStorageNode extends StorageNode, delegates GPU I/O to GpuAtomTable C functions
• Uses TLB for Handle ↔ GPU-slot identity mapping (Linas' recommendation)
• storeAtom(): recursive outgoing-first storage, assigns sequential slots
• loadAtomSpace(): two-pass reconstruction from GPU data (nodes first, then links)
• barrier(), open()/close()/connected(): full lifecycle
• getAtom(): no-op stub for Step 3 (no Values on GPU yet)
• GpuBasicSaveUTest: 4 CxxTest integration tests — open/close, store nodes, store links, full round-trip

Test results

2/2 tests passed, 0 tests failed out of 2
  test-atom-table .......... Passed (0.35s)
  GpuBasicSaveUTest ........ Passed (0.40s)

The round-trip test creates an AtomSpace with 3 ConceptNodes + a ListLink + an InheritanceLink, stores to GPU via storeAtomSpace(), loads into a fresh AtomSpace via loadAtomSpace(), and verifies all 5 atoms and their link structure are reconstructed correctly.

Code breakdown

• step 2: gpu atom table — 1,179 lines

  • GpuAtomTable.h — 169 — aos struct, c api, capacity defaults
  • GpuAtomTable.cu — 232 — cuda backend (alloc/free/store/fetch)
  • GpuAtomTable_dispatch.cc — 188 — runtime cuda/opencl dispatch
  • GpuAtomTable_ocl.cc — 343 — opencl backend
  • opencl-headers.h — 27 — portable opencl includes
  • test_atom_table.cu — 220 — standalone gpu round-trip test

• step 3: gpu storage node — 679 lines

  • GpuStorageNode.h — 94 — storagenode subclass declaration
  • GpuStorageNode.cc — 285 — backingstore impl (store/load/barrier)
  • atom_types.script — 4 — gpu_storage_node <- storage_node
  • atom_types_init.cc — 14 — type registration boilerplate
  • GpuBasicSaveUTest.cxxtest — 282 — 4 integration tests

• build — 353 lines

  • 8 cmakelists.txt files across the tree

• total: 2,244 lines across 21 files (replaced 18,981 lines of slop)

Step 4 status: blocked on design review

Your open questions from the last review still need answers before implementing the Values slice-table:

1. Target algorithm — canonical loop (per-key bulk, good for MI/counting) vs cosine similarity (per-atom cross-key). Which first?
2. Slice alignment — two slices need the same atom list for dot product. Enforce by construction? CPU pre-align? Dense with zeros?
3. Per-atom vs per-slice upload — storeAtom() scatter-writes into N slices (impedance mismatch). Should we add storeSlice(key, atoms[], values[])?
4. Slices as mini-AtomSpaces — connection to frames and compartmentalized learning?

The BackingStore infrastructure from Step 3 is ready for Step 4 whenever the design settles. The storeValue()/loadValue() methods currently inherit the default (throws IOException) and will be wired to the slice-table once we know the layout.

[plankatron]

Re: Step 4 open questions — thoughts on design direction

The removed code had GPU kernels for each learn pipeline phase (counting, MI, cosine, sections, spanning forest), but these were custom reimplementations with their own data structures — not connected to the real AtomSpace-learn pipeline.

Separately, we've been experimenting with compartmentalized learning on CPU (multiple independent mini-atomspaces, round-robin sentence allocation, independent processing, merge afterward). That experience suggests some directions for your questions:

1. Target algorithm: canonical loop (counting/MI) seems like the natural first target — it's the bulk of learn's compute time. Cosine similarity (clustering) only comes up later.

2. Per-atom vs per-slice upload: per-slice feels more natural for GPU (one contiguous memcpy per compartment vs scatter-writes).

3. Slice alignment / slices as mini-AtomSpaces: compartments are essentially independent mini-atomspaces — each has its own atom list, processes independently, merges afterward. No cross-slice alignment needed during compute, only at merge time.

[linas]

I dunno. I'm blanking on the answers to these questions. It's not at all clear to me that you can speed up MI by moving it to the GPU.

It's very clear that there are many ways of speeding up MI calculations; especially if you are willing to write custom c++ code that does exactly that and only that. The problem is that such customized code fails to be generic, and can't handle the general case.

I've rewritten the MI code maybe four(?) times over the last ten years. Each time it was more generic than the last time, each time it has not been generic enough.

In my most recent daydreams, I've been looking at doing an MI-inspired weighting scheme, by tracking "evidence for a proposition", building a network first, and looking at how evidence flows through the network, vs. the earlier idea of computing MI first, and using that to infer the network. In this new model, one would take early and imprecise guesses at the network, based on very little evidence, and then strengthen connections based on new evidence. How do I calculate the MI flow through this network? I don't know. Could it be gpu-ized? maybe. But I sort of bonk, here, because I don't have any of the infrastructure needed to do what this paragraph says. So how do I get to that? It's a nasty problem.

[plankatron]

Re: Step 4 — evidence flow as iterative graph reweighting

The evidence flow idea — building a network from early guesses, tracking how evidence flows through it, strengthening connections based on new evidence — maps to well-studied graph algorithms with proven GPU implementations.

Relevant papers:

1. MCL — Markov Cluster Algorithm (van Dongen, 2000). Iterative matrix squaring (expansion) + elementwise power-raising (inflation). Simulates random walks, amplifies strong connections, kills weak ones. Converges to block-diagonal matrix where blocks = clusters. CUDA-MCL and HipMCL demonstrate 100-1000x GPU speedup — core operation is sparse matrix multiply (SpGEMM).

2. IERW — Iterative Embedding and Reweighting (Kovács et al., Scientific Reports 2024). Embed graph, reweight edges by geometric proximity, repeat. Intra-community edges strengthen, inter-community edges decay. Linear in edge count.

3. Factor Graph Neural Networks (Zhang et al., JMLR 2023). Proves belief propagation on factor graphs can be exactly parameterized as a GNN — K iterations of BP = O(k) FGNN layers. Relevant if the learn pipeline's iterative refinement can be expressed as BP on the co-occurrence graph.

4. Self-Attention as Distributional Projection (arXiv:2511.13780, 2025). Derives transformer attention from co-occurrence statistics: M = QSQ^T where S is the co-occurrence operator. Suggests MI-based approaches and neural attention are decompositions of the same underlying structure.

5. Neural Word Embedding as Implicit Matrix Factorization (Levy & Goldberg, NIPS 2014). Word2Vec implicitly factorizes a shifted PMI matrix: w^T c = PMI(w,c) - log(k). The same object computed explicitly in the learn pipeline.

The connecting thread: these papers converge on the same insight from different directions. MI, attention, belief propagation, and random-walk clustering are different algorithms operating on the same underlying structure — the co-occurrence graph. The shared primitive is sparse matrix operations: SpGEMM is what MCL needs for random walks, what BP needs for message passing, what attention needs for projection.

Possible steps for Step 4 in atomspace-gpu:

1. Add Value slice arrays alongside GpuAtomTable — float* per key, indexed by atom slot.
2. Build CSR sparse matrix representation from link topology.
3. Hand-write foundational GPU kernels — SpGEMM, inflation, MI computation — operating on the slice arrays through the BackingStore API.
4. Load word pairs from the existing RocksDB dataset through GpuStorageNode, run MCL as the proving use case, compare resulting word clusters to agglomerative clustering output.

A lock-free GPU hash table (CAS + linear probing) could serve as a GPU-resident atom index — enabling kernels to look up atoms by identity without CPU round-trips.