Skip to content

v0.2.0 Stage C checkpoint package

Latest

Choose a tag to compare

@reshuibuduo reshuibuduo released this 06 Jun 06:48

TMCRA TokenGraph-LLM Stage C / Dynamic Token Graph Decoder V3

This release provides the Stage C public model package for TMCRA TokenGraph-LLM. Release assets are model artifacts only. Algorithm source, graph builders, full-chain training scripts, and evaluation templates are maintained in the GitHub source repository on main.

Source repository: https://github.com/reshuibuduo/TMCRA-TokenGraph-LLM
Hugging Face model page: https://huggingface.co/2009YU/TMCRA-TokenGraph-LLM

Model

  • Asset: tmcra_tokengraph_stagec_model_package_20260606.zip
  • SHA256: cc23285628eaed47c20009b6be6b5eb0600ded57ac2e09519370d97158fecd33
  • Checkpoint: token_graph_dynamic_decoder_v3.pt
  • Parameters: about 114.6M
  • Architecture: dim=512, graph_layers=8, decoder_layers=10, untied embeddings
  • Training: bf16, about 1.03M effective token-graph samples, 62,000 steps

Package contents

  • checkpoint/token_graph_dynamic_decoder_v3.pt
  • dataset_metadata/tokenizer.json
  • dataset_metadata/manifest.json
  • training_summary_stagec_public.json
  • docs/ technical overview and smoke benchmark notes
  • SHA256SUMS.txt

Full-chain source update

The source repository now includes:

  • open-corpus schema2 conversion scripts;
  • optional semantic teacher annotation scripts;
  • token graph dataset builders;
  • Stage C full-chain and sharded training templates;
  • graph ablation and token attribution evaluation.

Start with:

  • docs/FULL_CHAIN_TRAINING.md
  • docs/FULL_CHAIN_TRAINING_ZH.md
  • scripts/run_stagec_full_chain_template.sh
  • scripts/run_stagec_sharded_training_template.sh

Smoke results

  • Stage C normal: total loss 6.512117, lm loss 4.641285
  • Stage C no_edges: total loss 8.310654, lm loss 5.790666
  • Stage C shuffle_edges: total loss 7.702783, lm loss 5.169387
  • TinyStories smoke: normal avg words 73.88
  • BLiMP smoke: 59%-64% on three small likelihood subsets

These are smoke tests, not leaderboard claims. Stage C is still an early graph-native language model prototype, not a production LLM.

The previous v0.1.0-prototype release is now considered the legacy small prototype checkpoint.