Training

Training

Trainer<G> drives the CFR / MCCFR iterations over a Game and produces a Profile. One training iteration updates every player. For the target player, the trainer traverses the tree (dispatch decided by Algorithm). It accumulates counterfactual regret and an average-strategy sum into tables keyed by player-namespaced information set.

Building a trainer

let mut trainer = Trainer::new(game)
    .algorithm(Algorithm::ExternalSampling)
    .regret(RegretVariant::DCFRPlus { alpha: 1.5, gamma: 4.0 })
    .seed(0xABCD);
trainer.train(20_000).unwrap();
let profile = trainer.profile();

Trainer::new(game) uses defaults: external-sampling MCCFR with DCFR+, workers = available_parallelism(). Changing the algorithm, regret variant, pruning mode, or seed resets the accumulated tables. Mixing a new accumulation semantics with stale statistics would be wrong.

The single-call Profile::train(game, iters) covers the common case.

Feature flags

regret exposes three Cargo feature flags that change what the trainer stores or how it runs.

• parallel (on by default): train launches workers rayon tasks, as described in Parallelism.
• f32-tables: stores the average-strategy and regret accumulators as f32 instead of f64. The accumulation math still runs in f64; only the persisted tables are narrowed, which roughly halves table memory at the cost of rounding in the stored accumulators.
• deep: builds the separate, single-threaded deep trainer. See Deep-CFR.md.

Sampling algorithms (Algorithm)

• ExternalSampling (default): samples all opponent and chance nodes; traverses every action of the investigating player. General-purpose workhorse for large games (Lanctot et al. 2009).
• ChanceSampling: samples chance; traverses every action of all players. Lower variance for games with little chance.
• OutcomeSampling: samples a single full trajectory with importance weighting. It stores one trajectory rather than the whole tree, so memory use is lower, but per-sample variance is higher and each step is cheap. Uses an epsilon-greedy behavior policy (exploration_rate, default 0.6).
• FullTree: exact full-tree CFR, no sampling. Reference baseline for small games.

All enums are #[non_exhaustive]: match with a _ arm downstream.

Regret / averaging variants (RegretVariant)

Each variant is composable on top of any sampling algorithm. Defaults to DCFRPlus { alpha: 1.5, gamma: 4.0 } (Xu et al. 2022, AutoCFR, Eq. 5).

• Vanilla: plain CFR (Zinkevich et al. 2007).
• CFRPlus: CFR+ (Tammelin 2014): floor regrets at zero, linearly increasing average-strategy weight.
• LinearCFR: Linear CFR (Brown & Sandholm 2019): weight every iteration by t in both regrets and the average; equivalent to DCFR { 1, 1, 1 }.
• DCFR { alpha, beta, gamma }: Discounted CFR (Brown & Sandholm 2019): discount positive/negative regrets by t^α/(t^α+1) and t^β/(t^β+1); average weight t^γ. Recommended (3/2, 0, 2).
• DCFRPlus { alpha, gamma }: DCFR discounting plus the CFR+ zero-floor on regrets, average weight t^γ (recommended γ = 4). The discount denominator is (t^α + 1), not (t^α + α). The +alpha form is a known typo in parts of the literature.
• PDCFRPlus { alpha, gamma }: Predictive Discounted CFR+ (Xu et al. 2024, IJCAI): DCFR+ with optimistic (predictive) regret matching; computes the next strategy from a one-step-ahead predicted regret. Recommended (2.3, 5).
• HSDCFR { gamma_init, total } / HSCFRPlus { gamma_init, total }: Hyperparameter-Schedule DCFR / CFR+ (Zhang et al. 2024), scheduling exponents as a function of the current iteration and a known total budget.

Regret matching maps a regret vector to a strategy: σ(a) = R⁺(a) / Σ_b R⁺(b), uniform when the sum is non-positive. R⁺(a) = max(0, R(a)). The free function regret::regret_matching(regret, out) writes this mapping into out if you need it directly.

Pruning and variance reduction

• Pruning::None (default).
• Pruning::Safe: regret-based safe pruning (Brown & Sandholm 2015/2019): prunes information sets whose average strategy puts (near-)zero mass on an action and whose regret is negative. Never changes the fixed point, so it is safe to flip on. .safe_pruning() is the one-call opt-in.
• Pruning::Probabilistic { warmup, probability }: Pluribus-style: after warmup iterations, skip a near-zero-mass action with probability probability. Faster than Safe but not provably safe.
• VarianceReduction::ControlVariate (VR-MCCFR, Bošanský et al.): only affects OutcomeSampling: center each sampled return on a per-information-set running mean, then add the baseline back. Unbiased, lower variance.

chance_samples(k) does multi-chance sampling (k re-rolls per chance node), a variance-reduction trick; 1 disables it.

Parallelism

With feature = "parallel" (on by default), train launches workers rayon tasks. Each worker trains its own sharded table from an empty start under an independent, well-scrambled seed (seed ^ w*MIX), runs the full iteration budget, and its average-strategy accumulators are folded back (averaging K independent runs, the faithful, variance-reducing way to parallelize CFR). The regret / predicted buffers are not merged.

Why per-worker tables

The regret buffer is non-linear (the discount multiplies the entire previous buffer), so concurrent updates from different global iterations interleave and corrupt the recurrence. Per-worker buffers keep every algorithm family numerically identical to a serial run.

train vs train_split

With workers = W, train(N) launches W independent runs of the full N budget. It bags W runs. So train(N) does W*N compute, but the DCFR discount / average-weight timeline advances by exactly N, not W*N. iterations_done moves by N. Wall-clock speedup is a Wx reduction in variance at a fixed discount schedule, not a Wx speedup of a fixed budget.

train_split(total, W) divides total across W non-overlapping windows, so the workers together perform exactly total iterations and average, a true Wx wall-clock speedup at a fixed total budget.

The strategy returned by train(N) is not bit-identical between parallel and serial builds (parallel averages W runs; serial averages one). Use a fixed workers = 1 in both, or train_split with a fixed worker count, for bit-stable results.

Determinism

Reproducible for a fixed workers count: same seed + same total budget + same worker count reproduces the same bits. It is not guaranteed bit-for-bit identical between different worker counts (floating-point addition is not associative, and rayon's reduce fold order differs).

Incomplete regret_table() after parallel training

Only the avg accumulator is folded back. The per-worker regret / predicted buffers are not merged, so Trainer::regret_table after any parallel call reflects only whatever the local shard held, treat it as stale. Use Trainer::profile for the accurate average strategy.

Boundary

This rayon sharding is the only place the crate parallelizes. DeepTrainer (src/deep/deep.rs) and regret::subgame re-solving (Blueprint training + solve_subgame*) are single-threaded; parallelize those by running independent processes and ensembling their output profiles.

Curriculum blending (blend_opponent)

blend_opponent(player, policy, schedule) / blend_all(...) blends a scripted/heuristic/random opponent policy into training for player. At global iteration t the player's action is drawn from policy with probability schedule.value(t), else from the current CFR strategy. Schedule is Constant(c), Geometric { start, rate }, or Linear { start, end, iters }.

Blending is not a Nash solver

It solves a modified game in which the blended player may act via policy. The average strategy at a fully-decayed blend rejoins the original fixed point, but while value(t) > 0 it is not a Nash equilibrium of the original game, so exploitability_* numbers against the original game are a sensitivity signal, not a convergence proof. Setting a blend resets the accumulated tables.

Progress hook

on_iteration(cb) registers a callback invoked once per train / train_split window with the new global iteration count and the current average-strategy Profile. It is not called per worker or per internal iteration and costs nothing when unset.