Skip to content

Arctic-Platform v0.2.0

Choose a tag to compare

@github-actions github-actions released this 03 Nov 06:17
· 87 commits to main since this release

What's Changed

  • bump v0.1.5.dev0 by @sfc-gh-mwyatt in #123
  • Add shared PEFT wrapping and an ordered model patch by @sfc-gh-abkulkarni in #124
  • Add class-based loss registry and grouped distillation by @sfc-gh-thonguyen in #130
  • Port shared MoE runtime and reconcile Qwen3.5 loading by @sfc-gh-abkulkarni in #125
  • Automated releases by @sfc-gh-mwyatt in #135
  • Move Qwen dense model setup into Arctic Platform by @sfc-gh-abkulkarni in #126
  • Onboard GLM MoE models in Arctic Platform by @sfc-gh-abkulkarni in #136
  • Move Arctic Inference into arctic_platform.inference by @sfc-gh-abkulkarni in #137
  • Migrate DSS GRPO loss contracts to Arctic Platform by @sfc-gh-thonguyen in #138
  • Drop the cortex-training dependency by @sfc-gh-abkulkarni in #143
  • Remove stale scripts from copied Arctic-Inference code by @sfc-gh-mwyatt in #144
  • skyrl/cortex: use FlashAttention 3 instead of sdpa by @sfc-gh-kganesan in #145
  • Remove more unused files from Arctic-Inference copy by @sfc-gh-mwyatt in #147
  • Opt-in fixes: packed seq-mean-* per sequence; padded sequence-IS advantages over the loss mask- #66 by @sfc-gh-sbekman in #134
  • cortex: route forward (no-grad log-probs) through the /operation envelope by @sfc-gh-kganesan in #95
  • Keep unconfirmed stream cleanup visible after the stream retires by @sfc-gh-rsureshbabu in #150
  • Deliver stream events in batches and merge undelivered deltas by @sfc-gh-rsureshbabu in #151
  • Remove HTTP on-prem server and client by @sfc-gh-mwyatt in #155
  • Simplify on-prem now that Ray is the only option by @sfc-gh-mwyatt in #157
  • router-replay: let trainers route rows without a capture instead of failing the exchange by @sfc-gh-thonguyen in #158
  • Onboard additional MoE model families by @sfc-gh-abkulkarni in #140
  • Reject DFlash drafts that fall outside the structured-output grammar by @sfc-gh-abkulkarni in #159
  • Apply DFlash2 NaN fixes without Arctic optimizations by @sfc-gh-abkulkarni in #146
  • Gather Qwen3.8 QSA keys before compression under context parallelism by @sfc-gh-abkulkarni in #182
  • Run the streaming CPU tests in CI by @sfc-gh-rsureshbabu in #154
  • Add logit_bias, JSON structured output, thinking budget and logprobs to streaming by @sfc-gh-rsureshbabu in #152
  • Upgrade Arctic Inference to vLLM 0.31.0 by @sfc-gh-abkulkarni in #197
  • Clear the sampler prefix cache on keep-mode weight syncs by default by @sfc-gh-thonguyen in #174
  • Complete AP model ownership contracts by @sfc-gh-abkulkarni in #173
  • Stream chat prompts rendered with the model's own template by @sfc-gh-rsureshbabu in #153
  • Optional post-mask signed-mass restoration (ratio_mask_rebalance) by @seanbell in #164
  • Reject unchecked DFlash drafts in Model Runner V2 too by @seanbell in #194
  • Keep GLM-5.3-Flash indexer pages legal for DeepGEMM by @sfc-gh-abkulkarni in #206
  • Split rl/processors batch/meta into separate model-forward and loss-context bags by @sfc-gh-truwase in #131
  • Preserve high expert IDs and deliver router replay to GLM5Next/Qwen4Exp MoEs by @seanbell in #186
  • Retain frozen n-gram tables in explicit HF checkpoint exports by @seanbell in #191
  • Keep Qwen3.8 PLE history inside logical packed rows by @seanbell in #188
  • Add reusable cluster API foundations by @sfc-gh-mwyatt in #207
  • Route Qwen3.8 PLE embedding lookups to owning EP ranks by @sfc-gh-abkulkarni in #212
  • Keep Qwen3.8 sparse-attention compression and routing within packed rows by @seanbell in #195
  • Keep GLM-5.3 packed attention within logical rows by @sfc-gh-abkulkarni in #213
  • tinker: run cookbook recipes by importing arctic_platform.integrations.tinker by @sfc-gh-kganesan in #110
  • Add a trainer-lifecycle callback contract by @sfc-gh-truwase in #181
  • Simplify the Cortex transport into one route table and one send path by @sfc-gh-mwyatt in #216
  • Attach the engine console tail to a failed vLLM engine startup by @seanbell in #196
  • Add Cortex job, capacity, and checkpoint operations by @sfc-gh-mwyatt in #217
  • Fix GLM-5.3 native FP8 TP weight synchronization by @sfc-gh-abkulkarni in #223
  • Wire Nemotron sequence parallelism through Mamba context parallel by @sfc-gh-abkulkarni in #221
  • Accept token-id prompts as 1-D integer tensors in /generate by @sfc-gh-asu in #228
  • Add a model option registry, per-loader profiles and a coverage check by @sfc-gh-thonguyen in #193
  • Match vLLM's fp32 precision in three Qwen3.5 trainer operations, behind a switch by @sfc-gh-thonguyen in #202
  • Port the multi-replica semi-p server into arctic_platform.inference by @sfc-gh-mhidayetoglu in #149
  • Extend the draft max_position_embeddings override to DFlash by @seanbell in #201
  • Add the DSS token-budget packer as an uncalled common library by @sfc-gh-truwase in #148
  • Resolve PEFT-wrapped models before attention-mask routing by @sfc-gh-abkulkarni in #234
  • Add Cortex checkpoint load, live log tail, and router replay to the unified client by @sfc-gh-mwyatt in #229
  • Bound GLM-5.3 sparse-indexer score memory by @sfc-gh-sbekman in #249
  • Avoid retained FP32 MoE routing activations by @sfc-gh-sbekman in #252
  • Add Cortex streaming generate to the unified client by @sfc-gh-mwyatt in #254
  • Fix GLM-5.3 sparse attention dispatch and tiled backward by @sfc-gh-sbekman in #251
  • Run GLM-5.3 head-parallel KDA through causal-conv1d and FLA kernels by @sfc-gh-abkulkarni in #253
  • Own the generic weight-sync fallback and emit checkpoint names by @sfc-gh-abkulkarni in #227
  • SkyRL on Cortex: CPU driver, long context, and placement by @sfc-gh-kganesan in #156
  • Harness chat endpoint for SkyRL on Cortex by @sfc-gh-kganesan in #210
  • Append the engine process's own startup failure to a failed vLLM engine startup by @sfc-gh-wesong in #233
  • add kv_group_size control by @sfc-gh-jaelee in #225
  • Own expert-parallel setup, model specs, and the training engine by @sfc-gh-abkulkarni in #214
  • Round the GatedDeltaNet output norm once, as vLLM does by @sfc-gh-wenli in #256
  • Pass dense loader options through as one dictionary by @sfc-gh-abkulkarni in #267
  • Train DeepSeek-V4-Flash with its sparse attention stack by @sfc-gh-abkulkarni in #250
  • Add post_fwd_only_microbatch to the trainer Callback contract by @sfc-gh-truwase in #230
  • [debug] Add a retained per-rank GPU memory safety reserve by @sfc-gh-sbekman in #245
  • Preserve Persistent N-Gram Buffers In MoE Weight Exports by @sfc-gh-sbekman in #268
  • Add text_config_overrides: one rope/config override shared by the trainer and the sampler by @seanbell in #199
  • mRoPE+YaRN: compute the correction range from the original length (equivalent to vllm#58879) by @seanbell in #200
  • Refresh retired Mamba checkpoints through the completion release (hybrid prefix-cache cliff) by @seanbell in #178
  • Fix Cortex client transport correctness by @sfc-gh-mwyatt in #258
  • Close remaining Cortex client parity gaps by @sfc-gh-mwyatt in #260
  • GLM-5.3-Flash optimization by @sfc-gh-jaelee in #269
  • new liger-kernel ver broke a test by @sfc-gh-sbekman in #274
  • Move router-stats into AP with Reduce.SUM and public façades by @sfc-gh-truwase in #231
  • GRPO: sampler-anchored KL term (sampler_kl_coef; k1/k2/k3 estimators, ratio cap, per-token weights) by @seanbell in #276
  • Don't block the Ray transport loop on training RPCs by @sfc-gh-truwase in #104
  • Update CODEOWNERS by @sfc-gh-jrasley in #295
  • Send model_provider arctic and request Liger kernels with model_options.liger by @sfc-gh-abkulkarni in #219
  • Add the ArcticTraining FLOPs counter by @sfc-gh-abkulkarni in #285
  • GRPO: accept sampler_kl_weights as one weight per sequence by @seanbell in #288
  • Keep a near-certain target's logprob and gradient in the chunked LM heads (softmax collapse) by @seanbell in #296
  • Point on-prem sampling at the in-tree Arctic Inference pool by @sfc-gh-truwase in #261
  • Move optimizer normalization into arctic_platform.common; forward sampling TP to Cortex by @sfc-gh-mwyatt in #275
  • Keep near-certain grouped log-probs and their distillation gradient (softmax collapse and cancellation) by @seanbell in #297
  • deepspeed dep update by @sfc-gh-sbekman in #272
  • semi-p: Ray logs, pinned double buffer, onboarding verdicts by @sfc-gh-mhidayetoglu in #278
  • semi-p: serve restored engines through AsyncLLM by @sfc-gh-mhidayetoglu in #290
  • [fix] Restore LM-head gradient reduction during tiled backward by @sfc-gh-sbekman in #198
  • Upgrade the inference extra to vLLM 0.32.0 by @sfc-gh-abkulkarni in #273
  • Keep a near-certain label's logprob and gradient in the unchunked compute_logprobs path (softmax collapse) by @seanbell in #323
  • Train DeepSeek-V4.1-Flash text and LoRA on the official DeepSeek-V4/V4.1-Flash checkpoints, and sample the adapters in vLLM for RL by @sfc-gh-abkulkarni in #280
  • Remove unused cortex-client by @sfc-gh-mwyatt in #324
  • Keep the server's error message on Cortex HTTP errors by @sfc-gh-mwyatt in #314
  • feat(prime-rl): add prime-rl → Cortex launcher integration by @sfc-gh-boliu in #220
  • DFlash2 train support by @sfc-gh-jaelee in #237
  • Refresh GLM-5.3 KDA's cached short-conv weight after every weight sync by @seanbell in #343
  • Keep _dss_weight_export_buffers out of weight sync (Qwen4Exp n-gram buffers fail the first sync after #268) by @seanbell in #345
  • Qwen3.8: sum the QSA attention backward's key/value gradients in fp32 by @seanbell in #349
  • Export Qwen3.8's EP-sharded n-gram table through rank 0's host memory (weights-only save OOMs on every GPU) by @seanbell in #350
  • Fix the hybrid state KV block zeroer for packed caches; rename dflash2_nan_fix to hybrid_state_fixes by @sfc-gh-abkulkarni in #363
  • Document and test the existing gradient accumulation precision contract by @sfc-gh-sbekman in #341
  • Round GLM-5.3 norms and Qwen3.8-Flash-Next QSA rotary, attention and gate once, from fp32, as the sampler does by @seanbell in #357
  • GLM-5.3 (+ Flash-Next GDN beta): compute router logits, the mHC remix and the KDA beta gate in fp32, as the sampler does by @seanbell in #354
  • GLM-5.3: keep the checkpoint's FP32 mHC base/scale and KDA A_log/dt_bias FP32 through DeepSpeed by @seanbell in #353
  • Weight sync: hold one rebuilt oversized tensor at a time on each sampler rank (two whole copies today) by @seanbell in #355
  • Chunked LM heads: sum bf16 tiles in fp32 when fp32_lm_head is false (weight and hidden-state gradients) by @seanbell in #367
  • Router replay: let a receiver take only a token slice of each capture (sequence-parallel ranks receive sp_size x the routing they use) by @seanbell in #348
  • Write the LoRA adapter pack under activation checkpointing (weights-only checkpoints saved 0 adapter tensors on PEFT 0.21) by @seanbell in #372
  • GLM-5.3-Flash LoRA: map fused_qkv_a_proj so q_a/kv_a adapters reach the sampler; refuse kv_b_proj LoRA on MLA by @seanbell in #351
  • Apply the existing GLM optimizer policy to GLM-5.3 by @sfc-gh-sbekman in #333
  • Expose speculative-decoding acceptance and preemption counters from sampling by @seanbell in #301
  • Qwen 3.8 Flash Next LoRA optimization by @sfc-gh-jaelee in #315
  • Qwen3.8 Flash Next full FT optimization by @sfc-gh-jaelee in #316
  • Qwen3.8-Flash-Next: compute the hyper-connection mix and injection in fp32 and round once, as the sampler does by @seanbell in #375
  • GLM 5.3 Flash Full FT optimization by @sfc-gh-jaelee in #317
  • GLM MoE DSA loader: accept debug.full_determinism (the fixed-order combine could not be requested on GLM-5.2) by @seanbell in #366
  • Qwen3.8: keep the QSA key gradient at sp 1 (allocate dK/dV buffers contiguous) by @seanbell in #359
  • Exclude the Qwen3.8 PLE norm weights from weight decay by @seanbell in #360
  • DeepSpeed worker: default grad_accum_dtype per key, not per block by @sfc-gh-sbekman in #374
  • Apply Gemma's logit softcap in the chunked LM head and accumulate its hidden gradient in float32 by @sfc-gh-abkulkarni in #310
  • Run FusedAdam for GLM-MoE models by default (torch AdamW keeps bf16 moments under bf16 masters) by @sfc-gh-abkulkarni in #383
  • Weight GLM-5.3-Flash and Qwen3.8-Flash-Next expert rows on the fp32 down-projection accumulator and round once, as the sampler does by @seanbell in #373
  • Add Harbor RL integration (arctic_platform/integrations/harbor) and recipe (recipes/rl/harbor) by @sfc-gh-kganesan in #294
  • Router self-replay: drop a failed call's queued experts on the replay-off path too (one failed forward broke every later call) by @seanbell in #365
  • Clone Arctic-Platform main in the code-localization recipe by @sfc-gh-kganesan in #387
  • Harbor: make the README's setup, SWE-smith export, and harbor run commands work from a fresh clone by @sfc-gh-kganesan in #389
  • OpenHands code localization: install [sft,cortex] on the driver, not [rl] by @sfc-gh-kganesan in #391
  • Harbor RL on Snowflake sandboxes, end to end: base_image, errored rollouts, sandbox reliability, session renewal, and a pre-GPU task check by @sfc-gh-kganesan in #404
  • Apply routing scores in the MXFP4 expert forward by @sfc-gh-abkulkarni in #406
  • Harbor RL: evaluate a trained model with harbor run, train on any Harbor task set, and a lighter driver by @sfc-gh-kganesan in #405
  • Add the packed-sequence weighted logprob loss by @sfc-gh-truwase in #262
  • Fix DeepSeek V4.1 HF checkpoint export and keep its FP32 weights in FP32 by @sfc-gh-abkulkarni in #415
  • Keep TP ranks in step during speculative decoding (Model Runner V2) by @sfc-gh-wenli in #318
  • Add the TRL training client and server loss by @sfc-gh-truwase in #264
  • Finish each EP export collective on non-master ranks before queuing the next by @sfc-gh-wenli in #421
  • Gather EP export tensors as bytes so e8m0 and packed fp4 scales reach the checkpoint by @sfc-gh-abkulkarni in #422

New Contributors

Full Changelog: v0.1.4...v0.2.0