How to resume wandb runs?

[GF-Huang]

Now, even if I use --agent.resume True for training, I still got multi runs on wandb. And I ask AI for the source of mjlab, it tell me wandb.init(...) not use the resume param. Any other ways to use resume for wandb?

reference: https://docs.wandb.ai/models/runs/resuming

[CasperKristiansson]

--agent.resume True resumes the training checkpoint, not the W&B run. In current mjlab, that flag only selects and loads a checkpoint; RSL-RL 5.4.2 then calls wandb.init() without an id or resume, so W&B creates a new run.

Set W&B's run ID and resume mode in the environment when restarting:

WANDB_RUN_ID=<existing-run-id> WANDB_RESUME=must \
  uv run train <task> --agent.resume True

Use the ID from the existing W&B run, not its display name. If the checkpoint is only in W&B, add --wandb-run-path <entity>/<project>/<run-id> as well; that chooses the checkpoint, while the two environment variables choose the W&B run.

[GF-Huang]

I'm on Windows, and not use uv, I use command train.exe <task> ... to run, can this WANDB_RUN_ID=<existing-run-id> WANDB_RESUME=must train.exe <task> --agent.resume True works on Windows?

[CasperKristiansson]

Yes. W&B uses process environment variables, so train.exe works; the inline syntax I showed was for bash/zsh. In PowerShell, use:

$env:WANDB_RUN_ID = "<existing-run-id>"
$env:WANDB_RESUME = "must"
train.exe <task> --agent.resume True

[GF-Huang]

Thank you so much.

[GF-Huang]

The only downside is that adding/changing --agent.max-iterations triggers an error.

wandb: ERROR Attempted to change value of key "train_cfg" from {'clip_actions': None, 'load_run': '.*', 'max_iterations': 50, 'wandb_tags': [], 'multi_gpu': None, 'algorithm': {'max_grad_norm': 1, 'entropy_coef': 0.005, 'lam': 0.95, 'num_mini_batches': 4, 'rnd_cfg': None, 'symmetry_cfg': None, 'desired_kl': 0.01, 'schedule': 'adaptive', 'use_clipped_value_loss': True, 'normalize_advantage_per_mini_batch': True, 'value_loss_coef': 1, 'gamma': 0.99, 'num_learning_epochs': 5, 'clip_param': 0.2, 'learning_rate': 0.001, 'optimizer': 'adam'}, 'resume': True, 'upload_model': True, 'critic': {'activation': 'elu', 'hidden_dims': [32, 32], 'obs_normalization': True}, 'load_checkpoint': 'model_.*.pt', 'seed': 42, 'save_interval': 50, 'run_name': '', 'actor': {'distribution_cfg': {'init_std': 1}, 'obs_normalization': True, 'activation': 'elu', 'hidden_dims': [32, 32]}, 'experiment_name': 'arm_reach', 'class_name': 'OnPolicyRunner', 'num_steps_per_env': 400, 'wandb_project': 'Arm-Reach-Small-Reset-Noise', 'logger': 'wandb', 'obs_groups': {'critic': ['critic'], 'actor': ['actor']}} to {'seed': 42, 'num_steps_per_env': 400, 'max_iterations': 100, 'obs_groups': {'actor': ['actor'], 'critic': ['critic']}, 'save_interval': 50, 'experiment_name': 'arm_reach', 'run_name': '', 'logger': 'wandb', 'wandb_project': 'Arm-Reach-Small-Reset-Noise', 'wandb_tags': [], 'resume': True, 'load_run': '.*', 'load_checkpoint': 'model_.*.pt', 'clip_actions': None, 'upload_model': True, 'class_name': 'OnPolicyRunner', 'actor': {'hidden_dims': [32, 32], 'activation': 'elu', 'obs_normalization': True, 'distribution_cfg': {'init_std': 1.0}}, 'critic': {'hidden_dims': [32, 32], 'activation': 'elu', 'obs_normalization': True}, 'algorithm': {'num_learning_epochs': 5, 'num_mini_batches': 4, 'learning_rate': 0.001, 'schedule': 'adaptive', 'gamma': 0.99, 'lam': 0.95, 'entropy_coef': 0.005, 'desired_kl': 0.01, 'max_grad_norm': 1.0, 'value_loss_coef': 1.0, 'use_clipped_value_loss': True, 'clip_param': 0.2, 'normalize_advantage_per_mini_batch': True, 'optimizer': 'adam', 'rnd_cfg': None, 'symmetry_cfg': None}, 'multi_gpu': None}
wandb: ERROR If you really want to do this, pass allow_val_change=True to config.update()
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\Huang\miniconda3\envs\MuJoCo\Scripts\train.exe\__main__.py", line 5, in <module>
    sys.exit(main())
             ~~~~^^
  File "C:\Users\Huang\miniconda3\envs\MuJoCo\Lib\site-packages\mjlab\scripts\train.py", line 254, in main
    launch_training(task_id=chosen_task, args=args)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Huang\miniconda3\envs\MuJoCo\Lib\site-packages\mjlab\scripts\train.py", line 203, in launch_training
    run_train(task_id, args, log_dir)
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Huang\miniconda3\envs\MuJoCo\Lib\site-packages\mjlab\scripts\train.py", line 175, in run_train
    runner.learn(
    ~~~~~~~~~~~~^
      num_learning_iterations=cfg.agent.max_iterations, init_at_random_ep_len=True
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "C:\Users\Huang\miniconda3\envs\MuJoCo\Lib\site-packages\rsl_rl\runners\on_policy_runner.py", line 74, in learn
    self.logger.init_logging_writer()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "C:\Users\Huang\miniconda3\envs\MuJoCo\Lib\site-packages\rsl_rl\utils\logger.py", line 117, in init_logging_writer
    self.writer.store_config(self.env_cfg, self.cfg)
    ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Huang\miniconda3\envs\MuJoCo\Lib\site-packages\rsl_rl\utils\wandb_log_writer.py", line 65, in store_config
    wandb.config.update({"train_cfg": train_cfg})
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Huang\miniconda3\envs\MuJoCo\Lib\site-packages\wandb\sdk\wandb_config.py", line 188, in update
    sanitized = self._update(d, allow_val_change)
  File "C:\Users\Huang\miniconda3\envs\MuJoCo\Lib\site-packages\wandb\sdk\wandb_config.py", line 181, in _update
    sanitized = self._sanitize_dict(
        parsed_dict, allow_val_change, ignore_keys=locked_keys
    )
  File "C:\Users\Huang\miniconda3\envs\MuJoCo\Lib\site-packages\wandb\sdk\wandb_config.py", line 268, in _sanitize_dict
    k, v = self._sanitize(k, v, allow_val_change)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Huang\miniconda3\envs\MuJoCo\Lib\site-packages\wandb\sdk\wandb_config.py", line 292, in _sanitize
    raise config_util.ConfigError(
    ...<4 lines>...
    )
wandb.sdk.lib.config_util.ConfigError: Attempted to change value of key "train_cfg" from {'clip_actions': None, 'load_run': '.*', 'max_iterations': 50, 'wandb_tags': [], 'multi_gpu': None, 'algorithm': {'max_grad_norm': 1, 'entropy_coef': 0.005, 'lam': 0.95, 'num_mini_batches': 4, 'rnd_cfg': None, 'symmetry_cfg': None, 'desired_kl': 0.01, 'schedule': 'adaptive', 'use_clipped_value_loss': True, 'normalize_advantage_per_mini_batch': True, 'value_loss_coef': 1, 'gamma': 0.99, 'num_learning_epochs': 5, 'clip_param': 0.2, 'learning_rate': 0.001, 'optimizer': 'adam'}, 'resume': True, 'upload_model': True, 'critic': {'activation': 'elu', 'hidden_dims': [32, 32], 'obs_normalization': True}, 'load_checkpoint': 'model_.*.pt', 'seed': 42, 'save_interval': 50, 'run_name': '', 'actor': {'distribution_cfg': {'init_std': 1}, 'obs_normalization': True, 'activation': 'elu', 'hidden_dims': [32, 32]}, 'experiment_name': 'arm_reach', 'class_name': 'OnPolicyRunner', 'num_steps_per_env': 400, 'wandb_project': 'Arm-Reach-Small-Reset-Noise', 'logger': 'wandb', 'obs_groups': {'critic': ['critic'], 'actor': ['actor']}} to {'seed': 42, 'num_steps_per_env': 400, 'max_iterations': 100, 'obs_groups': {'actor': ['actor'], 'critic': ['critic']}, 'save_interval': 50, 'experiment_name': 'arm_reach', 'run_name': '', 'logger': 'wandb', 'wandb_project': 'Arm-Reach-Small-Reset-Noise', 'wandb_tags': [], 'resume': True, 'load_run': '.*', 'load_checkpoint': 'model_.*.pt', 'clip_actions': None, 'upload_model': True, 'class_name': 'OnPolicyRunner', 'actor': {'hidden_dims': [32, 32], 'activation': 'elu', 'obs_normalization': True, 'distribution_cfg': {'init_std': 1.0}}, 'critic': {'hidden_dims': [32, 32], 'activation': 'elu', 'obs_normalization': True}, 'algorithm': {'num_learning_epochs': 5, 'num_mini_batches': 4, 'learning_rate': 0.001, 'schedule': 'adaptive', 'gamma': 0.99, 'lam': 0.95, 'entropy_coef': 0.005, 'desired_kl': 0.01, 'max_grad_norm': 1.0, 'value_loss_coef': 1.0, 'use_clipped_value_loss': True, 'clip_param': 0.2, 'normalize_advantage_per_mini_batch': True, 'optimizer': 'adam', 'rnd_cfg': None, 'symmetry_cfg': None}, 'multi_gpu': None}
If you really want to do this, pass allow_val_change=True to config.update()

[GF-Huang]

I asked AI, it said that I can't change anything of RslRlOnPolicyRunnerCfg and ManagerBasedRlEnvCfg; otherwise, it will triggers above errors.