ValueError: 'Elo' is not a valid CompletionCriteriaSettings.MeasureType

[jporubci]

Describe the bug
Elo is supposed to be a valid measure for curriculum learning, but it is not implemented.

To Reproduce
Steps to reproduce the behavior:

1. Create a training configuration file with curriculum learning, self play, and measure Elo. Here is the one I am using:

   behaviors:
     AgentBehavior:
       trainer_type: poca
       summary_freq: 50000
       time_horizon: 1024
       max_steps: 6e7
       keep_checkpoints: 5
       even_checkpoints: false
       checkpoint_interval: 500000
       threaded: false
   
       network_settings:
         hidden_units: 512
         num_layers: 3
         normalize: false
         vis_encode_type: simple
         conditioning_type: hyper
   
         # LSTM
         memory:
           memory_size: 256
           sequence_length: 128
   
       hyperparameters:
         # Common Trainer Configurations
         learning_rate: 3e-4
         batch_size: 512
         buffer_size: 16384
         learning_rate_schedule: linear
   
         # PPO/MA-POCA-specific Configurations
         beta: 5.0e-3
         epsilon: 0.2
         lambd: 0.95
         num_epoch: 3
   
       reward_signals:
         extrinsic:
           strength: 1.0
           gamma: 0.99
   
       self_play:
         save_steps: 50000
         team_change: 200000
         swap_steps: 20000
         window: 10
         play_against_latest_model_ratio: 0.5
         initial_elo: 1200.0
   
   environment_parameters:
     spawn_radius:
       curriculum:
         - name: Close
           value: 1.0
           completion_criteria:
             measure: Elo
             behavior: AgentBehavior
             threshold: 1250.0
             min_lesson_length: 50
             signal_smoothing: false
             require_reset: true
         - name: Mid
           value: 6.0
           completion_criteria:
             measure: Elo
             behavior: AgentBehavior
             threshold: 1300.0
             min_lesson_length: 50
             signal_smoothing: false
             require_reset: true
         - name: Long
           value: 12.5
           completion_criteria:
             measure: Elo
             behavior: AgentBehavior
             threshold: 1350.0
             min_lesson_length: 50
             signal_smoothing: false
             require_reset: true
         - name: Max
           value: 17.5
           completion_criteria:
             measure: Elo
             behavior: AgentBehavior
             threshold: 1400.0
             min_lesson_length: 50
             signal_smoothing: false
             require_reset: true
   
   checkpoint_settings:
     run_id: poca
     force: false
   
   engine_settings:
     width: 84
     height: 84
     quality_level: 1
     time_scale: 1
     target_frame_rate: -1
     capture_frame_rate: 60
     no_graphics: true
     no_graphics_monitor: true
   
   env_settings:
     num_envs: 1
     num_areas: 9
     seed: 42
   

2. Run mlagents-learn on this file.

Console logs / stack traces

ValueError: 'Elo' is not a valid CompletionCriteriaSettings.MeasureType

Environment (please complete the following information):

• Unity Version: Unity 6 (6000.0.25f1)
• OS + version: macOS Sequoia 15.1
• ML-Agents version: 3.0.0
• mlagents-learn version: 1.1.0
• Torch version: 2.2.2
• Environment: Custom

[jporubci]

I just realized I don't have to specify a completion criteria for the last lesson, but removing it didn't fix the problem.

[jporubci]

Related to #5646. Copying the relevant code from the dev_curriculum_with_elo branch fixed the errors. I am curious, are there plans to implement Elo as a completion criteria measure?

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale. Please open a new issue for related bugs.