Add tasks from Profet

[lebrice]

This is a draft, I will update this description in the future.

Description

• Adds tasks from OpenML (Removed, will tackle this in another PR)
• Adds tasks from Profet

TODOs

• De-couple everything out of the warmstart repo
• Add external dependency for simple_parsing
• Add a new extras_require for profet
  • Add checks so that the profet tasks are truly optional (things dont crash if the profet extras aren't installed, tests are skipped properly. **I'd actually need some help with this @bouthilx **
  • Test that all the required packages are correctly listed (e.g. by creating a blank new conda env and then running tests)
• Update the profet benchmark example script
• Tests for Profet tasks (Add tasks from Profet #632 (comment))
  • Arguments
    • benchmark
    • max_trials
    • task_id
    • seed
    • input_dir
    • checkpoint_dir
    • train_config. This one would require to mock the model trained and some functions used to train it to verify that the training config is passed down the call stack properly. No need to test all the way down, only at end of stack is fine (deepest function called where the config is actually used).
    • device
  • Behaviors
    • Testing the checkpoint based on hash. No clash, unique and reproducible.
    • self.name. Should it contain something else than task_id? Hash maybe?
    • Test seeding (test_call_is_reproducible)
    • Re-using checkpoint (test_instantiating_task)
    • configuration (test_configuration)
    • With gradients (test_with_gradients)
• Add documentation? docstrings everywhere :)
  - [ ] Tests for Quadratics task
  - [ ] Same basic tests as other tasks
  - [ ] Test with_context (Add tasks from Profet #632 (comment))

Discussions not resolved yet:

• Seeding: Add tasks from Profet #632 (comment)
• Random forest predictor: Add tasks from Profet #632 (comment)
• Citations:
  • Add tasks from Profet #632 (comment)
  • Add tasks from Profet #632 (comment)
• Getting rid of global configurations: Add tasks from Profet #632 (comment)
• Documentation of Profet Task: Add tasks from Profet #632 (comment)
• Should this be an issue? Add tasks from Profet #632 (comment)
• Seeding context manager: Add tasks from Profet #632 (comment)
• Seeding location: Add tasks from Profet #632 (comment)
• Seeding issue: Add tasks from Profet #632 (comment)
• Comment to remove: Add tasks from Profet #632 (comment)
• Discussion on support multiple types in call:
  • Add tasks from Profet #632 (comment)
  • Add tasks from Profet #632 (comment)
• Clarification? Add tasks from Profet #632 (comment)
• Quadraticts HParams, is this adressed? Add tasks from Profet #632 (comment)
• Docstring clarification: Add tasks from Profet #632 (comment)
• Potential simplification of the subspace wrapper: Add tasks from Profet #632 (comment)
• Clarification question: Add tasks from Profet #632 (comment)
• Is this fixed? Add tasks from Profet #632 (comment)
• Data test parametrization: Add tasks from Profet #632 (comment)
• Question on a test mock: Add tasks from Profet #632 (comment)
• Data download test timeout: Add tasks from Profet #632 (comment)
• Test verifying model call when trained model is cached: Add tasks from Profet #632 (comment)
• Configuration and arguments testing: Add tasks from Profet #632 (comment)
• Forrester documentation: Add tasks from Profet #632 (comment)

[bouthilx]

Thanks for going through all this work to add Profet!