Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JIT CSE Optimization - Add a gymnasium environment for reinforcement learning #101856

Merged
merged 78 commits into from
May 9, 2024
Merged

JIT CSE Optimization - Add a gymnasium environment for reinforcement learning #101856

merged 78 commits into from
May 9, 2024

Commits on May 8, 2024

  1. Initial code

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    9d940a2 View commit details
    Browse the repository at this point in the history

  2. Add notes

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    0c31d02 View commit details
    Browse the repository at this point in the history

  3. Add CSE_HeuristicRLHook

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    8a5acd5 View commit details
    Browse the repository at this point in the history

  4. Move metric print location, double -> int

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    476063a View commit details
    Browse the repository at this point in the history

  5. Produce non-viable entries, fix output issue

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    5de67e1 View commit details
    Browse the repository at this point in the history

  6. Shuffle features by type

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    64b4821 View commit details
    Browse the repository at this point in the history

  7. Initial JitEnv - not yet working

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    0aa1d8d View commit details
    Browse the repository at this point in the history

  8. Change to snake_case

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    4f9fe2a View commit details
    Browse the repository at this point in the history

  9. Initial RL implementation with stable-baselines3

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    fe5d334 View commit details
    Browse the repository at this point in the history

  10. Enable parallel processing, fix some errors

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    dd762f8 View commit details
    Browse the repository at this point in the history

  11. Clean up train.py, allow algorithm selection

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    690c4c4 View commit details
    Browse the repository at this point in the history

  12. Fix paths

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    b8fd437 View commit details
    Browse the repository at this point in the history

  13. Fix issue with null result

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    8ee8a0f View commit details
    Browse the repository at this point in the history

  14. Save method indexes

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    313788d View commit details
    Browse the repository at this point in the history

  15. Check if process is still running

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    2d9e02f View commit details
    Browse the repository at this point in the history

  16. Up argument count before warning

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    39a46b5 View commit details
    Browse the repository at this point in the history

  17. Track more statistics on tensorboard

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    11b4346 View commit details
    Browse the repository at this point in the history

  18. Fix an issue where we didn't let the model know it shouldn't pick som… 

    …ething
    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    678c5ee View commit details
    Browse the repository at this point in the history

  19. Reward improvements 

    - Scale up rewards.
- Clamp rewards to [-1, 1]
- Reward/penalize when complete if there are better/worse CSEs (this is very slow)
- Reward when complete based on whether we beat the heuristic or not
    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    5d5f47e View commit details
    Browse the repository at this point in the history

  20. Update jitenv.py to remove unused import

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    c32f704 View commit details
    Browse the repository at this point in the history

  21. Fix inverted graph

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    0f8b109 View commit details
    Browse the repository at this point in the history

  22. Split data into test/train

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    303c642 View commit details
    Browse the repository at this point in the history

  23. Refactor for clarity

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    81d8314 View commit details
    Browse the repository at this point in the history

  24. Use numpy for randomness

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    6265e40 View commit details
    Browse the repository at this point in the history

  25. Add open questions

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    12ae7f5 View commit details
    Browse the repository at this point in the history

  26. Fix a couple of model saving issues

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    e735ee0 View commit details
    Browse the repository at this point in the history

  27. Refactor and cleanup

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    b3032a0 View commit details
    Browse the repository at this point in the history

  28. Add evaluate.py

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    b76896c View commit details
    Browse the repository at this point in the history

  29. Fix inverted test/train

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    ecdde9b View commit details
    Browse the repository at this point in the history

  30. Add a way to get the probabilities of actions

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    7982aa4 View commit details
    Browse the repository at this point in the history

  31. Rename file

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    cb6dbab View commit details
    Browse the repository at this point in the history

  32. Clean up imports

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    8653120 View commit details
    Browse the repository at this point in the history

  33. Changed action space 

    - 0 to action_space.n-2 are now the CSEs to apply instead of adding and subtracting 1 to the action.
- 0 no longer means terminate, instead the action from the model of n-1 is the terminate signal.  This is not passed to the JIT.
    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    ca067c1 View commit details
    Browse the repository at this point in the history

  34. Add field validator for perf_score 

    This shouldn't happen but it's important enough to validate
    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    dd2702a View commit details
    Browse the repository at this point in the history

  35. Update applicability to ensure we have at least enough viable candida… 

    …tes and not more than total
    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    2c91107 View commit details
    Browse the repository at this point in the history

  36. Fix a few bugs with evaluate

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    430e5c9 View commit details
    Browse the repository at this point in the history

  37. Fix test/train split, some extra output

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    18dd0f1 View commit details
    Browse the repository at this point in the history

  38. Remove dead code, simplify format

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    0ec0e40 View commit details
    Browse the repository at this point in the history

  39. Rename JitEnv -> JitCseEnv

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    191137b View commit details
    Browse the repository at this point in the history

  40. More renames

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    74f628d View commit details
    Browse the repository at this point in the history

  41. Try to factor the observaiton space

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    01cc413 View commit details
    Browse the repository at this point in the history

  42. Fix test/train split

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    42593ff View commit details
    Browse the repository at this point in the history

  43. Reward cleanup 

    - Split reward function into shallow and deep.
    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    6343d01 View commit details
    Browse the repository at this point in the history

  44. Remove 0 perfscore check

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    727d869 View commit details
    Browse the repository at this point in the history

  45. Enable deep rewards

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    f63f576 View commit details
    Browse the repository at this point in the history

  46. Fix issue where jit failed and produced None method

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    b629873 View commit details
    Browse the repository at this point in the history

  47. Simplify deeper rewards

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    0779315 View commit details
    Browse the repository at this point in the history

  48. Update todo

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    b2c76e4 View commit details
    Browse the repository at this point in the history

  49. Add reward customization

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    87341a2 View commit details
    Browse the repository at this point in the history

  50. Clean up __all__

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    adbd9de View commit details
    Browse the repository at this point in the history

  51. Fix issue where we would JIT the first CSE candidate in reset 

    This was leftover code from the previous design of RLHook.
    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    f799976 View commit details
    Browse the repository at this point in the history

  52. Add two new features, emit selected sequence

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    dbc6902 View commit details
    Browse the repository at this point in the history

  53. Jit one less method per cse chosen in deep rewards

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    bdf9761 View commit details
    Browse the repository at this point in the history

  54. Use info dictionary instead of a specific state

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    17dead9 View commit details
    Browse the repository at this point in the history

  55. Fix segfault due to null variable

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    c7647fa View commit details
    Browse the repository at this point in the history

  56. Add superpmi_context 

    Getting the code well-factored so it's easy to modify.
    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    db2566a View commit details
    Browse the repository at this point in the history

  57. Add tensorboard entry for invalid choices, clear results

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    b1cfb46 View commit details
    Browse the repository at this point in the history

  58. Close the environment

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    58d8e16 View commit details
    Browse the repository at this point in the history

  59. Add documentation for JIT changes

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    453dbde View commit details
    Browse the repository at this point in the history

  60. Rename method

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    fa6c60d View commit details
    Browse the repository at this point in the history

  61. Normalize observation

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    eb023e2 View commit details
    Browse the repository at this point in the history

  62. Set return type hint for clarity

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    5dbbc51 View commit details
    Browse the repository at this point in the history

  63. Add RemoveFeaturesWrapper

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    b6b32d9 View commit details
    Browse the repository at this point in the history

  64. Update docstring

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    d10a28b View commit details
    Browse the repository at this point in the history

  65. Rename function

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    26d9ed3 View commit details
    Browse the repository at this point in the history

  66. Move feature normalization to a wrapper 

    - Also better wrapper factoring.
    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    5e39ded View commit details
    Browse the repository at this point in the history

  67. Remove import

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    7e39ed3 View commit details
    Browse the repository at this point in the history

  68. Fix warning

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    f880166 View commit details
    Browse the repository at this point in the history

  69. Fix Windows issue

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    b1ba57e View commit details
    Browse the repository at this point in the history

  70. Properly log when using A2C

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    333236b View commit details
    Browse the repository at this point in the history

  71. Add readme

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    8f277ac View commit details
    Browse the repository at this point in the history

  72. Change argument name

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    5e399ce View commit details
    Browse the repository at this point in the history

  73. Remove whitespace change

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    566c27a View commit details
    Browse the repository at this point in the history

  74. Format fixes

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    80bb745 View commit details
    Browse the repository at this point in the history

  75. Fix formatting

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    b54e32f View commit details
    Browse the repository at this point in the history

  76. Update readme: Fix grammar, add a note about evaluation

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    6b9bdac View commit details
    Browse the repository at this point in the history

  77. Fixed incorrect filename in readme

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    dee6f4a View commit details
    Browse the repository at this point in the history

  78. Save more data to .json in preparation of other model kinds

    @leculver
    leculver committed May 8, 2024
    Configuration menu
    Copy the full SHA
    05d8574 View commit details
    Browse the repository at this point in the history