[Feature] [Tabular] Integrating Bench and Dashboard in CI #3527
Conversation
prateekdesai04
changed the title
|
Job PR-3527-4d232c7 is done. |
| @@ -143,4 +170,3 @@ jobs: | |||
| repository: ${{ github.event.inputs.repository }} | |||
| comment-id: ${{ github.event.inputs.comment-id }} | |||
| body: ${{ steps.website.outputs.body }} | |||
|
|
|||
| push: | ||
| branches: ["master"] | ||
| schedule: | ||
| - cron: '0 0 * * 0' |
There was a problem hiding this comment.
Can we change this to be around 2am in the morning PST. By default, this is UTC time
|
|
||
| env: | ||
| AG_MODULE: tabular | ||
| AG_PRESET: medium |
There was a problem hiding this comment.
We might want to discuss with the team 1. how frequent we want to run the benchmark 2. what preset we want to compare (running all presets are expensive) 3. what benchmark we want to run (current just test, which is dummy). For now this is good
There was a problem hiding this comment.
Got it, will add it as a parking lot for today
| @@ -35,6 +35,7 @@ jobs: | |||
| token: ${{ secrets.PAT }} | |||
| commands: | | |||
| benchmark | |||
| benchmark_master | |||
There was a problem hiding this comment.
I don't think this does anything and it's not needed too. Master won't be triggered by slash command
|
Job PR-3527-d40926c is done. |
|
Job PR-3527-11e5e38 is done. |
* 'master' of https://github.com/awslabs/autogluon: [Lightning] Update lightning import (autogluon#3526) [Feature] [Tabular] Integrating Bench and Dashboard in CI (autogluon#3527) [timeseries] Automatically remove short time series from training data (autogluon#3469) [timeseries] Fix timeseries tutorial on Colab (autogluon#3522) [AutoMM] Support torch.compile() (autogluon#3520)
Description of changes:
Feature addition, integration of AG Bench with AG Dashboard.
Now, a contributor can benchmark their PRs and get a visualization of comparison w.r.t latest benchmark run on master branch
Link to test runs:
Benchmark on master branch
https://github.com/prateekdesai04/autogluon/actions/runs/6192027450
Benchmark + Evaluation on dev branch
https://github.com/prateekdesai04/autogluon/actions/runs/6192306818
Link to slash command dispatch
prateekdesai04#20 (comment)
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.