Add init_eval to evaluation hook #3550
Conversation
Codecov Report
@@ Coverage Diff @@
## master #3550 +/- ##
==========================================
+ Coverage 60.77% 60.91% +0.13%
==========================================
Files 205 205
Lines 13781 13804 +23
Branches 2332 2340 +8
==========================================
+ Hits 8376 8409 +33
+ Misses 4981 4968 -13
- Partials 424 427 +3
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
mmdet/core/evaluation/eval_hooks.py
Outdated
| """ | ||
|
|
||
| def __init__(self, | ||
| dataloader, | ||
| interval=1, | ||
| tmpdir=None, | ||
| start=None, |
There was a problem hiding this comment.
interval and start is a group of arguments. It is better to put them together and tmpdir afterwards.
| return | ||
| if self.start is not None and runner.epoch >= self.start: | ||
| self.after_train_epoch(runner) | ||
| self.initial_epoch_flag = False |
There was a problem hiding this comment.
There seem to be some issues with the logic. I suppose the expected behaviors are as follows.
- start=None, interval=1: perform evaluation after each epoch.
- start=1, interval=1: perform evaluation after each epoch.
- start=None, interval=2: perform evaluation after epoch 2, 4, 6, etc
- start=1, interval=2: perform evaluation after epoch 1, 3, 5, etc
- start=0/-1, interval=1: perform evaluation after each epoch and before epoch 1.
- resuming from epoch i, start = x (x<=i), interval =1: perform evaluation after each epoch and before the first epoch.
- resuming from epoch i, start = i+1/None, interval =1: perform evaluation after each epoch.
There was a problem hiding this comment.
All are satisfied except 4. If start=1 and interval=2, then the evaluation is still conducted after epoch 2, 4, 6. Is it okay?
There was a problem hiding this comment.
- For case 2, the current implementation will perform evaluation after each epoch and before epoch 2.
- Since we specify start and interval, it may be more natural to adopt the
range()like behavior.
There was a problem hiding this comment.
As for case 2, the initial_epoch_flag will be False after the first epoch, and thus there would be no evaluation at the start of the second epoch.
I agree with the range-like behavior and will modify the code accordingly.
…to init_eval # Conflicts: # mmdet/core/evaluation/eval_hooks.py # tests/test_data/test_dataset.py
No description provided.