Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Polyak averaging for test-time data-augmetation #54

Open
GilesStrong opened this issue Jun 5, 2020 · 0 comments
Open

Polyak averaging for test-time data-augmetation #54

GilesStrong opened this issue Jun 5, 2020 · 0 comments
Labels
enhancement New feature or request good first issue Good for newcomers low priority Not urgent and won't degrade with time

Comments

@GilesStrong
Copy link
Owner

Current status

When data-augmentation is applied at test-time, the final prediction is based on the original data and the augmented data. This if okay, with the current data-augmentation in LUMIN, since it (should) result in physically valid events which are as likely as the original event.

Potential problem

The user, or future updates of LUMIN, may add data-augmentation which only produces data which is similar to the actual data, but is either not strictly physical, or as a differing probability of being.

Possible solution

In these cases is might be advantages to for the final prediction via Polyak averaging of the score on the original data, and on the augmented data, e.g.:

score = (beta*score on original data) + ((1-beta)*mean score on augmented data)

Beta would need to be an optional argument when calling .predict* methods of Ensemble and Model, and also Model.evaluate*. Beta could also be set as an property of e.g. HEPAugFoldYielder, and the the relevant methods could then see whether a beta had been set for the data, to avoid having to explicitly pass it every time.

@GilesStrong GilesStrong added enhancement New feature or request good first issue Good for newcomers low priority Not urgent and won't degrade with time labels Jun 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers low priority Not urgent and won't degrade with time
Projects
None yet
Development

No branches or pull requests

1 participant