Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Overfitting detection as an AbstractRepairStep component #1208

Merged
merged 2 commits into from
Apr 19, 2021

Conversation

andre15silva
Copy link
Contributor

@andre15silva andre15silva commented Apr 6, 2021

Closes #1135

Why not a separate pipeline step?

Currently repair steps call recordPatches when they finish their work, which calls the notifiers. If, after that, we have logic that can filter/rank/etc the patches, it messes up that logic.

What is there to finish?

Are the patch ranking methods for ODS working (tests are ignored). If so, wouldn't it make more sense to have different modes for the different features? Instead of OVERFITTING we would have ODS_P4J and ODS_S4R for example. @monperrus wdyt?

Also, should ranking be done inside each repair step or overall? The later makes more sense to me, but would require a separate step for it and would mess up the observer pattern mentioned above.

ChangeLog

  • Add missing parameters to control patch ranking, filtering and classification to LauncherUtils and RepairnatorConfig. Update test accordingly.
  • Move ranking (to finish), filtering and classification logic to a method in AbstractRepairStep. Update test accordingly. Ranking and filtering tests to be activated/written.

@monperrus
Copy link
Contributor

Cool to make progress on this topic.

Are the patch ranking methods for ODS working (tests are ignored).

We can remove the ranking code (outdated plus not researched anymore). We'll start from scratch again with #1210

If so, wouldn't it make more sense to have different modes for the different features

No. We want to simply take the best feature set identified by @SophieHYe in the ODS paper.

@andre15silva andre15silva marked this pull request as ready for review April 14, 2021 09:20
@andre15silva andre15silva changed the title wip: Overfitting detection as an AbstractRepairStep component Overfitting detection as an AbstractRepairStep component Apr 14, 2021
@andre15silva
Copy link
Contributor Author

This is ready. The only test failing is a flaky test (identified in #1176 )

@monperrus
Copy link
Contributor

Thanks. What's the intention behind PATCH_FILTERING_MODE?

@monperrus
Copy link
Contributor

The only test failing is a flaky tes

Could you remove / deflakify it in a separate PR? Thanks!

@andre15silva
Copy link
Contributor Author

Thanks. What's the intention behind PATCH_FILTERING_MODE?

In the case there is a new filtering mode in the future (in the case there are more than one filtering options for the same classification mode), the developer will just need to add a new mode.

Could you remove / deflakify it in a separate PR? Thanks!

Done. Opened #1214

…pse#1135

Signed-off-by: André Silva <andre15andre@hotmail.com>
Signed-off-by: André Silva <andre15andre@hotmail.com>
@andre15silva
Copy link
Contributor Author

Ready

@monperrus monperrus changed the title Overfitting detection as an AbstractRepairStep component refactor: Overfitting detection as an AbstractRepairStep component Apr 19, 2021
@monperrus monperrus merged commit 0cf2f78 into eclipse:master Apr 19, 2021
@monperrus
Copy link
Contributor

Great, thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Overfitting detection as pipeline step or AbstractRepairStep component
2 participants