Add doc page and example for shift detection #244
Conversation
Coverage reportThe coverage rate went from None of the new lines are part of the tested code. Therefore, there is no coverage data about them. |
| :py:class:`~renate.shift.detector.ShiftDetector`, which defines the main interface. Once a | ||
| :code:`detector` object has been initialized, one calls :code:`detector.fit(dataset_ref)` on a | ||
| reference dataset (a PyTorch dataset object). This reference dataset characterizes the expected | ||
| data distribution. It may, e.g., be the validation set used during the previous fitting of the |
There was a problem hiding this comment.
nit: dataset used for training would be easier to understand for inexperienced readers
There was a problem hiding this comment.
I'm hesitant, because it would be dangerous to use the actual training set if the feature extractor has seen that.
| extractor. | ||
|
|
||
| .. literalinclude:: ../../examples/shift_detection/image_shift_detection.py | ||
| :caption: Example |
There was a problem hiding this comment.
I would be good to explain this example a bit more. At least break it down in a few pieces capturing the main aspects: created dataset, sample reference set (why?), sampler query set, extract features and perform test.
It doesn't need to explain everything but a couple of sentences explaining why certain things are done will greatly help.
There was a problem hiding this comment.
I think the comments in the example script explain those things. I can expand them a bit. Or would you prefer having these explanations separate from the code?
* Add NLP Components to Benchmarking (#213) * Robust Integration Tests (#214) * Update Renate Config Example (#226) * Make Wild Time Available in Benchmarking (#187) * Fix `target_column` bug in `HuggingFaceTextDataModule` (#233) * Add MMD covariate shift detector (#237) * Add KS covariate shift detector (#242) * Update dependabot.yml (#248) * Update versions of some requirements (#247) * Add doc page and example for shift detection (#244) * Bump version (#252) --------- Co-authored-by: Lukas Balles <lukas.balles@gmail.com>
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.