🚀 add subset to slice datasets

[MightyStud]

📝 Description

Added the functionality to only take a percentage of the subset of the data, Afaik, you only can use all the data for training/testing, but with this PR you can use a subset of the data which is quite handy when experimenting.

✨ Changes

add subset argument to folder class

Select what type of change your PR is:

• 🐞 Bug fix (non-breaking change which fixes an issue)
• 🔨 Refactor (non-breaking change which refactors the code base)
• [X ] 🚀 New feature (non-breaking change which adds functionality)
• 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
• 📚 Documentation update
• 🔒 Security update

✅ Checklist

Before you submit your pull request, please make sure you have completed the following steps:

• 📋 I have summarized my changes in the CHANGELOG and followed the guidelines for my type of change (skip for minor changes, documentation updates, and test enhancements).
• [X ] 📚 I have made the necessary updates to the documentation (if applicable).
• [ X] 🧪 I have written tests that support my changes and prove that my fix is effective or my feature works (if applicable).

For more information about code review checklists, see the Code Review Checklist.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[alexriedel1]

Hey, this functionality is already built in Lightning:
https://lightning.ai/docs/pytorch/stable/common/trainer.html#trainer-class-api

limit_train_batches (Union[int, float, None]) – How much of training dataset to check (float = fraction, int = num_batches). Default: 1.0.

limit_val_batches (Union[int, float, None]) – How much of validation dataset to check (float = fraction, int = num_batches). Default: 1.0.

limit_test_batches (Union[int, float, None]) – How much of test dataset to check (float = fraction, int = num_batches). Default: 1.0.

You can pass trainer arguments to the Engine class

[samet-akcay]

Hi @MightyStud, thanks for creating this PR. As @alexriedel1 mentioned above, this is already possible with Lightning. For details, you could refer here
https://lightning.ai/docs/pytorch/stable/common/trainer.html#limit-train-batches
https://lightning.ai/docs/pytorch/stable/common/trainer.html#limit-test-batches
https://lightning.ai/docs/pytorch/stable/common/trainer.html#limit-val-batches

Here is how you could use this:

from anomalib.data import MVTec
from anomalib.models import Patchcore
from anomalib.engine import Engine

# Create datamodule and model, whichever data and model you want to import.
datamodule = MVTec()
model = Patchcore()

# default used by the Trainer
engine = Engine(limit_val_batches=1.0)

# run through only 25% of the validation set each epoch
engine = Engine(limit_val_batches=0.25)

# run for only 10 batches
engine = Engine(limit_val_batches=10)

# disable validation
engine = Engine(limit_val_batches=0)

[samet-akcay]

I think we close this PR as the feature is already available. We would welcome more of your contributions in the future. Thanks a lot for the effort!