Custom feature extractor

[jpcbertoldo]

I have a use case where I want to use a custom feature extractor (weights should be downloaded from a given url).
I'm looking for a generic feature extractor, the analogous of the folder dataset for feature extraction.

@ashwinvaidya17 I don't know what the torchfx is, but could your #675 eventually cover this use case?

If not, are you planning to merge it soon (so I can build on top of it)?

[ashwinvaidya17]

Sure! Do you want just the weights or even the ability to pass custom backbone architecture?

[jpcbertoldo]

Looks good but I in my use case I'd like to (0) instantiate my CustomModel from an arbitrary .py not necessarily in the path and doing so from the config file.

Does your design cover (0)? (I couldn't read the PR)

---

I guessed there would be some (1) dynamic loading of module containing the class and (2) a dynamic instantiation of this (generic, arbitray) class.

Because of (1) I expected to have some args like module_path="/path/to/module.py" and class_name="PytorchModelClass", and because of (2) I expected some arg like init_args (which would allow the user to properly instantiate from the config file).

Does it make sense?

---

If your design covers the case of using an instantiated CustomModel (i.e. steps (1) and (2) are assumed to be done) I think it's best to leave the rest (step (0)) to build later -- in any case I think TorchFXFeatureExtractor should not encapsulate the logic of the step (0).

[ashwinvaidya17]

My design expects the class to be passed to the feature extractor and has support for extra kwargs to pass to the model when initializing it. But I'll discuss the design with the rest of the team to figure out what's the best approach here. If we deprecate TImm feature extractor then we can have more flexibility when designing torchfx feature extractor class. Otherwise we might have to ensure that both have similar parameters and scope.

[jpcbertoldo]

My design expects the class to be passed to the feature extractor and has support for extra kwargs to pass to the model when initializing it

By initializing you mean CustomClass(**init_args), right?
If yes, then we can figure out the dynamic load later : )

If we deprecate TImm feature extractor then we can have more flexibility when designing torchfx feature extractor class.

I don't see why deprecate timm, it's just a different use case (when the pretrained model is publicly available in timm).

Otherwise we might have to ensure that both have similar parameters and scope.

Couldn't they have specific parameters?
I mean, sounds natural to me that such feature extractor (as I described before) would have more parameters.

[ashwinvaidya17]

By initializing you mean CustomClass(**init_args), right?

Yes. In that case let's figure out dynamic loading later.

I don't see why deprecate timm, it's just a different use case (when the pretrained model is publicly available in timm).

Maybe we can keep it.

Couldn't they have specific parameters? I mean, sounds natural to me that such feature extractor (as I described before) would have more parameters.

Yes. Ideally it would be nice if we can just swap the feature extractors without changing how they are called. But I think it depends on how we decide to go forward. I have a feeling that we are going to end up with specific parameters between both as torchfx is more flexible.

[jpcbertoldo]

Yes. Ideally it would be nice if we can just swap the feature extractors without changing how they are called. But I think it depends on how we decide to go forward. I have a feeling that we are going to end up with specific parameters between both as torchfx is more flexible.

Yes, sounds normal, the use cases just have different requirements :)

Tnx for clarifying ashwin!

[samet-akcay]

https://anomalib.readthedocs.io/en/latest/markdown/guides/how_to/models/feature_extractors.html