Add DSR model

[phcarval]

Description

• Implementation of the DSR model, from https://github.com/VitjanZ/DSR_anomaly_detection
• References [Task]: Implementation of newer SOTA unsupervised anomaly detection methods #1121
• Architecture works without crashes (AFAIK), but might not yield the expected performance
• Pretraining needs weights : automatic weight download needs to be set up (unsure how, it's not on TIMM)
• Third phase of training not implemented yet (unsure of how to preceed)
• Still various pre-commit warnings flaring up

Changes

• Bug fix (non-breaking change which fixes an issue)
• Refactor (non-breaking change which refactors the code base)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Checklist

• My code follows the pre-commit style and check guidelines of this project.
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing tests pass locally with my changes
• I have added a summary of my changes to the CHANGELOG (not for minor changes, docs and tests).

[phcarval]

Added automatic pretrained weight downloading. Now the model can train on its own, but the third phase of training is still missing.
These additions would benefit from being integrated to download.py (as well as the anomaly generator should be added to the already existing files... but this is more complex since DSR has its own way of generating anomalies).

[phcarval]

I tested the DSR model by loading the authors' pretrained weights and finding the exact same results as them. The following changes are necessary to reproduce :

• load_state_dict() for each module in DsrModel.__init__()
• normalization: none in config.yaml
• normalization_method: none in config.yaml
• eval batch size is 1
• comment image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) in data/utils/image.py
• set weight_file to None in test.py

Still training left to test as well as implementing the third phase of training.

[phcarval]

Training seems to work on my end. Only thing left to implement is the upsampling module.

[samet-akcay]

I fixed it using manual optimization. However, it doesn't improve maintainability and perhaps it should be changed if we find a way to make it work using core Lightning functionality.

Thanks for the fix @phcarval! I agree with @blaz-r. Your solution also shows that this could be done manually via vanilla torch, which demonstrates flexibility. I think it would be a good example to some users/developers for future implementations.

[phcarval]

I fixed it using manual optimization. However, it doesn't improve maintainability and perhaps it should be changed if we find a way to make it work using core Lightning functionality.

Thanks for the fix @phcarval! I agree with @blaz-r. Your solution also shows that this could be done manually via vanilla torch, which demonstrates flexibility. I think it would be a good example to some users/developers for future implementations.

Thanks for the feedback! My concern was that, AFAIK, it is the only method that uses a different optimization system in Anomalib and so it could create somewhat of an issue down the line if the method is refactored, as optimization is supposed to be "automatic" in Lightning. It's probably a very minor problem though.

Anyway, I have no more outstanding issues on my end!

[samet-akcay]

Thanks @phcarval, this is a huge PR :)

I've got some minor comments, but the overall code quality is quite high! Thanks again for your contribution and patience!

[samet-akcay]

left over?

[samet-akcay]

is anomaly_map really a true_mask?

[samet-akcay]

By the way, there is one failed test, mainly because DsrModel returns a dict. We probably need a separate PR to handle torch outputs that are dict

[phcarval]

I went though all requested changes @samet-akcay !

[samet-akcay]

Is it possible if model_outputs have the same keys for anomaly_maps and pred_scores? The reason for asking is that we need standard keys for the TorchInferencer stuff?

The tests are failing since the output model is returning a different output than what is expected here

anomalib/src/anomalib/deploy/inferencers/torch_inferencer.py

Lines 155 to 185 in 5eb20c6

	def post_process(self, predictions: Tensor, metadata: dict | DictConfig | None = None) -> dict[str, Any]:
	"""Post process the output predictions.
	Args:
	predictions (Tensor): Raw output predicted by the model.
	metadata (dict, optional): Meta data. Post-processing step sometimes requires
	additional meta data such as image shape. This variable comprises such info.
	Defaults to None.
	Returns:
	dict[str, str | float | np.ndarray]: Post processed prediction results.
	"""
	if metadata is None:
	metadata = self.metadata
	if isinstance(predictions, Tensor):
	anomaly_map = predictions.detach().cpu().numpy()
	pred_score = anomaly_map.reshape(-1).max()
	else:
	# NOTE: Patchcore `forward`` returns heatmap and score.
	# We need to add the following check to ensure the variables
	# are properly assigned. Without this check, the code
	# throws an error regarding type mismatch torch vs np.
	if isinstance(predictions[1], (Tensor)):
	anomaly_map, pred_score = predictions
	anomaly_map = anomaly_map.detach().cpu().numpy()
	pred_score = pred_score.detach().cpu().numpy()
	else:
	anomaly_map, pred_score = predictions
	pred_score = pred_score.detach()

[samet-akcay]

I have create PR #1354 to handle cases like this. DsrModel needs to return anomaly_map and pred_score though.

[phcarval]

@samet-akcay have you had the time to test the model? I seem to have very bad segmentation results, this might be due to the optimizer change. I would like to know if you had that problem too.

[phcarval]

When removing the default imagenet normalization parameter, training seems to be going much better and my results are close to the paper's. Only problem is the one I describe in issue #1380 where images are segmented even if they're classified as non-anomalous.

[phcarval]

When removing the default imagenet normalization parameter, training seems to be going much better and my results are close to the paper's. Only problem is the one I describe in issue #1380 where images are segmented even if they're classified as non-anomalous.

Actually, this is not a problem since DSR outputs a score that does not depend on the final segmentation map. I believe all outstanding issues are solved now :)

[samet-akcay]

Thanks @phcarval, looking great! Really appreciate your patience, and thanks again for this great contribution!