PatchCore uses a lot of CPU resources.

[laogonggong847]

hello @samet-akcay, hello @ashwinvaidya17

PatchCore takes a lot of CPU resources. Is it caused by building the memory bank? And is it possible to put this part of computation into GPU.

There is a strange phenomenon after converting my trained model to ONNX using the official code provided: "Using this onnx model under CPU is slow but usable for inference; however, using GPU prompts me that the GPU's memory overflow". I didn't understand if this situation is caused by Patchcore itself or a flaw in the algorithm design when implementing patchcore.

[blaz-r]

Hi,

There are mainly two reasons why PatchCore uses so much memory.

First one is that it's memory-bank based method, this means that the features are stored in memory and later used for kNN. Although the features are sub-sampled (using coreset sub-sampling), they still have a noticeable memory footprint.

The other reason is implementation based, PatchCore in Anomalib does the kNN search by calculating a distance matrix. This matrix can become really large, resulting in the issues - however, with reasonable image dimensions, it should still fit inside GPU memory, unless your GPu memory is really small.

Hope this helps.

[samet-akcay]

If we can make use of https://github.com/facebookresearch/faiss library, the model might become more efficient. The problem with faiss is that we need to ensure that we could export the models to onnx and openvino. This is the main deal breaker.

[blaz-r]

Faiss is not easy to get working on a GPU as is, so I'm not sure how that'd go when exporting. Another possibility is an iterative distance calculation for cases with constrained GPU memory (at the cost of inference time).