Allow different backbones for bottleneck features

[lwzhaojun]

Does the program now support EfficientNet weighting as backbone? My model: (batch, channel, width, height )= (1802, 1, 62, 96).output_size=(1803,1280). Here's how I used it:

fastdup.run('/home/datasets', work_dir='out', nearest_neighbors_k=478, model_path='efficientnetb1.ort', d=1280)
When I use the model, I get the following error, what parameters need to be adjusted in the source code to apply, can you give some suggestions?
Found total 479 images to run on Failed assertion false /home/ubuntu/visual_database/cxx/src/image_to_blob.h:222 Failed assertion false /home/ubuntu/visual_database/cxx/src/image_to_blob.h:222 free(): corrupted unsorted chunks Segmentation fault (core dumped)

[dbickson]

HI @lwzhaojun you did everything correct, it should have worked, we did not debug black and white images yet (i.e. channels = 1), can you send us your ort model and we will make sure it works end to end.

[lwzhaojun]

In fact, the RGB image model has also been tried, and the same error will be reported. It looks like it's caused by a model input mismatch.
Failed assertion blob.size[0] == NETWORK_DIMENSIONS[1] 224.0000 96.0000 /home/ubuntu/visual_database/cxx/src/image_to_blob.h:155 Failed assertion blob.size[0] == NETWORK_DIMENSIONS[1] 224.0000 96.0000 /home/ubuntu/visual_database/cxx/src/image_to_blob.h:155 terminate called without an active exception Failed assertion blob.size[0] == NETWORK_DIMENSIONS[1] 224.0000 96.0000 /home/ubuntu/visual_database/cxx/src/image_to_blob.h:155 Aborted (core dumped)

Here is my single-channel model.
efficientb1.zip

The following is the three-channel model.
efficientb1_3.zip
Please help me see it.

[dbickson]

hi @lwzhaojun. We have identified the source of the error that the expected image size is 96x62 while so far we support default image sizes of 224x224. Give us a couple of days so we could update the code to add non square image support.

[dbickson]

BTW I was not able to download the first zip file if you can send it again please. I was able to download the RGB model.

[lwzhaojun]

Thank you for your support. After I modified the RGB image size to 224x224×3, The following error occurred:
Failed assertion ptr[j] >= MIN_FEATURE_ALLOWED_VAL && ptr[j] < MAX_FEATURE_ALLOWED_VAL /home/ubuntu/visual_database/cxx/src/data_debug.hpp:354
If you can, I hope you can help debug the 96×62×1. Below is the model of 96×62×1.
efficientb1.zip
Below is the model of 224x224×3.
efficientnet-224rgb.zip

[dbickson]

HI @lwzhaojun we have just released version 0.152 which both supports efficient net non square RGB (3 channels) and black and white (1 channel). The only thing we did not handle is the normalization phase after reading the image. We see on the web there are references for normalization for efficientnet (for example lukemelas/EfficientNet-PyTorch#255) can you please verify which normalization you need so we could support it? BTW we fixed also the 224x224 assertion.

[dbickson]

hi @lwzhaojun did you have a chance to check this? Can I close the issue?

[lwzhaojun]

I'm sorry to verify it now, I just verified that the program can run whether it is an RGB model, a single-channel model, or a non square input. Thank you very much for your great support.
Regarding normalization, we haven't normalized before, but I think normalization is still very important, and I will do a version of normalization recently, and I will have to ask you to support it at that time.
By the way, did you only update to the pip library, not to the github source code?
Finally, you can close the issue. Thanks again.

[dbickson]

hi @lwzhaojun we are glad to hear the update is working for you!
I believe it will be useful to add normalization, send me the normalization you need, we will add some way for the user to provide it. The changes we did were in our c++ engine which is not open source thus you did not see changes in the python side.
Best,