Could you tell me how to train on oxford5k?

[gsygsy96]

From your paper, it could be find that u evaluate paris dataset with model trained on Oxford5k. But i don't know what protocol you set on Oxford5k and Paris.
Have you cropped Oxford5k' query images? And Junk images are splited to no-match images?

[ducha-aiki]

"Training" there means k-means vocabulary learning, not the training HardNet itself.

[gsygsy96]

Sorry, I just understand your method. Firstly you use HardNet model trained on UBS dataset to extract 128 features. Then, K-means method is used to learn vocabulary on Oxford5k dataset. At the end, you use the BOW model test on Paris6k.

But I still confused how you deal with "Junk" images? Do you treat them as null images?

[ducha-aiki]

@filipradenovic was doing this experiment. But if I remember correctly, they were ignored - as suggested in original paper protocol. Btw, there is new protocol here https://github.com/filipradenovic/revisitop

[filipradenovic]

@shanYanGuan junk images are always ignored, ie treated as null images. However, in the new protocols described in https://github.com/filipradenovic/revisitop, null set is different depending on the protocol difficulty. More details can be found in the paper.

[gsygsy96]

Thank you bro! @ducha-aiki @filipradenovic
But I still have a question. For oxford5k or Paris, we always train our model on one dataset then evaluate on another dataset. Why? Why not split the dataset to two train and test sets.And train/evaluate on the same dataset.

[ducha-aiki]

Well... first, datasets are quite small. Second, we want to see how method generalize.
But most important - because we want to compare to previous methods, so we should take protocol which is commonly used.

[gsygsy96]

ok, I see. Hardnet is to extract an image descriptor. This descriptor is designed to deal with matching problem. So I thought it should contain the key point informations. And Patch Datasets is more about viewpoint or illumination changes on the same objects. But image retrieval is to find the same class object not the same objects. So why hardnet can do well in Oxford5k?

[gsygsy96]

the key point is I think patch match task is different from image retrieval, for the reason that patch match is to fine the same object with different viewpoint or illumination, but image retrieval is to find the same image with same semantics So the descriptor for patch match can't adopt to image retrieval.

[ducha-aiki]

. But image retrieval is to find the same class object not the same objects.

You are wrong here. Image retrieval, at least in classical computer vision sense is about finding SAME object under different conditions, don`t mix it with "similarity search". E.g. see here

https://www.kaggle.com/c/landmark-retrieval-challenge
"Image retrieval is a fundamental problem in computer vision: given a query image, can you find similar images in a large database? This is especially important for query images containing landmarks, which accounts for a large portion of what people like to photograph.

In this competition, Kagglers are given query images and, for each query, are expected to retrieve all database images containing the same landmarks (if any)."

[ducha-aiki]

@shanYanGuan but you are right, that actually Oxford5k task is "particular object retrieval" or "particular instance retrieval" to be precise. It is just community used to call it "image retrieval".

[gsygsy96]

You mean image retrieval is to retrieve all database images containing the same content? But image retrieval based on Hash method seems different. For example, cifair10 is to retrieve the same class image. But image content in same class is inconsistent.

[gsygsy96]

cifar10... sorry for wrong split. E.g. below images is airplane, but is different airplane.
https://www.cs.toronto.edu/~kriz/cifar-10-sample/airplane1.png
https://www.cs.toronto.edu/~kriz/cifar-10-sample/airplane4.png

[gsygsy96]

https://www.cs.toronto.edu/~kriz/cifar.html

[ducha-aiki]

CIFAR is classification problem, not retrieval. Or you can call it "category retrieval". And Oxford5k is "instance retrieval".

[gsygsy96]

So Nework for match cannot be used to category retrieval. Am I right?

[ducha-aiki]

@shanYanGuan yes, you are right.

[gsygsy96]

Sounds strange. And if match network just can get similarity about same object not same category, what meaning it has?

[ducha-aiki]

@shanYanGuan when you are looking for YOUR phone or dog, do you want to find your phone or dog, or any kind would suffice?
If you want to see NotreDame or White House, do you want to see them exactly, or any house would suffice?
When you are doing 3d reconstruction of object, do you want to have a mess of all objects of the same category?

[gsygsy96]

OK, but why match network can't get high semantics, such as content that image have? If they can get some high semantics clues, they should be able to classify if they are same category.

[ducha-aiki]

I don`t know this for sure. I guess, that networks do only things, which we train them for. If we train them for semantics, they do semantics. If we train for correspondences - they do correspondences, etc. These are just different tasks. Why trucks cannot compete with F1 cars?

[gsygsy96]

ok. I can get you. Maybe I should do some experiments to solve this problem. Thanks for your instructive answer!