Interpretable Semantic Photo Geolocation

This repository contains a re-implementation of our paper Interpretable Semantic Photo Geolocation.

Semantic Partitioning (SemP)

Subpackage semantic_partitioning contains:

• script for reverse geocoding
• raw dataset visualization
• scripts to construct the semantic partitioning (SemP)

See semantic_partitioning/README.md for details.

Classification

Subpackage geo_classification contains:

• script to train from scratch
• evaluation pipeline including testsets
• pretrained models (EfficientNet-B4):

See geo_classification/README.md for details.

Extended MP-16 Dataset (EMP-16)

To overcome the need for a full installation of a reverse geocoder such as Nominatim, we provide the postprocessed output of the reverse geocoding for the MP-16 dataset¹ along with the validation set (YFCC-Val26k) which originally comprising photos and respective GPS coordinates. Both datasets are subsets of the YFCC100M dataset² which are crawled from Flickr.

Further details: semantic_partitioning/README.md

Concept Influence

We provide the underlying functionality to compute the presented concept influence metric based on given semantic maps and attribution/explanation maps. Please note, that the computation of both semantic maps and explanation maps are not part of this repository.

---

Requirements

conda env create -f environment.yml
conda activate github_semantic_geo_partitioning
# cd in respective subpackages
cd semantic_partitioning
cd geo_classification

Citation

@InProceedings{Theiner_2022_WACV,
    author    = {Theiner, Jonas and M\"uller-Budack, Eric and Ewerth, Ralph},
    title     = {Interpretable Semantic Photo Geolocation},
    booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
    month     = {January},
    year      = {2022},
    pages     = {750-760}
}

Licence

This work is published under the GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007. For details please check the LICENSE file in the repository.

References

Footnotes

1. Larson, M., Soleymani, M., Gravier, G., Ionescu, B., & Jones, G. J. (2017). The benchmarking initiative for multimedia evaluation: MediaEval 2016. IEEE MultiMedia, 24(1), 93-96. ↩

2. Thomee, B., Shamma, D. A., Friedland, G., Elizalde, B., Ni, K., Poland, D., ... & Li, L. J. (2016). YFCC100M: The new data in multimedia research. Communications of the ACM, 59(2), 64-73. ↩