Challenging deep image descriptors for retrieval in heterogeneous iconographic collections

[chullhwan-song]

https://arxiv.org/abs/1909.08866v1

[chullhwan-song]

abstract

• 기존 많은 sota image retrieval 기술 이용
• 데이터셋이 다른듯하다.
  • facing a panel of complex variations appearing in heterogeneous image datasets
  • 이러한 특성은,
    • multi-source, multi-date, multi-view contents
  • Alegoria dataset 라는 데이터 소개
    • 공개는 안한듯..
    • consisting of 12,952 iconographic contents representing landscapes of the French territory, and encapsultating a large range of intra-class variations of appearance which were finely labelled.
    • 아래 Fig.1을 보면, 데이터셋의 특성이 어떤지를 더 자세히 알 수 있을듯..
      
• 이 연구에서는 7개의 유명한 feature 적용하여 테스트
  • Six deep features (DELF, NetVLAD, GeM, MAC,RMAC, SPoC) and a hand-crafted local descriptor (ORB)
    • ORB는 오랫만에 언급되어 있는데, SIFT/SURF가 더 낫지 않았을까한다. ORB는 한 동영상에서 frame to frame의 tracking에 더 adapted한 feature가 아닌가 한다. > large scale image retrieval 분야에 성격상 안어울리듯하다.

HETEROGENEOUS COLLECTIONS OF IMAGES

• 내생각엔 이 논문의 가장 큰 기여는 이 데이터셋을 모았다는것?인듯보인다.ㅎ
• 근데 공개안한데이터셋을 contribution이라고 할수 있나??ㅎ

ALEGORIA dataset

• Fig.1
• 12952 grey & color images of outdoor scenes.
• jpeg
• max size(w or h) > 800 pixels
• Street-view, oblique, vertical aerial images, sketches, or old postcards 같은 류의 이미지 특징.
• 여기에다가, different viewpoints, by different type of cameras, different periods , sometimes even under different weather conditions 이러한 속성들이 추가됨.
• 이러한 데이터들을, geographic iconographic contents라 부르는것 같다.
  • French territory > 프랑스영토안의 지역사진들..
  • 전후부터 현재까지의 기간의 사진들
  • buildings (also stadiums and train stations), churches and cathedrals, historical sites (e.g. palaces,
    the most important monuments of Paris), seasides, suburbs of large cities, countrysides, etc.
    • neuralcode 나 google landmark데이터셋들과는 같은면도 있으나..Fig.1처럼 많이 다름.
• 평가를 위해, 681개 이미지
  • 39 class안의 적어도 10개이상의 이미지.
  • for example Eiffel Tower, Arc de Triomphe, NotreDame de Paris, Sacré-Coeur Basilica, Palace of Versailles, Palace of Chantilly, Nanterre, Saint-Tropez, Stadium Lyon Gerland, Perrache train station.

Annotation of the appearance variations

• Alegoria dataset is a good illustration of a highly multi-source, multi-date and multi-view dataset. = heterogeneity
  • highlight significant variations of appearance,
    • landscape transformations (site development, seasonal changes in vegetation)
    • perspective (significant change in angle of view)
    • quality (color, B&W or sepia old photos).
  • 10 variation (정리하기도 모그래서..ㅠ)
    • including the usual Scale, Illumination and Orientation changes, plus variations that are more specific to cultural heritage : Alterations (chemical degradations or damages on the photographic paper before digitization), Color domains (grayscale, sepia, etc.), Domain representation (picture, drawing painting), Time changes (impact of large time spans) ; and general indicators of difficulty like Clutter, Positionning (when the main object of interest is not central to the picture) and Undistinctiveness (when the object of interest is not clear even to the human eye).

DEEP FEATURES

• 생각해보니, 여기에 사용되었던 feature들은 거의다 리뷰해보았다.
• DELF
• NetVLAD
• GeM
• MAC
• RMAC
• SPoC > 이것도 많이 언급 > average pooling
• ORB > sift같은 local descriptor

실험