Skip to content

Latest commit

 

History

History
175 lines (146 loc) · 10.6 KB

results.md

File metadata and controls

175 lines (146 loc) · 10.6 KB

Code for our ICML-2020 paper Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation.

Framework:

Results:

Table 2 [UDA results on Digits]

Methods S->M U->M M->U Avg.
srconly (2019) 71.5 85.5 82.5
srconly (2020) 69.2 89.8 77.6
srconly (2021) 69.7 88.7 79.0
srconly (Avg.) 70.2 88.0 79.7 79.3
SHOT-IM (2019) 98.9 98.6 97.8
SHOT-IM (2020) 99.0 97.8 97.7
SHOT-IM (2021) 98.9 97.6 97.7
SHOT-IM (Avg.) 99.0 97.6 97.7 98.2
SHOT (2019) 98.8 98.6 98.0
SHOT (2020) 99.0 97.6 97.8
SHOT (2021) 99.0 97.7 97.7
SHOT (Avg.) 98.9 98.0 97.9 98.3
Oracle (2019) 99.2 99.2 97.1
Oracle (2020) 99.2 99.2 97.0
Oracle (2021) 99.3 99.3 97.0
Oracle (Avg.) 99.2 99.2 97.0 98.8

Table 3 [UDA results on Office]

Methods A->D A->W D->A D->W W->A W->D Avg.
srconly (2019) 79.9 77.5 58.9 95.0 64.6 98.4
srconly (2020) 81.5 75.8 61.6 96.0 63.3 99.0
srconly (2021) 80.9 77.5 60.2 94.8 62.9 98.8
srconly (Avg.) 80.8 76.9 60.3 95.3 63.6 98.7 79.3
SHOT-IM (2019) 88.8 90.7 71.7 98.5 71.7 99.8
SHOT-IM (2020) 92.6 92.2 72.4 98.4 71.1 100.
SHOT-IM (2021) 90.6 90.7 73.3 98.0 71.2 99.8
SHOT-IM (Avg.) 90.6 91.2 72.5 98.3 71.4 99.9 87.3
SHOT (2019) 93.4 88.8 74.9 98.5 74.6 99.8
SHOT (2020) 95.0 92.0 75.7 98.6 73.7 100.
SHOT (2021) 93.8 89.7 73.6 98.2 74.6 99.8
SHOT (Avg.) 94.0 90.1 74.7 98.4 74.3 99.9 88.6

Table 4 [UDA results on Office-Home]

Methods Ar->Cl Ar->Pr Ar->Re Cl->Ar Cl->Pr Cl->Re Pr->Ar Pr->Cl Pr->Re Re->Ar Re->Cl Re->Pr Avg.
srconly (2019) 45.2 67.2 75.0 52.4 62.6 64.9 52.4 40.6 73.0 65.0 43.8 78.1
srconly (2020) 44.2 67.2 74.4 52.3 63.1 64.5 53.1 41.0 73.7 65.3 46.8 77.9
srconly (2021) 44.5 67.7 74.8 53.4 62.4 64.9 53.4 40.4 72.9 65.7 45.8 78.1
srconly (Avg.) 44.6 67.3 74.8 52.7 62.7 64.8 53.0 40.6 73.2 65.3 45.4 78.0 60.2
SHOT-IM (2019) 56.5 77.1 80.8 67.7 73.3 75.1 65.5 54.5 80.6 73.4 57.2 84.0
SHOT-IM (2020) 54.7 76.3 80.2 66.8 75.8 76.2 65.6 53.9 80.7 73.6 58.3 83.5
SHOT-IM (2021) 54.9 76.4 80.1 66.2 73.8 75.0 65.7 56.1 80.7 74.2 59.6 82.9
SHOT-IM (Avg.) 55.4 76.6 80.4 66.9 74.3 75.4 65.6 54.8 80.7 73.7 58.4 83.4 70.5
SHOT (2019) 57.3 79.3 81.8 68.1 77.1 78.0 67.8 55.0 82.5 73.2 58.5 84.1
SHOT (2020) 57.1 77.5 81.6 68.4 78.2 77.9 67.0 55.6 82.4 73.6 60.2 84.6
SHOT (2021) 57.0 77.6 81.0 67.5 79.2 78.3 67.3 54.1 81.6 73.0 57.8 84.2
SHOT (Avg.) 57.1 78.1 81.5 68.0 78.2 78.1 67.4 54.9 82.2 73.3 58.8 84.3 71.8

Table 5 [UDA results on VisDA-C]

Methods plane bcycl bus car horse knife mcycl person plant sktbrd train truck Per-class
srconly (2019) 57.1 20.5 48.6 60.8 66.2 3.6 80.7 23.9 38.5 31.0 87.0 10.7
srconly (2020) 65.1 18.9 57.2 66.9 69.9 11.0 84.7 23.9 69.4 34.0 83.8 9.3
srconly (2021) 60.5 25.5 47.0 75.2 61.3 4.2 81.1 21.9 63.9 26.7 83.1 4.0
srconly (Avg.) 60.9 21.6 50.9 67.6 65.8 6.3 82.2 23.2 57.3 30.6 84.6 8.0 46.6
SHOT-IM (2019) 94.3 86.6 78.1 54.0 91.0 92.3 79.1 78.9 88.4 86.0 88.0 50.7
SHOT-IM (2020) 93.4 87.1 80.4 51.7 91.5 92.9 80.0 78.0 89.6 85.1 87.2 51.3
SHOT-IM (2021) 93.5 85.7 77.6 46.3 90.5 95.1 77.9 78.1 89.7 85.0 88.5 51.2
SHOT-IM (Avg.) 93.7 86.4 78.7 50.7 91.0 93.5 79.0 78.3 89.2 85.4 87.9 51.1 80.4
SHOT (2019) 93.8 89.0 81.4 57.0 93.4 94.7 81.3 80.3 90.5 89.1 85.3 58.4
SHOT (2020) 94.5 87.3 80.0 57.1 93.1 94.5 82.0 80.7 91.7 89.4 87.0 58.3
SHOT (2021) 94.7 89.1 78.7 57.8 92.8 95.5 78.8 79.9 92.4 89.0 86.6 57.9
SHOT (Avg.) 94.3 88.5 80.1 57.3 93.1 94.9 80.7 80.3 91.5 89.1 86.3 58.2 82.9

Table 7 [PDA/ ODA results on Office-Home]

Methods@PDA Ar->Cl Ar->Pr Ar->Re Cl->Ar Cl->Pr Cl->Re Pr->Ar Pr->Cl Pr->Re Re->Ar Re->Cl Re->Pr Avg.
srconly (2019) 46.0 69.7 80.7 56.3 60.4 66.9 60.2 40.6 76.0 70.8 48.6 78.5
srconly (2020) 45.1 71.0 80.8 55.7 61.8 66.4 61.4 39.7 76.1 70.6 49.7 76.3
srconly (2021) 44.5 70.5 81.3 56.8 60.2 65.2 61.2 40.0 76.5 70.9 47.2 77.2
srconly (Avg.) 45.2 70.4 81.0 56.2 60.8 66.2 60.9 40.1 76.2 70.8 48.5 77.3 62.8
SHOT-IM (2019) 57.5 86.2 88.2 69.3 73.6 79.9 79.7 62.2 89.0 80.8 66.6 91.0
SHOT-IM (2020) 61.2 82.0 87.8 73.3 74.4 80.6 74.1 58.8 90.0 81.7 70.8 87.1
SHOT-IM (2021) 55.0 82.6 90.3 74.5 74.0 76.5 74.4 60.8 91.4 83.0 67.5 87.3
SHOT-IM (Avg.) 57.9 83.6 88.8 72.4 74.0 79.0 76.1 60.6 90.1 81.9 68.3 88.5 76.8
SHOT (2019) 65.0 85.0 93.3 75.7 79.3 88.9 80.5 65.3 90.1 80.9 67.0 86.3
SHOT (2020) 64.1 82.0 92.7 77.6 74.8 90.7 80.0 63.5 88.4 79.9 66.8 85.0
SHOT (2021) 65.2 88.7 92.2 75.7 78.8 86.8 78.5 64.1 90.1 80.9 65.3 86.0
SHOT (Avg.) 64.8 85.2 92.7 76.3 77.6 88.8 79.7 64.3 89.5 80.6 66.4 85.8 79.3
Methods@ODA Ar->Cl Ar->Pr Ar->Re Cl->Ar Cl->Pr Cl->Re Pr->Ar Pr->Cl Pr->Re Re->Ar Re->Cl Re->Pr Avg.
srconly (2019) 37.4 54.7 69.9 34.2 44.3 49.7 37.7 30.1 56.2 50.6 35.2 61.6
srconly (2020) 36.4 55.0 69.0 33.3 44.7 47.8 34.6 29.2 55.7 53.2 36.0 62.4
srconly (2021) 35.1 54.8 68.4 33.8 44.1 50.1 38.2 28.1 58.3 50.3 34.1 62.9
srconly (Avg.) 36.3 54.8 69.1 33.8 44.4 49.2 36.8 29.2 56.8 51.4 35.1 62.3 46.6
SHOT-IM (2019) 61.6 80.1 84.4 61.8 74.0 81.9 63.6 58.5 83.1 68.4 63.7 82.2
SHOT-IM (2020) 63.4 76.0 83.2 61.4 74.3 78.7 63.8 59.6 83.1 70.0 61.8 82.7
SHOT-IM (2021) 62.4 77.3 84.1 59.6 71.9 77.7 66.7 58.0 83.0 68.9 60.6 81.6
SHOT-IM (Avg.) 62.5 77.8 83.9 60.9 73.4 79.4 64.7 58.7 83.1 69.1 62.0 82.1 71.5
SHOT (2019) 63.9 80.6 85.6 63.6 77.1 83.2 64.9 58.3 83.2 69.7 65.2 82.8
SHOT (2020) 64.0 80.4 84.7 63.4 75.3 81.6 65.1 60.9 82.8 69.9 64.4 82.4
SHOT (2021) 65.6 80.2 83.8 62.2 73.7 78.8 65.9 58.8 83.9 69.2 64.1 81.7
SHOT (Avg.) 64.5 80.4 84.7 63.1 75.4 81.2 65.3 59.3 83.3 69.6 64.6 82.3 72.8

Table 8 [MSDA/ MTDA results on Office-Caltech]

Methods@MSDA ->A ->C ->D ->W Avg.
srconly (2019) 95.2 93.9 99.4 98.0
srconly (2020) 95.4 93.5 98.7 98.6
srconly (2021) 95.6 93.7 98.7 98.3
srconly (Avg.) 95.4 93.7 98.9 98.3 96.6
SHOT-IM (2019) 95.8 96.0 99.4 99.7
SHOT-IM (2020) 96.5 95.9 97.5 99.7
SHOT-IM (2021) 96.4 96.3 98.7 99.7
SHOT-IM (Avg.) 96.2 96.1 98.5 99.7 97.6
SHOT (2019) 96.2 95.9 98.7 99.7
SHOT (2020) 96.5 96.1 98.7 99.7
SHOT (2021) 96.6 96.6 98.1 99.7
SHOT (Avg.) 96.4 96.2 98.5 99.7 97.7
Methods@MTDA A-> C-> D-> W-> Avg.
srconly (2019) 90.4 95.9 90.3 90.6
srconly (2020) 91.2 95.9 90.2 91.1
srconly (2021) 90.5 96.5 90.2 91.1
srconly (Avg.) 90.7 96.1 90.2 90.9 92.0
SHOT-IM (2019) 96.6 97.5 96.3 96.0
SHOT-IM (2020) 95.1 96.7 96.3 96.4
SHOT-IM (2021) 95.4 97.3 96.3 96.0
SHOT-IM (Avg.) 95.7 97.2 96.3 96.1 96.3
SHOT (2019) 96.6 97.5 96.4 96.0
SHOT (2020) 95.4 97.0 96.5 96.7
SHOT (2021) 96.6 97.5 96.0 96.0
SHOT (Avg.) 96.2 97.3 96.3 96.2 96.5

Table 9 [PDA results on ImageNet->Caltech]

Methods@PDA 2019 2020 2021 Avg.
srconly 69.7 69.7 69.7 69.7
SHOT-IM 81.1 82.2 81.8 81.7
SHOT 83.2 83.3 83.4 83.3

Citation

If you find this code useful for your research, please cite our paper

@inproceedings{liang2020shot,
    title={Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation},
    author={Liang, Jian and Hu, Dapeng and Feng, Jiashi},
    booktitle={International Conference on Machine Learning (ICML)},
    pages={xx-xx},
    month = {July},
    year={2020}
}

Contact