Question about correctness of Domain Accuracy

[johanneszellinger]

Hello everybody, we are currently examining different DA method and tried to reproduce the SDAT paper results on our system.
So far I have made two trainings with all the settings and arguments I could gather from the paper for DomainNET. The val_acc do look quite promising:

real->sketch:

sketch -> clipart

However, the Domain accuracy is somewhat confusing for us (not smoothed):
real->sketch:

sketch -> clipart:

We would have expected a graph similar to this one from the paper for the Domain Accuracy:

However, it seems this was done with the Homeoffice dataset and different settings. Question is now, can we assume that our results are correct?

[rangwani-harsh]

Thanks for your question.

The Domain accuracy fluctuates as it's adversarial training (i.e. discriminator will try to increase the domain accuracy, whereas the feature extractor will decrease it).

In the DomainNet cases you have mentioned, you can observe the mean value of SDAT is similar to the plot provided by us. However, the fluctuations are function of dataset, hence can change with dataset and settings.

Hence, in my view the results produced by your experiments are correct in my opinion.

Thanks
Harsh

[johanneszellinger]

Perfect, thank you for the information. We where confused by the slight increase of the DA after the initial dip (better seen with a smoothed plot) - but with your explanation this makes more sense.