Questions about the easy_margin option

[fengshikun]

Thanks for your amazing work! After reading your paper and some code in this project, i have some questions:

1. What's the meaning of option easy_margin?

    if args.easy_margin:
      cond = mx.symbol.Activation(data=cos_t, act_type='relu')
      #cond_v = cos_t - 0.4
      #cond = mx.symbol.Activation(data=cond_v, act_type='relu')
    else:
      cond_v = cos_t - threshold
      cond = mx.symbol.Activation(data=cond_v, act_type='relu')

2. Why do relu activation on cos_t, will it let the minus value in fc7 not use amsoftmax's margin to do finnal softmax loss computing?

[nttstar]

'easy_margin' means we just add margin on the subset of "WX>0". It works but the defined formula is not monotone decreasing anymore.
'relu' here can be thought of 'if-else' condition.

[hustzeyu]

What are the advantages of "easy-margin"? @nttstar @fengshikun

[BrettLL]

hi,@nttstar , when I set easy_margin=false, what does "zy_keep = zy - smm" mean? According to my understandings, it means "zy_keep = scos(theta)-smsin(m)". however, I can not understand m*sin(m), waiting for your reply~