no torch.nn.functional.normalize

[eeric]

AttributeError: 'module' object has no attribute 'normalize'

File "/home/yq/work/face_class/diracnets/diracnet.py", line 97, in block
w = beta * F.normalize(w.view(w.size(0), -1)).view_as(w) + alpha * delta

[szagoruyko]

update your pytorch

[eeric]

unfortunately, I didn't look up it. Could you provide the link address?

[eeric]

def normalize(input, p=2, dim=1, eps=1e-12):
return input / input.norm(p, dim).clamp(min=eps).expand_as(input)
did the above function be for torch.nn.functional.normalize?

[szagoruyko]

https://github.com/pytorch/pytorch/blob/master/torch/nn/functional.py#L883

from http://pytorch.org: conda install -c soumith pytorch

[eeric]

the function was same that your provided, but the test accu was lower on CIFAR-10 set

the parameter named 'True' of input.norm was deleted, program was implemented successfully, as following
def normalize(input, p=2, dim=1, eps=1e-12):
return input / input.norm(p, dim).clamp(min=eps).expand_as(input),

modified as that w = beta * normalize(w.view(w.size(0), -1)).view_as(w) + alpha * delta

the training process below:
==> id: ./log/result (3/200), test_acc: 50.49

and test_acc value more volatile.

Train_loss was be 'nan' on the 87th iteration and test_acc reached 83.7%. From next iteration to end the test_acc became 10.0% , why?

[szagoruyko]

@eeric why did you modify the code? It might diverge for two reasons:

• your pytorch is old and has unfixed bugs
• the modifications you made are incorrect.

Please make sure you're running the code with latest pytorch and no modifications to the code.

[eeric]

1.maybe, I attempted to modify lr=0.01 and delete F.normalize.
the result was as seem be better,as following
(60/200), test_acc: 92.86

2.my edition accroding to script below:
pip install http://download.pytorch.org/whl/cu80/torch-0.1.12.post2-cp27-none-linux_x86_64.whl
pip install torchvision
how to your edition?
3.modified in diracnet.py: w = beta * normalize(w.view(w.size(0), -1)).view_as(w) + alpha * delta, normalize function as below:
def normalize(input, p=2, dim=1, eps=1e-12):
return input / input.norm(p, dim).clamp(min=eps).expand_as(input),
the parameter named 'True' of input.norm was deleted.
lr=0.01
during 200 iteration, train_loss wasn't be 'nan', and the highest value of test_acc reached 94.22%.