Using Windows, can't train model, getting errors about vgg16

[aaronjolson]

I am trying to work with the fast_neural_style example https://github.com/pytorch/examples/tree/master/fast_neural_style

I read through the readme several times and downloaded all needed files mentioned, then I ran

C:\Users\Aaron\Documents\fast-neural-style>python neural_style/neural_style.py train --dataset train2017 --style-image images/style-images/2.jpg --save-model-dir saved-models --epochs 2 --cuda 1
usage: neural_style.py train [-h] [--epochs EPOCHS] [--batch-size BATCH_SIZE]
                             --dataset DATASET [--style-image STYLE_IMAGE]
                             --vgg-model-dir VGG_MODEL_DIR --save-model-dir
                             SAVE_MODEL_DIR [--image-size IMAGE_SIZE]
                             [--style-size STYLE_SIZE] --cuda CUDA
                             [--seed SEED] [--content-weight CONTENT_WEIGHT]
                             [--style-weight STYLE_WEIGHT] [--lr LR]
                             [--log-interval LOG_INTERVAL]
neural_style.py train: error: the following arguments are required: --vgg-model-dir

Which the readme says nothing about. I then made an empty directory in the project root called vgg and ran the command again

(base) C:\Users\Aaron\Documents\fast-neural-style>python neural_style/neural_style.py train --dataset train2017 --style-image images/style-images/2.jpg --vgg-model-dir vgg  --save-model-dir saved-models --epochs 2 --cuda 1
C:\Users\Aaron\Anaconda3\lib\site-packages\torchvision\transforms\transforms.py:188: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  "please use transforms.Resize instead.")
'wget' is not recognized as an internal or external command,
operable program or batch file.
Traceback (most recent call last):
  File "neural_style/neural_style.py", line 210, in <module>
    main()
  File "neural_style/neural_style.py", line 204, in main
    train(args)
  File "neural_style/neural_style.py", line 41, in train
    utils.init_vgg16(args.vgg_model_dir)
  File "C:\Users\Aaron\Documents\fast-neural-style\neural_style\utils.py", line 70, in init_vgg16
    vgglua = load_lua(os.path.join(model_folder, 'vgg16.t7'))
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 606, in load_lua
    with open(filename, 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'vgg\\vgg16.t7'

So I went and downloaded wget for Windows https://eternallybored.org/misc/wget/ and added the exe to my path, restarted my Anaconda prompt and then it downloaded the vgg16 model.

(base) C:\Users\Aaron\Documents\fast-neural-style>python neural_style/neural_style.py train --dataset train2017 --style-image images/style-images/2.jpg --vgg-model-dir vgg  --save-model-dir saved-models --epochs 2 --cuda 1
C:\Users\Aaron\Anaconda3\lib\site-packages\torchvision\transforms\transforms.py:188: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  "please use transforms.Resize instead.")
--2018-06-29 18:06:05--  https://www.dropbox.com/s/76l3rt4kyi3s8x7/vgg16.t7?dl=1
Resolving www.dropbox.com (www.dropbox.com)... 162.125.3.1
Connecting to www.dropbox.com (www.dropbox.com)|162.125.3.1|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: /s/dl/76l3rt4kyi3s8x7/vgg16.t7 [following]
--2018-06-29 18:06:05--  https://www.dropbox.com/s/dl/76l3rt4kyi3s8x7/vgg16.t7
Reusing existing connection to www.dropbox.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://ucbb0bb9ba3b2c7e5d05c57f9bb8.dl.dropboxusercontent.com/cd/0/get/AKKR-EpfnYnls1gRv_prqzvUcfHyPNBGPkKr3WEMwMcxUlE_UfjKY6ilAGjky_PvlZh5Relm-vYj9pwE-IMHgu6s-kS6RaiKhVhrSw4MZ5Em334q_YXSRsW76rAchlu3wrl8lmayBboIaTfAdXUbPtEQm9f-YV03JYP-OdU3xKFypp438CuuX9l9kLZZ1rqeCe0/file?dl=1 [following]
--2018-06-29 18:06:06--  https://ucbb0bb9ba3b2c7e5d05c57f9bb8.dl.dropboxusercontent.com/cd/0/get/AKKR-EpfnYnls1gRv_prqzvUcfHyPNBGPkKr3WEMwMcxUlE_UfjKY6ilAGjky_PvlZh5Relm-vYj9pwE-IMHgu6s-kS6RaiKhVhrSw4MZ5Em334q_YXSRsW76rAchlu3wrl8lmayBboIaTfAdXUbPtEQm9f-YV03JYP-OdU3xKFypp438CuuX9l9kLZZ1rqeCe0/file?dl=1
Resolving ucbb0bb9ba3b2c7e5d05c57f9bb8.dl.dropboxusercontent.com (ucbb0bb9ba3b2c7e5d05c57f9bb8.dl.dropboxusercontent.com)... 162.125.3.6
Connecting to ucbb0bb9ba3b2c7e5d05c57f9bb8.dl.dropboxusercontent.com (ucbb0bb9ba3b2c7e5d05c57f9bb8.dl.dropboxusercontent.com)|162.125.3.6|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 553452665 (528M) [application/octet-stream]
Saving to: 'vgg/vgg16.t7'

vgg/vgg16.t7                  100%[=================================================>] 527.81M   484KB/s    in 13m 21s

2018-06-29 18:19:28 (675 KB/s) - 'vgg/vgg16.t7' saved [553452665/553452665]

Traceback (most recent call last):
  File "neural_style/neural_style.py", line 210, in <module>
    main()
  File "neural_style/neural_style.py", line 204, in main
    train(args)
  File "neural_style/neural_style.py", line 41, in train
    utils.init_vgg16(args.vgg_model_dir)
  File "C:\Users\Aaron\Documents\fast-neural-style\neural_style\utils.py", line 70, in init_vgg16
    vgglua = load_lua(os.path.join(model_folder, 'vgg16.t7'))
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 608, in load_lua
    return reader.read()
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 593, in read
    return self.read_object()
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 523, in wrapper
    result = fn(self, *args, **kwargs)
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 546, in read_object
    return reader_registry[cls_name](self, version)
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 243, in read_nn_class
    attributes = reader.read()
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 595, in read
    return self.read_table()
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 523, in wrapper
    result = fn(self, *args, **kwargs)
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 571, in read_table
    k = self.read()
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 595, in read
    return self.read_table()
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 523, in wrapper
    result = fn(self, *args, **kwargs)
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 572, in read_table
    v = self.read()
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 593, in read
    return self.read_object()
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 523, in wrapper
    result = fn(self, *args, **kwargs)
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 546, in read_object
    return reader_registry[cls_name](self, version)
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 318, in wrapper
    obj = build_fn(reader, version)
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 318, in wrapper
    obj = build_fn(reader, version)
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 243, in read_nn_class
    attributes = reader.read()
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 595, in read
    return self.read_table()
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 523, in wrapper
    result = fn(self, *args, **kwargs)
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 572, in read_table
    v = self.read()
  File "C:\Users\Aaron\Anaconda3\lib\site-packages\torch\utils\serialization\read_lua_file.py", line 598, in read
    "corrupted.".format(typeidx))
torch.utils.serialization.read_lua_file.T7ReaderException: unknown type id 1056626884. The file may be corrupted.

Not sure what to do next

Made an issue for three reasons

1. this should be mentioned in the README
2. In case anyone else runs into this
3. If anyone has any suggestions?

[soumith]

cc: @peterjc123 can you issue to make sure we use cross-platform download command. We can probably even use torch.utils.download_url (or something like that which we have)

[peterjc123]

So this issue can be split into 2 parts:

1. Cross-platform file download utilities. Unfortunately, the download_url is in pytorch/vision, should I move that to pytorch/pytorch, @soumith ?
2. Probably cross-platform load_lua. I'm sure how to do this. You may have to specify long_size=8 by yourself.

[soumith]

@peterjc123 torch.utils.modelzoo has _download_url_to_file, which is probably sufficient?
https://github.com/pytorch/pytorch/blob/master/torch/utils/model_zoo.py#L69

[soumith]

about (2), I can re-host the model after it's loaded via load_lua and then saved back to disk in Python pickle format

[peterjc123]

@soumith Okay!

[peterjc123]

@soumith Should I change this in model_zoo.py or move it somewhere around?

[soumith]

just change it in model_zoo.py for now.