OMG! RuntimeError: $ Torch: not enough memory: you tried to allocate 3421GB. Buy new RAM!?

[xiaoleihuang]

I was trying to run the program by
python train_wc.py --train_file ./data/eng.train --dev_file ./data/eng.testa --test_file ./data/eng.testb --checkpoint ./checkpoint/ner_ --caseless --fine_tune --high_way --co_train --least_iters 100 > record.txt

I got the following error:

Traceback (most recent call last):
File "train_wc.py", line 136, in
ner_model = LM_LSTM_CRF(len(l_map), len(c_map), args.char_dim, args.char_hidden, args.char_layers, args.word_dim, args.word_hidden, args.word_layers, len(f_map), args.drop_out, large_CRF=args.small_crf, if_highway=args.high_way, in_doc_words=in_doc_words, highway_layers = args.highway_layers)
File "/nas/home/xhuang/project_ner/LM-LSTM-CRF/model/lm_lstm_crf.py", line 63, in init
self.crf = crf.CRF_L(word_hidden_dim, tagset_size)
File "/nas/home/xhuang/project_ner/LM-LSTM-CRF/model/crf.py", line 29, in init
self.hidden2tag = nn.Linear(hidden_dim, self.tagset_size * self.tagset_size, bias=if_bias)
File "/nas/home/xhuang/anaconda3/envs/ner/lib/python3.6/site-packages/torch/nn/modules/linear.py", line 41, in init
self.weight = Parameter(torch.Tensor(out_features, in_features))
RuntimeError: $ Torch: not enough memory: you tried to allocate 3421GB. Buy new RAM! at /opt/conda/conda-bld/pytorch_1524584710464/work/aten/src/TH/THGeneral.c:218

[LiyuanLucasLiu]

It may due to the runtime environment or input format, pls include more details
btw, GPU is highly recommended for training NNs

[xiaoleihuang]

OK. Thanks. I think the main reason was I installed pytorch 0.4.0