Expected object of device type cuda but got device type cpu for argument #1 'self' in call to _th_mm

[curehabit]

Bug description
Hi, when use GPU to run personality_caption project, may cause this error in project/personality_captions/transresnet/modules:402-407:
Runtime Error: Expected object of device type cuda but got device type cpu for argument #1 'self' in call to _th_mm

Reproduction steps
Set config: no_cuda = False and run project personality_captions

Expected behavior
Variable 'context_encoding' and 'candidates_encoded[img_index]' should be in same GPU device.

Additional context
I change code like this and it works:

scores = torch.mm(
candidates_encoded[img_index].to(context_encoding.device)
if not one_cand_set
else candidates_encoded.to(context_encoding.device),
context_encoding.transpose(0, 1),
)

Thanks

[stephenroller]

Thanks for filing. Can you share the command you ran for context?

Assigning to @klshuster and @EricMichaelSmith. I think the patch above is probably enough.

[klshuster]

I am having trouble reproducing on my end, could you please share the exact command you ran @curehabit?

[stephenroller]

I think the key is —no-cuda

[klshuster]

I tried with and without --no-cuda to no avail

[curehabit]

Sorry, I found I commented projects/personality_captions/interactive line:278-279 because I want to run with cuda.

#opt['no_cuda'] = True
#opt['override']['no_cuda'] = True

Than run command

python projects/personality_captions/interactive.py -mf models:personality_captions/transresnet/model

and my config:(hidden some path information with XXX)

[ optional arguments: ]
[ activation: relu ]
[ attention_dropout: 0.2 ]
[ display_examples: False ]
[ display_ignore_fields: label_candidates,text_candidates ]
[ display_prettify: False ]
[ dropout: 0.0 ]
[ embedding_size: 300 ]
[ embeddings_scale: True ]
[ ffn_size: 1200 ]
[ fixed_cands_path: None ]
[ freeze_patience: -1 ]
[ interactive_task: True ]
[ learn_positional_embeddings: False ]
[ model_parallel: False ]
[ n_decoder_layers: -1 ]
[ n_encoder_layers: -1 ]
[ n_heads: 2 ]
[ n_layers: 2 ]
[ n_positions: 1000 ]
[ n_segments: 0 ]
[ one_cand_set: False ]
[ output_scaling: 1.0 ]
[ pretrained: False ]
[ relu_dropout: 0.2 ]
[ save_world_logs: False ]
[ share_word_embeddings: True ]
[ variant: aiayn ]
[ Main ParlAI Arguments: ]
[ batchsize: 1 ]
[ datapath: XXX/ParlAI/data ]
[ datatype: train ]
[ download_path: XXX/ParlAI/downloads ]
[ dynamic_batching: None ]
[ hide_labels: False ]
[ image_mode: raw ]
[ init_opt: None ]
[ loglevel: info ]
[ multitask_weights: [1] ]
[ numthreads: 1 ]
[ task: interactive ]
[ ParlAI Model Arguments: ]
[ dict_class: None ]
[ init_model: None ]
[ model: None ]
[ model_file: XXX/ParlAI/data/models/personality_captions/transresnet/model ]
[ Local Human Arguments: ]
[ local_human_candidates_file: None ]
[ single_turn: False ]
[ ParlAI Image Preprocessing Arguments: ]
[ image_cropsize: 224 ]
[ image_size: 256 ]
[ TransresnetModel arguments: ]
[ additional_layer_dropout: 0.2 ]
[ embedding_type: None ]
[ hidden_dim: 300 ]
[ image_features_dim: 2048 ]
[ learningrate: 0.0005 ]
[ load_encoder_from: None ]
[ no_cuda: False ]
[ num_layers_all: -1 ]
[ num_layers_image_encoder: 1 ]
[ num_layers_text_encoder: 1 ]
[ truncate: 32 ]
[ Dictionary Arguments: ]
[ bpe_debug: False ]
[ dict_endtoken: end ]
[ dict_file: None ]
[ dict_initpath: None ]
[ dict_language: english ]
[ dict_lower: False ]
[ dict_max_ngram_size: -1 ]
[ dict_maxtokens: -1 ]
[ dict_minfreq: 0 ]
[ dict_nulltoken: null ]
[ dict_starttoken: start ]
[ dict_textfields: text,labels ]
[ dict_tokenizer: re ]
[ dict_unktoken: unk ]
[ BPEHelper Arguments: ]
[ bpe_add_prefix_space: None ]
[ bpe_merge: None ]
[ bpe_vocab: None ]
[ hf_skip_special_tokens: True ]

and the error information:
Traceback (most recent call last):
File "XXX/miniconda3/envs/parlai/lib/python3.6/socketserver.py", line 320, in _handle_request_noblock
self.process_request(request, client_address)
File "XXX/miniconda3/envs/parlai/lib/python3.6/socketserver.py", line 351, in process_request
self.finish_request(request, client_address)
File "XXX/miniconda3/envs/parlai/lib/python3.6/socketserver.py", line 364, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "XXX/miniconda3/envs/parlai/lib/python3.6/socketserver.py", line 724, in init
self.handle()
File "XXX/miniconda3/envs/parlai/lib/python3.6/http/server.py", line 418, in handle
self.handle_one_request()
File "XXX/miniconda3/envs/parlai/lib/python3.6/http/server.py", line 406, in handle_one_request
method()
File "projects/personality_captions/interactive.py", line 225, in do_POST
model_response = self.interactive_running(postvars)
File "projects/personality_captions/interactive.py", line 205, in interactive_running
model_res = SHARED['agent'].act()
File "XXX/ParlAI/projects/personality_captions/transresnet/transresnet.py", line 209, in act
return self.batch_act([self.observation])[0]
File "XXX/ParlAI/projects/personality_captions/transresnet/transresnet.py", line 323, in batch_act
valid_obs, image_feats, personalities
File "XXX/ParlAI/projects/personality_captions/transresnet/transresnet.py", line 271, in eval_step
k=-1 if self.fixed_cands is None else 100,
File "XXX/ParlAI/projects/personality_captions/transresnet/modules.py", line 405, in choose_best_caption
context_encoding.transpose(0, 1),
RuntimeError: Expected object of device type cuda but got device type cpu for argument #1 'self' in call to _th_mm

[EricMichaelSmith]

@klshuster You have more experience with personality_captions than I do - do you think the proposed fix in the PR description would be enough? I just made a PR for this at #2963 and tested it out - for both CUDA enabled and disabled, I get a broken image, so I'm not sure if this fix is sufficient

[github-actions]

This issue has not had activity in 30 days. Please feel free to reopen if you have more issues. You may apply the "never-stale" tag to prevent this from happening.

[klshuster]

fix was merged a long time ago, please reopen if the issue persists