New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Predictor interface - GPU #799

Closed
tsankar opened this Issue Jun 14, 2017 · 2 comments

Comments

Projects
None yet
2 participants
@tsankar

tsankar commented Jun 14, 2017

As referenced in many other questions (#323 #503 etc.), the Predictor interface does not by default have GPU support. How can you run prediction on a pretrained network (such as bvlc_alexnet) using GPU?

I tried the following code:

device_opts = core.DeviceOption(caffe2_pb2.CUDA, 0)

init_def = caffe2_pb2.NetDef()
with open(INIT_NET, 'r') as f:
    init_def.ParseFromString(f.read())
    init_def.device_option.CopyFrom(device_opts)
    workspace.RunNetOnce(init_def.SerializeToString())

net_def = caffe2_pb2.NetDef()
with open(PREDICT_NET, 'r') as f:
    net_def.ParseFromString(f.read())
    net_def.device_option.CopyFrom(device_opts)
    workspace.CreateNet(net_def.SerializeToString())

print 'Running net...'

tic = time.time()
workspace.FeedBlob('data', img, device_opts.SerializeToString())
workspace.RunNet('AlexNet', 1)

INIT_NET, PREDICT_NET, and img are set in the same way as in the tutorial.

But I'm getting the following error:

  File "alexnet_pretrain.py", line 153, in <module>
    init_def.ParseFromString(f.read())
  File "C:\Users\tsankar\AppData\Local\Continuum\Anaconda2\lib\site-packages\google\protobuf\message.py", line 185, in ParseFromString
    self.MergeFromString(serialized)
  File "C:\Users\tsankar\AppData\Local\Continuum\Anaconda2\lib\site-packages\google\protobuf\internal\python_message.py", line 1063, in MergeFromString
    if self._InternalParse(serialized, 0, length) != length:
  File "C:\Users\tsankar\AppData\Local\Continuum\Anaconda2\lib\site-packages\google\protobuf\internal\python_message.py", line 1099, in InternalParse
    pos = field_decoder(buffer, new_pos, end, self, field_dict)
  File "C:\Users\tsankar\AppData\Local\Continuum\Anaconda2\lib\site-packages\google\protobuf\internal\decoder.py", line 610, in DecodeRepeatedField
    raise _DecodeError('Truncated message.')
google.protobuf.message.DecodeError: Truncated message.

For some reason there's a problem parsing the .pb file, but the Predictor interface worked totally fine without using this parsing. What am I doing wrong here? Is there a better way to do this? As many others have mentioned, the Predictor interface should definitely have an option to use GPU support.

Running on Windows 10, CUDA 8.0, cuDNN 6.0

@tsankar

This comment has been minimized.

Show comment
Hide comment
@tsankar

tsankar Jun 14, 2017

Got it working by changing open(INIT_NET, 'r') to open(INIT_NET, 'rb') and changing workspace.FeedBlob('data', img, device_opts.SerializeToString()) to workspace.FeedBlob('data', img, device_option=device_opts). I'm still quite new to Caffe2 and deep learning in general, so can someone confirm for me that this in fact performing prediction?

tsankar commented Jun 14, 2017

Got it working by changing open(INIT_NET, 'r') to open(INIT_NET, 'rb') and changing workspace.FeedBlob('data', img, device_opts.SerializeToString()) to workspace.FeedBlob('data', img, device_option=device_opts). I'm still quite new to Caffe2 and deep learning in general, so can someone confirm for me that this in fact performing prediction?

@salexspb

This comment has been minimized.

Show comment
Hide comment
@salexspb

salexspb Jun 14, 2017

Contributor

It looks good. One issue with manually serializing to string is that python protobufs have limit on size. But if you don't hit it, you should be good to go.

Contributor

salexspb commented Jun 14, 2017

It looks good. One issue with manually serializing to string is that python protobufs have limit on size. But if you don't hit it, you should be good to go.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment