New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cat't load pretrained model in gpu model #323

Open
dfcv24 opened this Issue Apr 20, 2017 · 12 comments

Comments

Projects
None yet
10 participants
@dfcv24

dfcv24 commented Apr 20, 2017

http://caffe2.ai/docs/tutorial-loading-pre-trained-models.html

with open("init_net.pb") as f:
init_net = f.read()
with open("predict_net.pb") as f:
predict_net = f.read()
print workspace.has_gpu_support
p = workspace.Predictor(init_net, predict_net)
results = p.run([img])

I changed my old caffe model to caffe2, and use workspace.predictor but find hundreds of times slower when inference,I find it use cpu rather than gpu.
And I have no idea the implementation of gpu. can someone help me ?
screenshot from 2017-04-20 12 07 56
screenshot from 2017-04-20 12 08 10

@Yangqing

This comment has been minimized.

Show comment
Hide comment
@Yangqing

Yangqing Apr 20, 2017

Member

@salexspb @bwasti

Ah right, to support flexible device placement, Caffe2 now explicitly requires one to set device affinity. Bram, Alex - would you guys like to make the predictor interface accept device specifications? Under the hood we can definitely route things to GPUs (i.e. loading things to GPU, and do computation on GPU too)

Member

Yangqing commented Apr 20, 2017

@salexspb @bwasti

Ah right, to support flexible device placement, Caffe2 now explicitly requires one to set device affinity. Bram, Alex - would you guys like to make the predictor interface accept device specifications? Under the hood we can definitely route things to GPUs (i.e. loading things to GPU, and do computation on GPU too)

@salexspb

This comment has been minimized.

Show comment
Hide comment
@salexspb

salexspb Apr 20, 2017

Contributor

@bwasti mentioned that he is going to take care of this along with examples for mobile format saving / loading.

Contributor

salexspb commented Apr 20, 2017

@bwasti mentioned that he is going to take care of this along with examples for mobile format saving / loading.

@sergey-serebryakov

This comment has been minimized.

Show comment
Hide comment
@sergey-serebryakov

sergey-serebryakov Apr 20, 2017

I was able to achieve custom placing by assigning device_option of NetDef structure:

def load_net_def(def_file, device_opts=None):
    if def_file is None:
        return None
    net_def = NetDef()
    with open(def_file) as fobj:
        net_def.ParseFromString(fobj.read())
        if device_opts is not None:
            net_def.device_option.CopyFrom(device_opts)
    return net_def

sergey-serebryakov commented Apr 20, 2017

I was able to achieve custom placing by assigning device_option of NetDef structure:

def load_net_def(def_file, device_opts=None):
    if def_file is None:
        return None
    net_def = NetDef()
    with open(def_file) as fobj:
        net_def.ParseFromString(fobj.read())
        if device_opts is not None:
            net_def.device_option.CopyFrom(device_opts)
    return net_def
@Yangqing

This comment has been minimized.

Show comment
Hide comment
@Yangqing

Yangqing Apr 21, 2017

Member

@sergey-serebryakov yup, that is the right way to do it. We'll definitely want to fold it into the predictor interface.

(Just in case you are wondering, we use C++ predictor and explicitly specify devices for serving, in a similar way as you do here)

Member

Yangqing commented Apr 21, 2017

@sergey-serebryakov yup, that is the right way to do it. We'll definitely want to fold it into the predictor interface.

(Just in case you are wondering, we use C++ predictor and explicitly specify devices for serving, in a similar way as you do here)

@zhangguanqun

This comment has been minimized.

Show comment
Hide comment
@zhangguanqun

zhangguanqun Apr 26, 2017

With Caffe, switch from CPU to GPU is pretty simple, but with Caffe2, I haven't done yet.
Better to write a tutorial that how to run a network with GPU.

zhangguanqun commented Apr 26, 2017

With Caffe, switch from CPU to GPU is pretty simple, but with Caffe2, I haven't done yet.
Better to write a tutorial that how to run a network with GPU.

@KeyKy

This comment has been minimized.

Show comment
Hide comment
@KeyKy

KeyKy Apr 27, 2017

caffe2/models#5
could anyone help me?

from caffe2.proto import caffe2_pb2
from caffe2.python import core, workspace, models
import numpy as np

net_def = caffe2_pb2.NetDef()
with open('./predict_net.pb') as f:
    net_def.ParseFromString(f.read())
device_opts = caffe2_pb2.DeviceOption()
device_opts.device_type = 1
device_opts.cuda_gpu_id = 3
net_def.device_option.CopyFrom(device_opts)
predict_net = net_def.SerializeToString()

with open("./init_net.pb") as f:
    init_net = f.read()

p = workspace.Predictor(init_net, predict_net)

img = np.zeros([10, 3, 227, 227], dtype=np.float32)
results = p.run([img])

However, i get wrong type for the Blob instance. Blob contains caffe2::Tensor<caffe2::CPUContext while caller expects caffe2::Tensorcaffe2::CUDAContext

KeyKy commented Apr 27, 2017

caffe2/models#5
could anyone help me?

from caffe2.proto import caffe2_pb2
from caffe2.python import core, workspace, models
import numpy as np

net_def = caffe2_pb2.NetDef()
with open('./predict_net.pb') as f:
    net_def.ParseFromString(f.read())
device_opts = caffe2_pb2.DeviceOption()
device_opts.device_type = 1
device_opts.cuda_gpu_id = 3
net_def.device_option.CopyFrom(device_opts)
predict_net = net_def.SerializeToString()

with open("./init_net.pb") as f:
    init_net = f.read()

p = workspace.Predictor(init_net, predict_net)

img = np.zeros([10, 3, 227, 227], dtype=np.float32)
results = p.run([img])

However, i get wrong type for the Blob instance. Blob contains caffe2::Tensor<caffe2::CPUContext while caller expects caffe2::Tensorcaffe2::CUDAContext

@zhangguanqun

This comment has been minimized.

Show comment
Hide comment
@zhangguanqun

zhangguanqun Apr 28, 2017

zhangguanqun commented Apr 28, 2017

@KeyKy

This comment has been minimized.

Show comment
Hide comment
@KeyKy

KeyKy May 1, 2017

@zhangguanqun , Maybe i found a solution, here is my code

import numpy as np
#import caffe2.python._import_c_extension as C
from caffe2.proto import caffe2_pb2
import os, time

import caffe2.python._import_c_extension as C


base_path = '/data2/models/bvlc_alexnet'

device_opts = caffe2_pb2.DeviceOption()
device_opts.device_type = caffe2_pb2.CUDA
device_opts.cuda_gpu_id = 0

init_def = caffe2_pb2.NetDef()
with open(os.path.join(base_path, 'init_net.pb'), 'r') as f:
    init_def.ParseFromString(f.read())
    init_def.device_option.CopyFrom(device_opts)
    C.run_net_once(init_def.SerializeToString())

net_def = caffe2_pb2.NetDef()
with open(os.path.join(base_path, 'predict_net.pb'), 'r') as f:
    net_def.ParseFromString(f.read())
    net_def.device_option.CopyFrom(device_opts)
    C.create_net(net_def.SerializeToString())

C.feed_blob('data', np.random.rand(1, 3, 227, 227).astype(np.float32),
        device_opts.SerializeToString())

start = time.time()
for i in range(1000):
    C.run_net('AlexNet', 1)
end = time.time()
print('Run time per RunNet: {}'.format((end - start) / 1000))

KeyKy commented May 1, 2017

@zhangguanqun , Maybe i found a solution, here is my code

import numpy as np
#import caffe2.python._import_c_extension as C
from caffe2.proto import caffe2_pb2
import os, time

import caffe2.python._import_c_extension as C


base_path = '/data2/models/bvlc_alexnet'

device_opts = caffe2_pb2.DeviceOption()
device_opts.device_type = caffe2_pb2.CUDA
device_opts.cuda_gpu_id = 0

init_def = caffe2_pb2.NetDef()
with open(os.path.join(base_path, 'init_net.pb'), 'r') as f:
    init_def.ParseFromString(f.read())
    init_def.device_option.CopyFrom(device_opts)
    C.run_net_once(init_def.SerializeToString())

net_def = caffe2_pb2.NetDef()
with open(os.path.join(base_path, 'predict_net.pb'), 'r') as f:
    net_def.ParseFromString(f.read())
    net_def.device_option.CopyFrom(device_opts)
    C.create_net(net_def.SerializeToString())

C.feed_blob('data', np.random.rand(1, 3, 227, 227).astype(np.float32),
        device_opts.SerializeToString())

start = time.time()
for i in range(1000):
    C.run_net('AlexNet', 1)
end = time.time()
print('Run time per RunNet: {}'.format((end - start) / 1000))
@RobertBiehl

This comment has been minimized.

Show comment
Hide comment
@RobertBiehl

RobertBiehl May 2, 2017

@KeyKy Any update on that?

RobertBiehl commented May 2, 2017

@KeyKy Any update on that?

@tdp2110

This comment has been minimized.

Show comment
Hide comment
@tdp2110

tdp2110 May 4, 2017

@sergey-serebryakov : do you know what kind of DeviceOption to set for CUDA? I've tried DeviceOption(caffe2_pb2.CUDA) but that hits @KeyKy's error.

tdp2110 commented May 4, 2017

@sergey-serebryakov : do you know what kind of DeviceOption to set for CUDA? I've tried DeviceOption(caffe2_pb2.CUDA) but that hits @KeyKy's error.

@sergey-serebryakov

This comment has been minimized.

Show comment
Hide comment
@sergey-serebryakov

sergey-serebryakov May 4, 2017

@tdp2110 This works for me:

device_opts = core.DeviceOption(caffe2_pb2.CUDA, gpu_id)

where gpu_id is a valid integer identifier of a GPU you want to use.

sergey-serebryakov commented May 4, 2017

@tdp2110 This works for me:

device_opts = core.DeviceOption(caffe2_pb2.CUDA, gpu_id)

where gpu_id is a valid integer identifier of a GPU you want to use.

@jnulzl

This comment has been minimized.

Show comment
Hide comment
@jnulzl

jnulzl Jun 25, 2017

@dfcv24 @zhangguanqun
Hello,I want to load pretrained model in gpu mode and then predict a image by Predictor interface, but I cannot do it. do you make it????? Can you give me a simple example?? Thank you so much!

jnulzl commented Jun 25, 2017

@dfcv24 @zhangguanqun
Hello,I want to load pretrained model in gpu mode and then predict a image by Predictor interface, but I cannot do it. do you make it????? Can you give me a simple example?? Thank you so much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment