Add gpu support for LRN #211

Closed
alexatknit opened this Issue Nov 13, 2015 · 17 comments

Comments

Projects
None yet
9 participants
@alexatknit

When I launch a network that uses local response normalization, it works perfectly on a CPU, but it appears to not have a gpu implementation and results in the following error when I switch to a gpu device:

tensorflow.python.framework.errors.InvalidArgumentError: Cannot assign a device to node 'LRN': Could not satisfy explicit device specification '/gpu:0' [[Node: LRN = LRN[alpha=0.0005, beta=0.75, bias=2, depth_radius=5, _device="/gpu:0"](conv1/conv1)]]

I could probably explicitly deploy this operation on the cpu, but it since this is a sliding window algorithm I'm surprised it doesn't have a gpu implementation.

@mrry

This comment has been minimized.

Show comment
Hide comment
@mrry

mrry Nov 13, 2015

Contributor

Thanks for this feature request! In the meantime, if you want to test your model without explicitly specifying the device for each op, you can do:

sess = tf.Session(tf.ConfigProto(allow_soft_placement=True))

...when constructing your session. This allows you to request a GPU device for any of your ops, and it will fall back to running on a CPU if there is no GPU kernel available.

Contributor

mrry commented Nov 13, 2015

Thanks for this feature request! In the meantime, if you want to test your model without explicitly specifying the device for each op, you can do:

sess = tf.Session(tf.ConfigProto(allow_soft_placement=True))

...when constructing your session. This allows you to request a GPU device for any of your ops, and it will fall back to running on a CPU if there is no GPU kernel available.

@alexatknit

This comment has been minimized.

Show comment
Hide comment
@alexatknit

alexatknit Nov 17, 2015

Just in case anyone is using this code, theres a typo. It should be:

sess = tf.Session(config=tf.ConfigProto(allow_soft_placement=True))

Just in case anyone is using this code, theres a typo. It should be:

sess = tf.Session(config=tf.ConfigProto(allow_soft_placement=True))
@jtoy

This comment has been minimized.

Show comment
Hide comment
@jtoy

jtoy Mar 23, 2016

just ran into this, anyone working on it?

jtoy commented Mar 23, 2016

just ran into this, anyone working on it?

@vrv

This comment has been minimized.

Show comment
Hide comment
@vrv

vrv Mar 23, 2016

Contributor

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/stream_executor/dnn.h#L822 does exist, if someone wants to plumb that call through, I think we'd have LRN support for GPU :)

Contributor

vrv commented Mar 23, 2016

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/stream_executor/dnn.h#L822 does exist, if someone wants to plumb that call through, I think we'd have LRN support for GPU :)

@apkuhar

This comment has been minimized.

Show comment
Hide comment
@apkuhar

apkuhar May 10, 2016

Any updates on this? not having this on GPU really slows things down.

apkuhar commented May 10, 2016

Any updates on this? not having this on GPU really slows things down.

@vrv

This comment has been minimized.

Show comment
Hide comment
@vrv

vrv May 11, 2016

Contributor

Right, cudnn has an LRN implementation, so the point is to have it use that one.

Contributor

vrv commented May 11, 2016

Right, cudnn has an LRN implementation, so the point is to have it use that one.

@rryan

This comment has been minimized.

Show comment
Hide comment
@rryan

rryan May 12, 2016

Member

FYI to anyone looking at contributing a fix for this -- there is some movement internally.

Member

rryan commented May 12, 2016

FYI to anyone looking at contributing a fix for this -- there is some movement internally.

@vrv

This comment has been minimized.

Show comment
Hide comment
@vrv

vrv May 12, 2016

Contributor

Thanks, removing the contributions welcome tag, since this is being worked on.

Contributor

vrv commented May 12, 2016

Thanks, removing the contributions welcome tag, since this is being worked on.

@girving

This comment has been minimized.

Show comment
Hide comment
@girving

girving Jun 6, 2016

Contributor

@vrv: Does the person working on it have a Github username?

Contributor

girving commented Jun 6, 2016

@vrv: Does the person working on it have a Github username?

@vrv

This comment has been minimized.

Show comment
Hide comment
@vrv

vrv Jun 6, 2016

Contributor

I don't know, @rryan do you know?

Contributor

vrv commented Jun 6, 2016

I don't know, @rryan do you know?

@rryan

This comment has been minimized.

Show comment
Hide comment
@rryan

rryan Jun 6, 2016

Member

Once Stream Executor support is finished I was planning to work on it -- so you can assign me if you'd like :).

Member

rryan commented Jun 6, 2016

Once Stream Executor support is finished I was planning to work on it -- so you can assign me if you'd like :).

@girving

This comment has been minimized.

Show comment
Hide comment
@girving

girving Jun 6, 2016

Contributor

@rryan: Assigned, thanks!

Contributor

girving commented Jun 6, 2016

@rryan: Assigned, thanks!

@aselle aselle added enhancement and removed enhancement labels Jun 27, 2016

@alvisememo

This comment has been minimized.

Show comment
Hide comment
@alvisememo

alvisememo Jul 6, 2016

Is there any news on this bug? Should I better compute it with different operations?

Is there any news on this bug? Should I better compute it with different operations?

@rryan

This comment has been minimized.

Show comment
Hide comment
@rryan

rryan Jul 11, 2016

Member

Sorry for the delay, I carved out some time to work on it this weekend and will be sending it out for review as soon as the GPU tests pass.

Member

rryan commented Jul 11, 2016

Sorry for the delay, I carved out some time to work on it this weekend and will be sending it out for review as soon as the GPU tests pass.

@alvisememo

This comment has been minimized.

Show comment
Hide comment
@alvisememo

alvisememo Jul 11, 2016

Thanks @rryan , looking forward!

Thanks @rryan , looking forward!

@rryan

This comment has been minimized.

Show comment
Hide comment
@rryan

rryan Jul 13, 2016

Member

Added in 35df3ed.

Member

rryan commented Jul 13, 2016

Added in 35df3ed.

@vrv

This comment has been minimized.

Show comment
Hide comment
@vrv

vrv Jul 13, 2016

Contributor

Yay!

Contributor

vrv commented Jul 13, 2016

Yay!

@vrv vrv closed this Jul 13, 2016

@aselle aselle added type:feature and removed enhancement labels Feb 9, 2017

Rbiessy pushed a commit to Rbiessy/tensorflow that referenced this issue Jun 15, 2018

[SYCL] Adds nan propagation to max pooling (#211)
Adds an option to the SYCL max pooling operation to either ignore NaNs
or to propagate them to the pool output.

Rbiessy pushed a commit to Rbiessy/tensorflow that referenced this issue Jun 15, 2018

[SYCL] Adds nan propagation to max pooling (#211)
Adds an option to the SYCL max pooling operation to either ignore NaNs
or to propagate them to the pool output.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment