Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify number of cores to be used with tensorflow #4314

Closed
Hudler opened this Issue Nov 7, 2016 · 4 comments

Comments

Projects
None yet
4 participants
@Hudler
Copy link

Hudler commented Nov 7, 2016

I am using keras on grid. There are many cores available, however I can't use all of them at same time. Therefore I would like to specify number of cores keras (tensorflow) can use. I have been looking for solution for quite some time. Is it possible?

@CCXD

This comment has been minimized.

Copy link

CCXD commented Nov 8, 2016

Python's GIL locks it to a single core per process. I believe TF supports asynchronous computation. Try putting train ops on separate threads

@Hudler

This comment has been minimized.

Copy link
Author

Hudler commented Nov 8, 2016

Of course I found the solution after posting the question...

config = tf.ConfigProto(intra_op_parallelism_threads=1, inter_op_parallelism_threads=1, \
                        allow_soft_placement=True, device_count = {'CPU': 1})
session = tf.Session(config=config)
K.set_session(session)

@Hudler Hudler closed this Nov 8, 2016

@bastia1989

This comment has been minimized.

Copy link

bastia1989 commented Oct 24, 2018

Of course I found the solution after posting the question...

config = tf.ConfigProto(intra_op_parallelism_threads=1, inter_op_parallelism_threads=1, \
                        allow_soft_placement=True, device_count = {'CPU': 1})
session = tf.Session(config=config)
K.set_session(session)

So this tells keras to use only 1 core, right? I would like to know if you set device_count = {'GPU':0} what would happen; will it use all detected CPU cores?

@mrgloom

This comment has been minimized.

Copy link

mrgloom commented Mar 20, 2019

Why allow_soft_placement=True is needed?

https://stackoverflow.com/questions/44873273/what-do-the-options-in-configproto-like-allow-soft-placement-and-log-device-plac

And maybe it's needed to set device_count={'GPU': 0} also?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.