Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Very slow startup even with increasing cache size #7

Closed
MarkusEich opened this issue Nov 18, 2015 · 3 comments
Closed

Very slow startup even with increasing cache size #7

MarkusEich opened this issue Nov 18, 2015 · 3 comments

Comments

@MarkusEich
Copy link

After starting the runme.sh script it takes EVERY time around 3! minutes until the window appears.
Even after setting the CUDA_CACHE_MAXSIZE=2147483648, as recommended.

Is there any way to speed up the process?

System used: Ubuntu 15.10, CUDA 7.5, NVIDIA GTX860M (4GB RAM), 16GB RAM, Intel I7 CPU

@padeler
Copy link
Member

padeler commented Nov 18, 2015

The slow startup speed is due to the JIT compilation of the CUDA code. Since the generated binaries are bigger than the default cache, the workaround is to increase the max cuda cache size (this is what nvidia also suggests). Maybe your cache size is set somewhere else? Try setting it in your .bashrc file

@MarkusEich
Copy link
Author

Checked setting with env. Gives a chache size of CUDA_CACHE_MAXSIZE=2147483648. Even tried with double size (4GB). The folder ~/.nv is there and readable/writable.

I read that JIT compilation takes place if the host architecture is different from the compile architecture. Maybe if all features are compiled for all architectures (i.e. sm_10,sm_13,sm_20,sm_30,sm_35), the JIT will not be necessary (but I might be wrong).

Anyone else having the slow start-up issue?

@MarkusEich
Copy link
Author

Solved. I had to set explicitly export CUDA_CACHE_DISABLE=0. Thought it would have been enabled by default. First run now takes 2 minutes, second run 2 seconds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants