Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to specify number of threads on model training #95

Closed
alastair opened this issue Mar 3, 2020 · 2 comments · Fixed by #116
Closed

Add option to specify number of threads on model training #95

alastair opened this issue Mar 3, 2020 · 2 comments · Fixed by #116

Comments

@alastair
Copy link
Member

alastair commented Mar 3, 2020

Tools like make and waf have a -j (jobs) flag to specify the number of threads to use.
In Gaia we just use the number of available threads.

It'd be nice to add this option to train_model_from_sigs and train_model.
Should this option be in the profile file? It sounds to me like it should instead be only a parameter, but train_model calls run_tests.runTests directly, which creates a ClassificationTaskManager (which only takes a profile file as an argument).
In order to get this parameter into ClassificationTaskManager, we might have to pass it as an argument through all of these files.

Ideas?

@dbogdanov
Copy link
Member

If we think that profile files are supposed to be shareable in order to reproduce an experiment across machines, then I don't think that the number of threads should really belong there.

@alastair
Copy link
Member Author

so we should definitely take a -j flag and pass it through all of the functions. We can still default to the number of detected cores

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants