Skip to content

Commit

Permalink
Use multiprocessing instead of psutil to count cores (#3593)
Browse files Browse the repository at this point in the history
* Changed _cpu_count to use multiprocessing

* Added RELEASE-NOTES entry

* remove redundant import
  • Loading branch information
fonnesbeck authored and aloctavodia committed Aug 16, 2019
1 parent 1642e73 commit 517f10d
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 8 deletions.
2 changes: 2 additions & 0 deletions RELEASE-NOTES.md
Expand Up @@ -15,8 +15,10 @@
- Parallelization of population steppers (`DEMetropolis`) is now set via the `cores` argument. ([#3559](https://github.com/pymc-devs/pymc3/pull/3559))
- SMC: stabilize covariance matrix [3573](https://github.com/pymc-devs/pymc3/pull/3573)
- SMC is no longer a step method of `pm.sample` now it should be called using `pm.sample_smc` [3579](https://github.com/pymc-devs/pymc3/pull/3579)
- Now uses `multiprocessong` rather than `psutil` to count CPUs, which results in reliable core counts on Chromebooks.
- `sample_posterior_predictive` now preallocates the memory required for its output to improve memory usage. Addresses problems raised in this [discourse thread](https://discourse.pymc.io/t/memory-error-with-posterior-predictive-sample/2891/4).


## PyMC3 3.7 (May 29 2019)

### New features
Expand Down
10 changes: 2 additions & 8 deletions pymc3/parallel_sampling.py
Expand Up @@ -434,13 +434,7 @@ def _cpu_count():
that half of the cpus are only hardware threads and ignore those.
"""
try:
import psutil
cpus = psutil.cpu_count(False)
except ImportError:
try:
cpus = multiprocessing.cpu_count() // 2
except NotImplementedError:
cpus = 1
if cpus is None:
cpus = multiprocessing.cpu_count() // 2
except NotImplementedError:
cpus = 1
return cpus

0 comments on commit 517f10d

Please sign in to comment.