Failing to get it working on mac osx 11.2

[smoh]

Hi, I'm trying out v3.0.0 available on pypi on mac osx 11.2.2 and getting errors difficult to understand running the eight schools example.

OS: mac osx 11.2.2
python: Python 3.9.2
C++ compiler: clang

❯ clang --version
Apple clang version 12.0.0 (clang-1200.0.32.29)
Target: x86_64-apple-darwin20.3.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Steps to reproduce the error:

conda create -n pystan3 python=3
conda activate pystan3
pip install pystan
python pystan3-test.py

pystan3-test.py has eight schools example copy-pasted.

Full traceback (This was the second time running so it used the cached model):

❯ python pystan3-test.py
Building...
Messages from stanc:
Warning:
  The parameter mu has no priors.
Warning:
  The parameter tau has no priors.
Found model in cache. Done.
Sampling...
Building...
Building...
Messages from stanc:
Warning:
  The parameter mu has no priors.
Warning:
  The parameter tau has no priors.
Found model in cache. Done.
Sampling...
Messages from stanc:
Warning:
  The parameter mu has no priors.
Warning:
  The parameter tau has no priors.
Found model in cache. Done.
Sampling...
Exception in callback handle_create_fit.<locals>._services_call_done({'done': True, 'metadata': {'fit': {'name': 'models/dvqzb...fits/3piop3es'}}, 'name': 'operations/3piop3es', 'result': {'code': 400, 'message': "Exception du...'\\'\\'\\n']`", 'status': 'Bad Request'}})(<Task finishe...executable.')>) at /Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/views.py:367
handle: <Handle handle_create_fit.<locals>._services_call_done({'done': True, 'metadata': {'fit': {'name': 'models/dvqzb...fits/3piop3es'}}, 'name': 'operations/3piop3es', 'result': {'code': 400, 'message': "Exception du...'\\'\\'\\n']`", 'status': 'Bad Request'}})(<Task finishe...executable.')>) at /Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/views.py:367>
Traceback (most recent call last):
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/sitException in callback handle_create_fit.<locals>._services_call_done({'done': True, 'metadata': {'fit': {'name': 'models/dvqzb...fits/zkaanggp'}}, 'name': 'operations/zkaanggp', 'result': {'code': 400, 'message': "Exception du...'\\'\\'\\n']`", 'status': 'Bad Request'}})(<Task finishe...executable.')>) at /Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/views.py:367
handle: <Handle handle_create_fit.<locals>._services_call_done({'done': True, 'metadata': {'fit': {'name': 'models/dvqzb...fits/zkaanggp'}}, 'name': 'operations/zkaanggp', 'result': {'code': 400, 'message': "Exception du...'\\'\\'\\n']`", 'status': 'Bad Request'}})(<Task finishe...executable.')>) at /Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/views.py:367>
Traceback (most recent call last):
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/views.py", line 392, in _services_call_done
    httpstan.cache.delete_fit(operation["metadata"]["fit"]["name"])
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/cache.py", line 140, in delete_fit
    path.unlink()
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/pathlib.py", line 1343, in unlink
    self._accessor.unlink(self)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/soh/Library/Caches/httpstan/4.4.1/models/dvqzbpzl/fits/zkaanggp.jsonlines.lz4'
e-packages/httpstan/views.py", line 392, in _services_call_done
    httpstan.cache.delete_fit(operation["metadata"]["fit"]["name"])
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/cache.py", line 140, in delete_fit
    path.unlink()
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/pathlib.py", line 1343, in unlink
    self._accessor.unlink(self)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/soh/Library/Caches/httpstan/4.4.1/models/dvqzbpzl/fits/3piop3es.jsonlines.lz4'
Exception in callback handle_create_fit.<locals>._services_call_done({'done': True, 'metadata': {'fit': {'name': 'models/dvqzb...fits/gjvmgwu6'}}, 'name': 'operations/gjvmgwu6', 'result': {'code': 400, 'message': "Exception du...'\\'\\'\\n']`", 'status': 'Bad Request'}})(<Task finishe...executable.')>) at /Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/views.py:367
handle: <Handle handle_create_fit.<locals>._services_call_done({'done': True, 'metadata': {'fit': {'name': 'models/dvqzb...fits/gjvmgwu6'}}, 'name': 'operations/gjvmgwu6', 'result': {'code': 400, 'message': "Exception du...'\\'\\'\\n']`", 'status': 'Bad Request'}})(<Task finishe...executable.')>) at /Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/views.py:367>
Traceback (most recent call last):
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/views.py", line 392, in _services_call_done
    httpstan.cache.delete_fit(operation["metadata"]["fit"]["name"])
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/cache.py", line 140, in delete_fit
    path.unlink()
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/pathlib.py", line 1343, in unlink
    self._accessor.unlink(self)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/soh/Library/Caches/httpstan/4.4.1/models/dvqzbpzl/fits/gjvmgwu6.jsonlines.lz4'
Exception in callback handle_create_fit.<locals>._services_call_done({'done': True, 'metadata': {'fit': {'name': 'models/dvqzb...fits/zjaa3wca'}}, 'name': 'operations/zjaa3wca', 'result': {'code': 400, 'message': "Exception du...'\\'\\'\\n']`", 'status': 'Bad Request'}})(<Task finishe...executable.')>) at /Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/views.py:367
handle: <Handle handle_create_fit.<locals>._services_call_done({'done': True, 'metadata': {'fit': {'name': 'models/dvqzb...fits/zjaa3wca'}}, 'name': 'operations/zjaa3wca', 'result': {'code': 400, 'message': "Exception du...'\\'\\'\\n']`", 'status': 'Bad Request'}})(<Task finishe...executable.')>) at /Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/views.py:367>
Traceback (most recent call last):
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/views.py", line 392, in _services_call_done
    httpstan.cache.delete_fit(operation["metadata"]["fit"]["name"])
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/cache.py", line 140, in delete_fit
    path.unlink()
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/pathlib.py", line 1343, in unlink
    self._accessor.unlink(self)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/soh/Library/Caches/httpstan/4.4.1/models/dvqzbpzl/fits/zjaa3wca.jsonlines.lz4'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/spawn.py", line 116, in spawn_main
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/spawn.py", line 125, in _main
    exitcode = _main(fd, parent_sentinel)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/spawn.py", line 125, in _main
    prepare(preparation_data)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/spawn.py", line 236, in prepare
    prepare(preparation_data)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
    _fixup_main_from_path(data['init_main_from_path'])
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/runpy.py", line 268, in run_path
    main_content = runpy.run_path(main_path,
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/runpy.py", line 268, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/runpy.py", line 97, in _run_module_code
    return _run_module_code(code, init_globals, run_name,
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
    _run_code(code, mod_globals, init_globals,
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/runpy.py", line 87, in _run_code
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/soh/pystan3-test.py", line 29, in <module>
    exec(code, run_globals)
  File "/Users/soh/pystan3-test.py", line 29, in <module>
    fit = posterior.sample(num_chains=2, num_samples=1000)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/stan/model.py", line 68, in sample
    fit = posterior.sample(num_chains=2, num_samples=1000)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/stan/model.py", line 68, in sample
    return self.hmc_nuts_diag_e_adapt(**kwargs)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/stan/model.py", line 88, in hmc_nuts_diag_e_adapt
    return self.hmc_nuts_diag_e_adapt(**kwargs)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/stan/model.py", line 88, in hmc_nuts_diag_e_adapt
    return self._create_fit(kwargs)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/stan/model.py", line 272, in _create_fit
    return self._create_fit(kwargs)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/stan/model.py", line 272, in _create_fit
    return asyncio.run(go())
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/asyncio/runners.py", line 44, in run
    return asyncio.run(go())
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
    return loop.run_until_complete(main)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/stan/model.py", line 202, in go
    return future.result()
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/stan/model.py", line 202, in go
    raise RuntimeError(operation["result"]["message"])
RuntimeError: Exception during call to services function: `RuntimeError('\n        An attempt has been made to start a new process before the\n        current process has finished its bootstrapping phase.\n\n        This probably means that you are not using fork to start your\n        child processes and you have forgotten to use the proper idiom\n        in the main module:\n\n            if __name__ == \'__main__\':\n                freeze_support()\n                ...\n\n        The "freeze_support()" line can be omitted if the program\n        is not going to be frozen to produce an executable.')`, traceback: `['  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/services_stub.py", line 103, in call\n    future = asyncio.get_running_loop().run_in_executor(executor, lazy_function_wrapper_partial)  # type: ignore\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/asyncio/base_events.py", line 814, in run_in_executor\n    executor.submit(func, *args), loop=self)\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/concurrent/futures/process.py", line 697, in submit\n    self._adjust_process_count()\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/concurrent/futures/process.py", line 675, in _adjust_process_count\n    p.start()\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/process.py", line 121, in start\n    self._popen = self._Popen(self)\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/context.py", line 284, in _Popen\n    return Popen(process_obj)\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in __init__\n    super().__init__(process_obj)\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__\n    self._launch(
process_obj)\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 42, in _launch\n    prep_data = spawn.get_preparation_data(process_obj._name)\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/spawn.py", line 154, in get_preparation_data\n    _check_not_importing_main()\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/spawn.py", line 134, in _check_not_importing_main\n    raise RuntimeError(\'\'\'\n']`
    raise RuntimeError(operation["result"]["message"])
RuntimeError: Exception during call to services function: `RuntimeError('\n        An attempt has been made to start a new process before the\n        current process has finished its bootstrapping phase.\n\n        This probably means that you are not using fork to start your\n        child processes and you have forgotten to use the proper idiom\n        in the main module:\n\n            if __name__ == \'__main__\':\n                freeze_support()\n                ...\n\n        The "freeze_support()" line can be omitted if the program\n        is not going to be frozen to produce an executable.')`, traceback: `['  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/services_stub.py", line 103, in call\n    future = asyncio.get_running_loop().run_in_executor(executor, lazy_function_wrapper_partial)  # type: ignore\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/asyncio/base_events.py", line 814, in run_in_executor\n    executor.submit(func, *args), loop=self)\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/concurrent/futures/process.py", line 697, in submit\n    self._adjust_process_count()\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/concurrent/futures/process.py", line 675, in _adjust_process_count\n    p.start()\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/process.py", line 121, in start\n    self._popen = self._Popen(self)\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/context.py", line 284, in _Popen\n    return Popen(process_obj)\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in __init__\n    super().__init__(process_obj)\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__\n    self._launch(process_obj)\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 42, in _launch\n    prep_data = spawn.get_preparation_data(process_obj._name)\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/spawn.py", line 154, in get_preparation_data\n    _check_not_importing_main()\n', '  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/spawn.py", line 134, in _check_not_importing_main\n    raise RuntimeError(\'\'\'\n']`
Traceback (most recent call last):
  File "/Users/soh/pystan3-test.py", line 29, in <module>
    fit = posterior.sample(num_chains=2, num_samples=1000)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/stan/model.py", line 68, in sample
    return self.hmc_nuts_diag_e_adapt(**kwargs)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/stan/model.py", line 88, in hmc_nuts_diag_e_adapt
    return self._create_fit(kwargs)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/stan/model.py", line 272, in _create_fit
    return asyncio.run(go())
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/stan/model.py", line 202, in go
    raise RuntimeError(operation["result"]["message"])
RuntimeError: Exception during call to services function: `BrokenProcessPool('A process in the process pool was terminated abruptly while the future was running or pending.')`, traceback: `['  File "/Users/soh/miniconda3/envs/pystan3/lib/python3.9/site-packages/httpstan/services_stub.py", line 150, in call\n    future.result()\n']`

All I understand is that some file was missing...:

[Errno 2] No such file or directory: '/Users/soh/Library/Caches/httpstan/4.4.1/models/dvqzbpzl/fits/zkaanggp.jsonlines.lz4'

Also, I think some documentation on how caching works would be really useful. I guess it is doing some sort of hashing of the stan code but is there any option to control where cache is stored?

Thanks for your help.

[riddell-stan]

Thanks for the report.

We haven't been testing on 11.2 (10.14 and 10.15 only).

I'll look into getting access to a 11.2 machine. Might take some time. None of the pystan developers have a macOS machine.

[ahartikainen]

Can you try running your code under the following block?

if __name__ == "__main__":

If that doesn't help, change multiprocess from spawn to fork

import multiprocessing
multiprocessing.set_start_method("fork")

[smoh]

Can you try running your code under the following block?

if __name__ == "__main__":

• If I put the whole code from import stan in main, it works fine:

❯ cat pystan3-test.py
if __name__ == "__main__":
    import stan

    schools_code = """
    data {
      int<lower=0> J;         // number of schools
      real y[J];              // estimated treatment effects
      real<lower=0> sigma[J]; // standard error of effect estimates
    }
    parameters {
      real mu;                // population treatment effect
      real<lower=0> tau;      // standard deviation in treatment effects
      vector[J] eta;          // unscaled deviation from mu by school
    }
    transformed parameters {
      vector[J] theta = mu + tau * eta;        // school treatment effects
    }
    model {
      target += normal_lpdf(eta | 0, 1);       // prior log-density
      target += normal_lpdf(y | theta, sigma); // log-likelihood
    }
    """

    schools_data = {"J": 8,
                    "y": [28,  8, -3,  7, -1,  1, 18, 12],
                    "sigma": [15, 10, 16, 11,  9, 11, 10, 18]}

    posterior = stan.build(schools_code, data=schools_data)
    fit = posterior.sample(num_chains=2, num_samples=1000)

~ via 🐍 v3.9.2 via 🅒 pystan3
❯ python pystan3-test.py
Building...
Messages from stanc:
Warning:
  The parameter mu has no priors.
Warning:
  The parameter tau has no priors.
Found model in cache. Done.
Sampling...
Messages received during sampling:
  Gradient evaluation took 2.3e-05 seconds
  1000 transitions using 10 leapfrog steps per transition would take 0.23 seconds.
  Adjust your expectations accordingly!
  Gradient evaluation took 2.2e-05 seconds
  1000 transitions using 10 leapfrog steps per transition would take 0.22 seconds.
  Adjust your expectations accordingly!
 4000/4000 [============================] 100%  1 sec/0
Done.

• But it looks like I need to be in ifmain block when I import stan because otherwise, it works but with "leaked semaphore objects" message:

❯ cat pystan3-test2.py
import stan

schools_code = """
data {
  int<lower=0> J;         // number of schools
  real y[J];              // estimated treatment effects
  real<lower=0> sigma[J]; // standard error of effect estimates
}
parameters {
  real mu;                // population treatment effect
  real<lower=0> tau;      // standard deviation in treatment effects
  vector[J] eta;          // unscaled deviation from mu by school
}
transformed parameters {
  vector[J] theta = mu + tau * eta;        // school treatment effects
}
model {
  target += normal_lpdf(eta | 0, 1);       // prior log-density
  target += normal_lpdf(y | theta, sigma); // log-likelihood
}
"""

schools_data = {"J": 8,
                "y": [28,  8, -3,  7, -1,  1, 18, 12],
                "sigma": [15, 10, 16, 11,  9, 11, 10, 18]}

if __name__ == "__main__":
    posterior = stan.build(schools_code, data=schools_data)
    fit = posterior.sample(num_chains=2, num_samples=1000)

~ via 🐍 v3.9.2 via 🅒 pystan3
❯ python pystan3-test2.py
Building...
Messages from stanc:
Warning:
  The parameter mu has no priors.
Warning:
  The parameter tau has no priors.
Found model in cache. Done.
Sampling...
Messages received during sampling:
  Gradient evaluation took 2.6e-05 seconds
  1000 transitions using 10 leapfrog steps per transition would take 0.26 seconds.
  Adjust your expectations accordingly!
  Gradient evaluation took 1.6e-05 seconds
  1000 transitions using 10 leapfrog steps per transition would take 0.16 seconds.
  Adjust your expectations accordingly!
 4000/4000 [============================] 100%  1 sec/0
Done.
/Users/soh/miniconda3/envs/pystan3/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 10 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

If that doesn't help, change multiprocess from spawn to fork

import multiprocessing
multiprocessing.set_start_method("fork")

Putting this before import stan also works:

Building...
Messages from stanc:
Warning:
  The parameter mu has no priors.
Warning:
  The parameter tau has no priors.
Found model in cache. Done.
Sampling...
Messages received during sampling:
  Gradient evaluation took 3.7e-05 seconds
  1000 transitions using 10 leapfrog steps per transition would take 0.37 seconds.
  Adjust your expectations accordingly!
  Gradient evaluation took 8e-06 seconds
  1000 transitions using 10 leapfrog steps per transition would take 0.08 seconds.
  Adjust your expectations accordingly!
 4000/4000 [============================] 100%  1 sec/0
Done.

[smoh]

Users having to put this

import multiprocessing
multiprocessing.set_start_method("fork")

every time when using pystan seems rather cumbersome but is this one of those 'it is what it is' situation for mac osx users with python>=3.8 due to default method being changed?

Also, for pystan can we ignore this remark on python documentation?
(On this page: https://docs.python.org/3/library/multiprocessing.html?highlight=multiprocessing#contexts-and-start-methods)

Changed in version 3.8: On macOS, the spawn start method is now the default. The fork start method should be considered unsafe as it can lead to crashes of the subprocess. See bpo-33725.

I admit I do not know the details of why it is unsafe.

[ahartikainen]

Great there is a way to get it working.

I think if __name__... is what python recommends as the correct way to handle this, but it is weird that the import needs to be in that block.

@riddell-stan do we start httpstan at import?

[riddell-stan]

Thanks. Glad you figured it out. We could really use an example which reliably fails on Github Actions (on the macOS machines). Without this we will have a hard time fixing or even learning about the problem.

…

On Sat, Mar 27, 2021, at 2:34 PM, Semyeong Oh wrote: Users having to put this import multiprocessing multiprocessing.set_start_method("fork") every time when using pystan seems rather cumbersome but is this one of those 'it is what it is' situation for mac osx users with python>=3.8 due to default method being changed? Also, for pystan can we ignore this remark on python documentation? > Changed in version 3.8: On macOS, the spawn start method is now the default. The fork start method should be considered unsafe as it can lead to crashes of the subprocess. See bpo-33725. I admit I do not know the details of why it is unsafe. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#247 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJQUBVQ62PCZ6UYRGPBBILTTFYQJXANCNFSM4Z5EPEPA>.

[riddell-stan]

This is fixed by stan-dev/httpstan#561.