Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Can't create the PPO model on Macbook M1 #1152

Closed
4 tasks done
nguyenhongson1902 opened this issue Nov 3, 2022 · 24 comments
Closed
4 tasks done

[Bug]: Can't create the PPO model on Macbook M1 #1152

nguyenhongson1902 opened this issue Nov 3, 2022 · 24 comments
Labels
bug Something isn't working

Comments

@nguyenhongson1902
Copy link

nguyenhongson1902 commented Nov 3, 2022

🐛 Bug

I'm using Macbook Pro M1 and running code in Jupyter Notebook. Every time I run the scripts, I get an error and the Kernel crashed. I've tried running the code in Google Colab and it went well. It seems to me that stable-baselines3 library doesn't support M1 chips. What do I need to do now to run the code locally?

To Reproduce


from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env

# Parallel environments
env = make_vec_env("CartPole-v1", n_envs=4)

model = PPO("MlpPolicy", env, verbose=1)
model.learn(total_timesteps=25000)
model.save("ppo_cartpole")

del model # remove to demonstrate saving and loading

model = PPO.load("ppo_cartpole")

obs = env.reset()
while True:
    action, _states = model.predict(obs)
    obs, rewards, dones, info = env.step(action)
    env.render()

Relevant log output / Error message

The Kernel crashed while executing code in the the current cell or a previous cell. Please review the code in the cell(s) to identify a possible cause of the failure. Click here for more info. View Jupyter log for further details.

Canceled future for execute_request message before replies were done

System Info

No response

Checklist

  • I have checked that there is no similar issue in the repo
  • I have read the documentation
  • I have provided a minimal working example to reproduce the bug
  • I've used the markdown code blocks for both code and stack traces.
@nguyenhongson1902 nguyenhongson1902 added the bug Something isn't working label Nov 3, 2022
@araffin araffin added the duplicate This issue or pull request already exists label Nov 3, 2022
@araffin
Copy link
Member

araffin commented Nov 3, 2022

I have checked that there is no similar issue in the repo

Duplicate of #914 #959 #1019
PR #951

EDIT: my bad, read too fast (my eyes got caught by the "M1" ^^")

@araffin araffin closed this as not planned Won't fix, can't repro, duplicate, stale Nov 3, 2022
@qgallouedec
Copy link
Collaborator

I'm not sure it is actually duplicated. Looking the code, it is supposed run on the CPU.
@nguyenhongson1902 in order for us to help you, you need to fill in the template issue completely, especially the system information section. Without this, I am not able to reproduce this bug. (I tried on my M1, everything works fine).

@qgallouedec qgallouedec reopened this Nov 3, 2022
@nguyenhongson1902
Copy link
Author

nguyenhongson1902 commented Nov 3, 2022

I'm not sure it is actually duplicated. Looking the code, it is supposed run on the CPU. @nguyenhongson1902 in order for us to help you, you need to fill in the template issue completely, especially the system information section. Without this, I am not able to reproduce this bug. (I tried on my M1, everything works fine).

@qgallouedec You're right. I've run the code on the CPU using Jupyter Notebook inside VSCode.
Here is the system information and environment I've used:
$-$ Machine: Macbook Pro M1
$-$ OS: macOS Ventura 13.0
$-$ Environment: Anaconda, Version 22.9.0
$-$ Relevant packages: pyglet 1.5.27, stable-baselines3[extra] 1.6.2, gym 0.21.0, tensorboard 2.10.1, torch 1.13.0

@qgallouedec
Copy link
Collaborator

Thank you. Can you make the code minimal? Therefore, the last line is the one that raises the issue.

I think it is a problem of gym rendering in the notebook.

@nguyenhongson1902
Copy link
Author

nguyenhongson1902 commented Nov 3, 2022

Thank you. Can you make the code minimal? Therefore, the last line is the one that raises the issue.

I think it is a problem of gym rendering in the notebook.

Okay, let me try to walk you through step by step to reproduce the error.

import gym
from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env

$\Rightarrow$ It works fine.

# Parallel environments
env = make_vec_env("CartPole-v1", n_envs=4)

$\Rightarrow$ It works ok as well.

model = PPO("MlpPolicy", env, verbose=1)
$\Rightarrow$ Boom, the error hit.

Error info:

The Kernel crashed while executing code in the the current cell or a previous cell. Please review the code in the cell(s) to identify a possible cause of the failure. Click [here](https://aka.ms/vscodeJupyterKernelCrash) for more info. View Jupyter [log](command:jupyter.viewOutput) for further details.
Canceled future for execute_request message before replies were done

Actually, when I run env = gym.make(environment_name), there's no problem at all.

@qgallouedec
Copy link
Collaborator

qgallouedec commented Nov 3, 2022

My intuition was bad. For the moment, I fail in reproducing the bug.
Can you provide the Jupyter logs?

@nguyenhongson1902
Copy link
Author

My intuition was bad. For the moment, I fail in reproducing the bug. Can you provide the Jupiter logs?

Sure, here you go.

Visual Studio Code (1.73.0, undefined, desktop)
Jupyter Extension Version: 2022.9.1202862440.
Python Extension Version: 2022.18.0.
Workspace folder /Users/sonnh/Coding/reinforcement-learning-tutorial
error 14:10:56.438: Exception while attempting zmq : [Error: No native build was found for platform=darwin arch=arm64 runtime=electron abi=106 uv=1 armv=8 libc=glibc node=16.14.2 electron=19.0.17
    loaded from: /Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/zeromq

	at load.path (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/node-gyp-build/index.js:1:3163)
	at load (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/node-gyp-build/index.js:1:648)
	at Object.<anonymous> (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/zeromq/lib/native.js:1:198)
	at u._compile (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/loader.js:4:1313)
	at Module._extensions..js (node:internal/modules/cjs/loader:1220:10)
	at Module.load (node:internal/modules/cjs/loader:1035:32)
	at Module._load (node:internal/modules/cjs/loader:876:12)
	at c._load (node:electron/js2c/asar_bundle:5:13343)
	at a._load (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:118:14563)
	at v._load (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:113:64900)
	at w._load (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:113:64268)
	at Module.require (node:internal/modules/cjs/loader:1059:19)
	at p (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/loader.js:4:699)
	at Object.<anonymous> (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/zeromq/lib/index.js:1:144)
	at u._compile (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/loader.js:4:1313)
	at Module._extensions..js (node:internal/modules/cjs/loader:1220:10)
	at Module.load (node:internal/modules/cjs/loader:1035:32)
	at Module._load (node:internal/modules/cjs/loader:876:12)
	at c._load (node:electron/js2c/asar_bundle:5:13343)
	at a._load (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:118:14563)
	at v._load (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:113:64900)
	at w._load (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:113:64268)
	at Module.require (node:internal/modules/cjs/loader:1059:19)
	at p (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/loader.js:4:699)
	at 92747 (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:17:593435)
	at __webpack_require__ (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:17:1480895)
	at l.zmqSupported (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:2:2220878)
	at l.isSupportedForLocalLaunch (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:2:2220621)
	at get isSupported [as isSupported] (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:2:2220531)
	at t.registerTypes (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:2:2235624)
	at /Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:2:1640908
	at /Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:2:1641663
	at /Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:2:1641671
	at t.activate (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:2:1641793)
	at On._callActivateOptional (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:104:17678)
	at On._callActivate (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:104:17338)
	at /Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:104:15133
	at process.processTicksAndRejections (node:internal/process/task_queues:96:5)
	at async w._activate (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:90:8304)
	at async w._waitForDepsThenActivate (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:90:8246)
	at async w._initialize (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:90:7610)]
User belongs to experiment group 'jupyterTestcf'
User belongs to experiment group 'jupyterEnhancedDataViewer'
info 14:10:56.640: Old kernelSpecs (created by Jupyter Extension) stored in directory /Users/sonnh/Library/Jupyter/kernels/__old_vscode_kernelspecs
info 14:10:56.645: LSP Notebooks experiment is disabled -- not in treatment group
info 14:10:56.648: Attempting to start a server because of preload conditions ...
info 14:10:56.656: Checking for server existence.
info 14:10:56.656: Checking for server usability.
info 14:10:56.962: Preferred Remote kernel for /Users/sonnh/Coding/reinforcement-learning-tutorial/tutorial.ipynb is undefined
info 14:10:57.022: Starting Jupyter Session startUsingPythonInterpreter, .jvsc74a57bd0f5b916051ec391ef3c1c0123575e59cad2c35863d294dd079abc5845c0e5babb./opt/anaconda3/envs/pytorch/python./opt/anaconda3/envs/pytorch/python.-m#ipykernel_launcher (Python Path: /opt/anaconda3/envs/pytorch, EnvType: Conda, EnvName: 'pytorch', Version: 3.9.7) for '/Users/sonnh/Coding/reinforcement-learning-tutorial/tutorial.ipynb' (disableUI=true)
info 14:10:57.046: Preferred Remote kernel for /Users/sonnh/Coding/reinforcement-learning-tutorial/tutorial.ipynb is undefined
info 14:10:57.046: Find preferred kernel for /Users/sonnh/Coding/reinforcement-learning-tutorial/tutorial.ipynb with metadata {"kernelspec":{"display_name":"Python 3.9.7 ('pytorch')","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.9.7"},"orig_nbformat":4,"vscode":{"interpreter":{"hash":"f5b916051ec391ef3c1c0123575e59cad2c35863d294dd079abc5845c0e5babb"}}} & preferred interpreter /usr/local/bin/python3
info 14:10:57.062: Preferred Remote kernel for /Users/sonnh/Coding/reinforcement-learning-tutorial/tutorial.ipynb is undefined
info 14:10:57.062: Preferred kernel .jvsc74a57bd0f5b916051ec391ef3c1c0123575e59cad2c35863d294dd079abc5845c0e5babb./opt/anaconda3/envs/pytorch/python./opt/anaconda3/envs/pytorch/python.-m#ipykernel_launcher is exact match or top match for non python kernels, (false, undefined, true, false)
info 14:10:57.062: PreferredConnection: .jvsc74a57bd0f5b916051ec391ef3c1c0123575e59cad2c35863d294dd079abc5845c0e5babb./opt/anaconda3/envs/pytorch/python./opt/anaconda3/envs/pytorch/python.-m#ipykernel_launcher found for NotebookDocument: /Users/sonnh/Coding/reinforcement-learning-tutorial/tutorial.ipynb
info 14:10:57.366: Registering dummy command feature
info 14:10:57.420: Registering dummy command feature
info 14:10:57.739: Got empty env vars with python /usr/local/bin/python3 in 693ms
info 14:10:57.740: Got empty env vars with python /usr/local/bin/python3 in 693ms
info 14:10:57.740: Got empty env vars with python /usr/local/bin/python3 in 694ms
info 14:10:57.740: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 695ms
info 14:10:57.740: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 694ms
info 14:10:57.740: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 694ms
info 14:10:57.741: Process Execution: > /usr/local/bin/python3 -m pip list
> /usr/local/bin/python3 -m pip list
info 14:10:57.743: Process Execution: > /usr/local/bin/python3 -c "import jupyter"
> /usr/local/bin/python3 -c "import jupyter"
info 14:10:57.743: Process Execution: > /usr/local/bin/python3 -c "import notebook"
> /usr/local/bin/python3 -c "import notebook"
info 14:10:57.795: Got env vars ourselves faster /opt/anaconda3/envs/pytorch/bin/python with env var count 42 in 774ms
info 14:10:57.800: Process Execution: > /opt/anaconda3/envs/pytorch/bin/python -m pip list
> /opt/anaconda3/envs/pytorch/bin/python -m pip list
info 14:10:57.929: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 1ms
info 14:10:57.929: Process Execution: > /usr/local/bin/python3 -m jupyter kernelspec --version
> /usr/local/bin/python3 -m jupyter kernelspec --version
info 14:10:58.297: Starting notebook server.
info 14:10:58.297: Connecting to server
info 14:10:58.297: Launching server
info 14:10:58.297: Starting Notebook
info 14:10:58.298: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 2ms
info 14:10:58.298: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 2ms
info 14:10:58.300: Generating custom default config at /var/folders/cz/t0wnr3c949j4t87gbzk2qkzc0000gn/T/5e30268b-d7a3-4d59-ab0e-14da044927f7/jupyter_notebook_config.py
info 14:10:58.300: Starting Jupyter Notebook
Starting Jupyter from /usr/local/bin/python3
info 14:10:58.301: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 1ms
info 14:10:58.301: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 0ms
info 14:10:58.302: Creating daemon pool for /usr/local/bin/python3 with env variables count 39
info 14:10:58.304: Process Execution: > /usr/local/bin/python3 -m vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.jupyter_daemon -v --ppid 44441
> /usr/local/bin/python3 -m vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.jupyter_daemon -v --ppid 44441
info 14:10:58.304: Process Execution: > /usr/local/bin/python3 -m vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.jupyter_daemon -v --ppid 44441
> /usr/local/bin/python3 -m vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.jupyter_daemon -v --ppid 44441
info 14:10:58.305: Process Execution: > /usr/local/bin/python3 -m vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.jupyter_daemon -v --ppid 44441
> /usr/local/bin/python3 -m vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.jupyter_daemon -v --ppid 44441
info 14:10:58.365: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 1ms
info 14:10:58.366: Process Execution: > /usr/local/bin/python3 (daemon) -m jupyter notebook --no-browser --notebook-dir="/Users/sonnh/Coding/reinforcement-learning-tutorial" --config=/var/folders/cz/t0wnr3c949j4t87gbzk2qkzc0000gn/T/5e30268b-d7a3-4d59-ab0e-14da044927f7/jupyter_notebook_config.py --NotebookApp.iopub_data_rate_limit=10000000000.0
> /usr/local/bin/python3 (daemon) -m jupyter notebook --no-browser --notebook-dir="/Users/sonnh/Coding/reinforcement-learning-tutorial" --config=/var/folders/cz/t0wnr3c949j4t87gbzk2qkzc0000gn/T/5e30268b-d7a3-4d59-ab0e-14da044927f7/jupyter_notebook_config.py --NotebookApp.iopub_data_rate_limit=10000000000.0
info 14:10:58.366: Waiting for Jupyter Notebook
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/traitlets/traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use '/Users/sonnh/Coding/reinforcement-learning-tutorial' instead of '"/Users/sonnh/Coding/reinforcement-learning-tutorial"' if you require traitlets >=5.
  warn(
info 14:10:58.662: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/traitlets/traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use '/Users/sonnh/Coding/reinforcement-learning-tutorial' instead of '"/Users/sonnh/Coding/reinforcement-learning-tutorial"' if you require traitlets >=5.
  warn(

[I 14:10:58.910 NotebookApp] Serving notebooks from local directory: /Users/sonnh/Coding/reinforcement-learning-tutorial
info 14:10:58.911: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/traitlets/traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use '/Users/sonnh/Coding/reinforcement-learning-tutorial' instead of '"/Users/sonnh/Coding/reinforcement-learning-tutorial"' if you require traitlets >=5.
  warn(
[I 14:10:58.910 NotebookApp] Serving notebooks from local directory: /Users/sonnh/Coding/reinforcement-learning-tutorial

[I 14:10:58.910 NotebookApp] Jupyter Notebook 6.4.11 is running at:
info 14:10:58.911: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/traitlets/traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use '/Users/sonnh/Coding/reinforcement-learning-tutorial' instead of '"/Users/sonnh/Coding/reinforcement-learning-tutorial"' if you require traitlets >=5.
  warn(
[I 14:10:58.910 NotebookApp] Serving notebooks from local directory: /Users/sonnh/Coding/reinforcement-learning-tutorial
[I 14:10:58.910 NotebookApp] Jupyter Notebook 6.4.11 is running at:

[I 14:10:58.910 NotebookApp] http://localhost:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
info 14:10:58.911: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/traitlets/traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use '/Users/sonnh/Coding/reinforcement-learning-tutorial' instead of '"/Users/sonnh/Coding/reinforcement-learning-tutorial"' if you require traitlets >=5.
  warn(
[I 14:10:58.910 NotebookApp] Serving notebooks from local directory: /Users/sonnh/Coding/reinforcement-learning-tutorial
[I 14:10:58.910 NotebookApp] Jupyter Notebook 6.4.11 is running at:
[I 14:10:58.910 NotebookApp] http://localhost:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76

info 14:10:58.912: Process Execution: > /usr/local/bin/python3 (daemon) ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/vscode_datascience_helpers/getServerInfo.py
> /usr/local/bin/python3 (daemon) ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/vscode_datascience_helpers/getServerInfo.py
info 14:10:58.912: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 0ms
[I 14:10:58.910 NotebookApp]  or http://127.0.0.1:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
info 14:10:58.914: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/traitlets/traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use '/Users/sonnh/Coding/reinforcement-learning-tutorial' instead of '"/Users/sonnh/Coding/reinforcement-learning-tutorial"' if you require traitlets >=5.
  warn(
[I 14:10:58.910 NotebookApp] Serving notebooks from local directory: /Users/sonnh/Coding/reinforcement-learning-tutorial
[I 14:10:58.910 NotebookApp] Jupyter Notebook 6.4.11 is running at:
[I 14:10:58.910 NotebookApp] http://localhost:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
[I 14:10:58.910 NotebookApp]  or http://127.0.0.1:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76

info 14:10:58.914: Process Execution: > /usr/local/bin/python3 (daemon) ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/vscode_datascience_helpers/getServerInfo.py
> /usr/local/bin/python3 (daemon) ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/vscode_datascience_helpers/getServerInfo.py
info 14:10:58.914: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 0ms
[I 14:10:58.911 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
info 14:10:58.914: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/traitlets/traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use '/Users/sonnh/Coding/reinforcement-learning-tutorial' instead of '"/Users/sonnh/Coding/reinforcement-learning-tutorial"' if you require traitlets >=5.
  warn(
[I 14:10:58.910 NotebookApp] Serving notebooks from local directory: /Users/sonnh/Coding/reinforcement-learning-tutorial
[I 14:10:58.910 NotebookApp] Jupyter Notebook 6.4.11 is running at:
[I 14:10:58.910 NotebookApp] http://localhost:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
[I 14:10:58.910 NotebookApp]  or http://127.0.0.1:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
[I 14:10:58.911 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

info 14:10:58.915: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 0ms
[C 14:10:58.914 NotebookApp] 
    
    To access the notebook, open this file in a browser:
        file:///Users/sonnh/Library/Jupyter/runtime/nbserver-44526-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
     or http://127.0.0.1:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
info 14:10:58.915: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/traitlets/traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use '/Users/sonnh/Coding/reinforcement-learning-tutorial' instead of '"/Users/sonnh/Coding/reinforcement-learning-tutorial"' if you require traitlets >=5.
  warn(
[I 14:10:58.910 NotebookApp] Serving notebooks from local directory: /Users/sonnh/Coding/reinforcement-learning-tutorial
[I 14:10:58.910 NotebookApp] Jupyter Notebook 6.4.11 is running at:
[I 14:10:58.910 NotebookApp] http://localhost:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
[I 14:10:58.910 NotebookApp]  or http://127.0.0.1:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
[I 14:10:58.911 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 14:10:58.914 NotebookApp] 
    
    To access the notebook, open this file in a browser:
        file:///Users/sonnh/Library/Jupyter/runtime/nbserver-44526-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
     or http://127.0.0.1:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76

info 14:10:58.916: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 1ms
info 14:10:58.996: Got empty env vars with python /usr/local/bin/python3 in 1068ms
info 14:10:59.023: Got empty env vars with python /usr/local/bin/python3 in 727ms
info 14:10:59.023: Got empty env vars with python /usr/local/bin/python3 in 727ms
info 14:10:59.023: Got empty env vars with python /usr/local/bin/python3 in 723ms
info 14:10:59.029: Got empty env vars with python /usr/local/bin/python3 in 728ms
info 14:10:59.030: Got empty env vars with python /usr/local/bin/python3 in 666ms
info 14:10:59.030: Got empty env vars with python /usr/local/bin/python3 in 118ms
info 14:10:59.030: Got empty env vars with python /usr/local/bin/python3 in 116ms
info 14:10:59.031: Got empty env vars with python /usr/local/bin/python3 in 116ms
info 14:10:59.033: Got empty env vars with python /usr/local/bin/python3 in 118ms
info 14:10:59.070: Connecting to process server
info 14:10:59.070: Connecting server kernel http://localhost:8888/
info 14:10:59.071: Creating server with url : http://localhost:8888/
info 14:10:59.074: Connection complete server
info 14:10:59.075: Server started.
info 14:10:59.116: Process Execution: > /usr/local/bin/python3 (daemon) ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/vscode_datascience_helpers/getServerInfo.py
> /usr/local/bin/python3 (daemon) ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/vscode_datascience_helpers/getServerInfo.py
info 14:10:59.126: Process Execution: > /usr/local/bin/python3 (daemon) ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/vscode_datascience_helpers/getServerInfo.py
> /usr/local/bin/python3 (daemon) ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/vscode_datascience_helpers/getServerInfo.py
[I 14:10:59.189 NotebookApp] Creating new notebook in 
info 14:10:59.203: Got env vars with python /opt/anaconda3/envs/pytorch/bin/python, with env var count 43 and custom env var count 0 in 2182ms
info 14:10:59.206: installMissingDependencies /opt/anaconda3/envs/pytorch/bin/python, ui.disabled=true for resource '/Users/sonnh/Coding/reinforcement-learning-tutorial/tutorial.ipynb'
info 14:10:59.206: Got env vars with python /opt/anaconda3/envs/pytorch/bin/python, with env var count 43 and custom env var count 0 in 0ms
info 14:10:59.206: Got env vars ourselves faster /opt/anaconda3/envs/pytorch/bin/python with env var count 42 in 0ms
info 14:10:59.210: Process Execution: > /opt/anaconda3/envs/pytorch/bin/python -c "import ipykernel"
> /opt/anaconda3/envs/pytorch/bin/python -c "import ipykernel"
info 14:10:59.479: Spec argv[0] updated from '/opt/anaconda3/envs/pytorch/bin/python' to '/opt/anaconda3/envs/pytorch/bin/python'
info 14:10:59.480: Got env vars ourselves faster /opt/anaconda3/envs/pytorch/bin/python with env var count 42 in 1ms
[I 14:10:59.524 NotebookApp] Kernel started: 16fc0738-7ed0-45f2-9d6f-cf9016fde4ea, name: python397jvsc74a57bd0f5b916051ec391ef3c1c0123575e59cad2c35863d294dd079abc5845c0e5babb
[W 14:10:59.538 NotebookApp] delete /tutorial-jvsc-ca0b217c-08e4-4c22-b10d-db9ac2d6ba9d4bd2e89d-8a8c-4215-a5e1-2d6b68104d3b.ipynb
info 14:11:00.394: Got env vars with python /opt/anaconda3/envs/pytorch/bin/python, with env var count 43 and custom env var count 0 in 914ms
info 14:11:00.438: Started session for kernel startUsingPythonInterpreter:.jvsc74a57bd0f5b916051ec391ef3c1c0123575e59cad2c35863d294dd079abc5845c0e5babb./opt/anaconda3/envs/pytorch/python./opt/anaconda3/envs/pytorch/python.-m#ipykernel_launcher
info 14:11:00.439: UpdateWorkingDirectoryAndPath in Kernel
info 14:11:00.949: Got env vars with python /opt/anaconda3/envs/pytorch/bin/python, with env var count 43 and custom env var count 0 in 0ms
info 14:11:00.949: Got env vars ourselves faster /opt/anaconda3/envs/pytorch/bin/python with env var count 42 in 1ms
info 14:11:00.952: Process Execution: > /opt/anaconda3/envs/pytorch/bin/python ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/printJupyterDataDir.py
> /opt/anaconda3/envs/pytorch/bin/python ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/printJupyterDataDir.py
[I 14:14:11.510 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
error 14:14:11.516: Error in waiting for cell to complete [Error: Canceled future for execute_request message before replies were done
	at a.KernelShellFutureHandler.dispose (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/@jupyterlab/services.js:2:32353)
	at /Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/@jupyterlab/services.js:2:26572
	at Map.forEach (<anonymous>)
	at y._clearKernelState (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/@jupyterlab/services.js:2:26557)
	at /Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/@jupyterlab/services.js:2:29000
	at process.processTicksAndRejections (node:internal/process/task_queues:96:5)]
warn 14:14:11.518: Cell completed with errors {
  message: 'Canceled future for execute_request message before replies were done'
}
info 14:14:11.519: Cancel all remaining cells true || Error || undefined
[I 14:14:11.517 NotebookApp] Starting buffering for 16fc0738-7ed0-45f2-9d6f-cf9016fde4ea:49b6661c-e9f4-4511-9f29-0b82883801d7
[I 14:14:11.537 NotebookApp] Restoring connection for 16fc0738-7ed0-45f2-9d6f-cf9016fde4ea:49b6661c-e9f4-4511-9f29-0b82883801d7

@qgallouedec
Copy link
Collaborator

Do you agree that the minimal code would be

from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env

env = make_vec_env("CartPole-v1", n_envs=4)
PPO("MlpPolicy", env)

Does it work without multiple envs?:

from stable_baselines3 import PPO

PPO("MlpPolicy", "CartPole-v1")

@nguyenhongson1902
Copy link
Author

nguyenhongson1902 commented Nov 3, 2022

Do you agree that the minimal code would be

from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env

env = make_vec_env("CartPole-v1", n_envs=4)
PPO("MlpPolicy", env)

Does it work without multiple envs?:

from stable_baselines3 import PPO

PPO("MlpPolicy", "CartPole-v1")

Without multiple envs, it still has the same error. For example:

import gym
from stable_baselines3 import PPO
from stable_baselines3.common.vec_env import DummyVecEnv

$\Rightarrow$ Works fine.

environment_name = 'CartPole-v1'
env = gym.make(environment_name)
env = DummyVecEnv([lambda: env])

$\Rightarrow$ No problems

model = PPO('MlpPolicy', env, verbose=0)
$\Rightarrow$ It got the same error as with multiple envs.
By the way, I'm using Python 3.9.7. I've included the illustration below.
image

@qgallouedec
Copy link
Collaborator

I have the same configuration as you and I do not encounter any errors.

I've noticed several times that the VSCode jupyter extension is buggy and sometimes behaving a bit strangely. I remember solving one of them by simply reinstalling the virtual environment completely and rebooting my laptop. I think it's really worth a try.

@qgallouedec
Copy link
Collaborator

Possibly memory-related: microsoft/vscode-jupyter#9741 (comment). It would explain why the reboot worked for me.

@araffin araffin removed the duplicate This issue or pull request already exists label Nov 3, 2022
@nguyenhongson1902
Copy link
Author

@qgallouedec I've tried rebooting the laptop and reinstalling the Anaconda virtual environment but the error remains the same :(

@qgallouedec
Copy link
Collaborator

Does the code fail when executed in a standard python file?

@nguyenhongson1902
Copy link
Author

Does the code fail when executed in a standard python file?

When running with normal python file (it's my test.py file in this case), terminal shows the error named zsh: segmentation fault python test.py

@nguyenhongson1902
Copy link
Author

nguyenhongson1902 commented Nov 4, 2022

I've found a package to help me find out the bug. Just import it and add faulthandler.enable() in my test.py file.

import gym
from stable_baselines3 import PPO, A2C, DQN
from stable_baselines3.common.vec_env import DummyVecEnv
import faulthandler

faulthandler.enable()

if __name__ == '__main__':
    environment_name = 'CartPole-v1'
    env = gym.make(environment_name)
    env = DummyVecEnv([lambda: env])

    model = PPO('MlpPolicy', env, verbose=0)
    # model = A2C('MlpPolicy', env, verbose=0)
    # model = DQN('MlpPolicy', env, verbose=0)

Having run the script with each of those models (PPO, A2C, DQN), I got the result:
With model PPO, the error is

Fatal Python error: Segmentation fault

Thread 0x0000000204bb92c0 (most recent call first):
  File "/opt/anaconda3/envs/rl/lib/python3.9/site-packages/torch/nn/init.py", line 484 in orthogonal_
  File "/opt/anaconda3/envs/rl/lib/python3.9/site-packages/stable_baselines3/common/policies.py", line zsh: segmentation fault  python test.py

With model A2C, the error is

Fatal Python error: Segmentation fault

Thread 0x00000002028832c0 (most recent call first):
Fatal Python error:   File Segmentation fault"

zsh: segmentation fault  python test.py

And with model DQN, it works fine.
It seems like there's a problem with PPO and A2C, I wonder. Do you have any idea to debug this segmentation error?

@qgallouedec
Copy link
Collaborator

qgallouedec commented Nov 4, 2022

Could you run this:

import torch.nn as nn
linear = nn.Linear(2, 2)
nn.init.orthogonal_(linear.weight)

@nguyenhongson1902
Copy link
Author

Could you run this:

import torch.nn as nn
linear = nn.Linear(2, 2)
nn.init.orthogonal_(linear.weight)

Yepp, no problems with that.
image

@qgallouedec
Copy link
Collaborator

Do you still encounter the error? What Python version do you use?

@vamsig7
Copy link

vamsig7 commented Dec 27, 2022

I can still notice this behaviour in sb3 1.6.2 and python3.9 .Exact same bug "Fatal Python error: Segmentation fault" in case of PPO but DQN is working fine. Please let me know for any additional log

@qgallouedec
Copy link
Collaborator

qgallouedec commented Dec 27, 2022

Have a MacBook Pro M1, I can't reproduce the error. The following works as expected:

from stable_baselines3 import PPO

PPO("MlpPolicy", "CartPole-v1")
  • OS: macOS-13.0.1-arm64-arm-64bit Darwin Kernel Version 22.1.0: Sun Oct 9 20:14:30 PDT 2022; root:xnu-8792.41.9~2/RELEASE_ARM64_T8103
  • Python: 3.9.16
  • Stable-Baselines3: 1.6.2
  • PyTorch: 1.13.1
  • GPU Enabled: False
  • Numpy: 1.24.1
  • Gym: 0.21.0

If anyone finds a code/config to reproduce this error, please post it here.

@vamsig7
Copy link

vamsig7 commented Dec 28, 2022

@qgallouedec the exact same two lines i tried running in my local vs code . I hope this has nothing do with which i de i am using.
OS: macOS 12.6.1 Darwin 21.6.0
Python: 3.9.13
stable-baselines3 :git+https://github.com/DLR-RM/stable-baselines3@e78ba6ffa40e76c06fce1b8047070c991fa5abef(same
crash seen with 1.6.2)
PyTorch: 1.13.1
GPU Enabled: False
Gym: 0.21.0
Added screenshot of vscode notebook and jupyter notebook(Anaconda navigator)

Screenshot 2022-12-28 at 6 34 19 PM

Screenshot 2022-12-28 at 6 41 40 PM

@qgallouedec
Copy link
Collaborator

What are the Jupyter logs?

@vamsig7
Copy link

vamsig7 commented Dec 28, 2022

Sorry for late response. I resolved my issue just now by reinstalling numpy which is causing the problem.
My issue is same as mentioned here microsoft/vscode-jupyter#9741
Tried to uses same package versions and mentioned in your prior comment and reinstalling numpy fixed issue
Thank you

@qgallouedec
Copy link
Collaborator

Late = 18 minutes ? 😆

Great to hear! I'm closing the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants