GPU Enabled: False on Chip Apple M1 Pro

[micheljperez]

Hi there,

Well, I'm using rl-baseline on a:

System Info
MacBook Pro Chip Apple M1Pro
OS: macOS-12.4-arm64-arm-64bit Darwin Kernel Version 21.5.0: Tue Apr 26 21:08:37 PDT 2022; root:xnu-8020.121.3~4/RELEASE_ARM64_T6000
Python: 3.8.9
Stable-Baselines3: 1.5.1a8
PyTorch: 1.13.0.dev20220629
Numpy: 1.23.0
Gym: 0.21.0

I'm using the lon huggingface RL course,unit-3.

Here's the command that I'm using in order to launch the training:

python train.py --algo dqn --env SpaceInvadersNoFrameskip-v4 -f logs/

I go to the logs and I got:

OS: macOS-12.4-arm64-arm-64bit Darwin Kernel Version 21.5.0: Tue Apr 26 21:08:37 PDT 2022; root:xnu-8020.121.3~4/RELEASE_ARM64_T6000
Python: 3.8.9
Stable-Baselines3: 1.5.1a8
PyTorch: 1.13.0.dev20220629
GPU Enabled: False
Numpy: 1.23.0
Gym: 0.21.0

So, GPU Enabled: False. I installed the last version of pytorch using:

pip3 install -U --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu

I checked the installation status:

Python 3.8.9 (default, Apr 13 2022, 08:48:06)
[Clang 13.1.6 (clang-1316.0.21.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
import torch

device = "mps" if torch.has_mps else "cpu"
print(f'Using device: {device}')
Using device: mps

I tested this too:
import torch

torch.has_mps
True

So, I don't understand why when I launch my training the GPU Enabled. continue saying False.

I'm looking forward to speed up my training using the GPU on my Apple M1 Pro. Could you please help me ?

Thanks !

[araffin]

probably related to DLR-RM/stable-baselines3#914
you should probably force the device until we add it as default to sb3

[micheljperez]

Oh thanks. I forced the device using:

python train.py --algo dqn --device mps --env SpaceInvadersNoFrameskip-v4 -f logs/

========== SpaceInvadersNoFrameskip-v4 ==========
Seed: 3369038554
Default hyperparameters for environment (ones being tuned will be overridden):
OrderedDict([('batch_size', 32),
('buffer_size', 100000),
('env_wrapper',
['stable_baselines3.common.atari_wrappers.AtariWrapper']),
('exploration_final_eps', 0.01),
('exploration_fraction', 0.14823),
('frame_stack', 4),
('gradient_steps', 1),
('learning_rate', 0.62701),
('learning_starts', 100000),
('n_timesteps', 10000000),
('optimize_memory_usage', True),
('policy', 'CnnPolicy'),
('target_update_interval', 1000),
('train_freq', 4)])
Using 1 environments
Creating test environment
A.L.E: Arcade Learning Environment (version 0.7.4+069f8bd)
[Powered by Stella]
Stacking 4 frames
Wrapping the env in a VecTransposeImage.
Stacking 4 frames
Wrapping the env in a VecTransposeImage.
Using mps device
Log path: logs//dqn/SpaceInvadersNoFrameskip-v4_20

At launch log I see: Using mps device however I continue see in the logs:

GPU Enabled:False

unzip SpaceInvadersNoFrameskip-v4.zip
Archive: SpaceInvadersNoFrameskip-v4.zip
extracting: data
extracting: pytorch_variables.pth
extracting: policy.pth
extracting: policy.optimizer.pth
extracting: _stable_baselines3_version
extracting: system_info.txt
(huggingenv) U@local:/Users/U/Documents/hugginface/RL/unit3/rl-baselines3-zoo/logs/dqn/SpaceInvadersNoFrameskip-v4_20 git:(master*) $ cat system_info.txt
OS: macOS-12.4-arm64-arm-64bit Darwin Kernel Version 21.5.0: Tue Apr 26 21:08:37 PDT 2022; root:xnu-8020.121.3~4/RELEASE_ARM64_T6000
Python: 3.8.9
Stable-Baselines3: 1.5.1a8
PyTorch: 1.13.0.dev20220629
GPU Enabled: False
Numpy: 1.23.0
Gym: 0.21.0

[qgallouedec]

Hi,

Your training uses GPU acceleration. Actually, GPU Enabled: False means CUDA is not available. Since we can now use mps as a device, I agree that this is misleading.

To solve this, we need to update sb3.common.utils.get_system_info to output something like GPU Acceleration: CUDA/MPS/None instead of GPU Enabled: True/False. However, the version of PyTorch that supports MPS (1.13) is not yet stable. I suggest to wait for the stable version to update sb3 accordingly.

Next time, please use the markdown code blocks for both code and stack traces.

[micheljperez]

thanks a lot ! and got it for markdown code block.

[araffin]

Closing in favor of DLR-RM/stable-baselines3#914