Q: reduced #process on GPU?

[mw66]

Hi,

Previously (a few days ago) in training I saw (args.num_processes + 1) processes on GPU using nvidia-smi, and GPU utilization high 80~90%.

With the latest code, I saw only one process on GPU, and sometimes GPU utilization only a few percent.

I just wondering what changed? and it's intended.

Thanks.

[ikostrikov]

There was a bug in PyTorch that initialized cuda in every separate process (that wasn't needed).

Apparently, it was fixed recently.

Does it change performance in any way?

[mw66]

Oh, I found I made a mistake: the single GPU process scenario is not an Atari game. (It's a game env that I created with no graphics/images). And I rerun the Atari game, it still use N+1 GPU processes, and performance is the same.

So I guess it's the gym game env that create a GPU process for each game in training; because Atari game are video (image) games? thus N+1 (main).

Comparing the two scenario, my next question is: so all the CNN network (CNNPolicy) are running on that single main CPU process, instead of each game's env GPU process?

Then I wonder if it's possible to distribute CNN computation on each of the GPU process and get some performance gain?

[ikostrikov]

Each process just initializes (but doesn't use CUDA), it's a bug (feature?) in PyTorch.

At the moment, all environments run in CPU threads. Then, the observations get concatenated and sent to GPU.

[mw66]

gym doe not depend on PyTorch, so it's because of WrapPyTorch, is this absolute necessary?

[ikostrikov]

Yes. It's because of data format.