'>' not supported between instances of 'float' and 'NoneType'

[MoMe36]

Hello,

When running PPO various environments, I get the following error message:
'>' not supported between instances of 'float' and 'NoneType'. I narrowed the error to the point where the Visdom server is initialized but I can't figure out why this happens.

Also, I used the following command to launch the training:
python main.py --env-name "mreacher-v0" --algo ppo --use-gae --lr 2.5e-4 --clip-param 0.1 --value-loss-coef 1 --num-processes 8 --num-steps 128 --num-mini-batch 4 --vis-interval 1 --log-interval 1

Thanks !

[MoMe36]

The environments is a custom one, but I get the same error when launching CartPole.

[ikostrikov]

Which version of python are you using?

[MoMe36]

I'm on 3.6.5

[MoMe36]

While the error occurs in a recurrent manner, the algorithm still functions.

'>' not supported between instances of 'float' and 'NoneType'
Updates 6029, num timesteps 6174720, FPS 2272 
 Last 10 training episodes: mean/median reward 1.0/1.0, min/max reward 1.0/1.0

Updates 6030, num timesteps 6175744, FPS 2273 
 Last 10 training episodes: mean/median reward 1.0/1.0, min/max reward 1.0/1.0

Updates 6031, num timesteps 6176768, FPS 2273 
 Last 10 training episodes: mean/median reward 1.0/1.0, min/max reward 1.0/1.0

Updates 6032, num timesteps 6177792, FPS 2273 
 Last 10 training episodes: mean/median reward 1.0/1.0, min/max reward 1.0/1.0

But I thought best to find where lies the error.

[ikostrikov]

Do you use the most recent PyTorch/baselines/gym?

I don't get this error for cartpole

[MoMe36]

I think I do. Could it be something with Visdom maybe ?

[ikostrikov]

It doesn't look like a vizdom problem. Can you provide more information?

Is your env on github?

[MoMe36]

I tried this debug:

def main():
    torch.set_num_threads(1)
    device = torch.device("cuda:0" if args.cuda else "cpu")

    input('Entering viz')
    if args.vis:
        from visdom import Visdom
        print(args.port)
        viz = Visdom(port=args.port)
        print(viz)
        win = None
    input('Exit')

Here's the output:

Entering viz
8097
'>' not supported between instances of 'float' and 'NoneType'
'>' not supported between instances of 'float' and 'NoneType'
'>' not supported between instances of 'float' and 'NoneType'
Visdom python client failed to establish socket to get messages from the server. This feature is optional and can be disabled by initializing Visdom with `use_incoming_socket=False`, which will prevent waiting for this request to timeout.
<visdom.Visdom object at 0x7fda77942278>
Exit

It is indeed: https://github.com/Mehd6384/RL/tree/master/GymReacher

[ikostrikov]

Did you launch visdom in the background? python -m visdom.server

For me it works for version '0.1.8.5'.

[MoMe36]

I did and I'm on the same version here which is strange.

Overall, the algorithm worked and returned a functional policy, I was just surprised by this message

[ikostrikov]

Ok. Then let's keep the issue open. Also could you create a similar issue for visdom repository? It looks like the problem is in visdom itself.

[ikostrikov]

The issue is already there
fossasia/visdom#467