len(tmp_observations) < 2 on PPO raise ValueError: The parameter probs has invalid values

[lorenzosteccanella]

It seems that your code produce error if the len of your trajectory < 2 ( len(tmp_observations) < 2). I tested this on PPO I don't know if this happens with all algorithms.

The error:

ValueError: The parameter probs has invalid values

[iffiX]

Could you please provide a minimal reproducible example and full stack trace, other log outputs, etc?

[lorenzosteccanella]

Here it is a minimal example to reproduce the error: testPPO.txt.

Just by setting max_steps = 0 in the provided example, in this case you will have len( tmp_observations ) == 1 and will raise the following error:

Traceback (most recent call last):
File ".../PPO issues.py", line 94, in
ppo.update()
File ".../lib/python3.8/site-packages/machin/frame/algorithms/ppo.py", line 146, in update
_, new_action_log_prob, * = self._eval_act(state, action)
File ".../lib/python3.8/site-packages/machin/frame/algorithms/a2c.py", line 258, in _eval_act
return safe_call(self.actor, state, action)
File ".../pytorch_env/lib/python3.8/site-packages/machin/frame/algorithms/utils.py", line 156, in safe_call
result = getattr(model, method)(**args_dict)
File ".../pytorch_env/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File ".../PPO issues.py", line 31, in forward
dist = Categorical(probs=probs)
File ".../lib/python3.8/site-packages/torch/distributions/categorical.py", line 64, in init
super(Categorical, self).init(batch_shape, validate_args=validate_args)
File ".../lib/python3.8/site-packages/torch/distributions/distribution.py", line 53, in init
raise ValueError("The parameter {} has invalid values".format(param))
ValueError: The parameter probs has invalid values

Process finished with exit code 1

[iffiX]

Sorry for the late comment, that's definitely a boundary case that needed to be fixed. I'm working on a paper recently and my computer also got hacked so this problem might hang around here a little longer, I'm really sorry for the inconvenience.