Advices for good training

[nndei]

Dear @bango123,

I am trying to train a scene for a simplified, yet more complex than the one of this work, task. I am struggling to obtain good results, if any to be honest...

I am wondering if you had the same issues. I am working with baselines / stable-baselines, without the behavioral cloning loss you used.

Did you do some practical work and found that without this BCL nothing converged?
Can you share some important things you learnt during the process?
May you share the training code you used so that I may try to replicate it? How long did the training take?

Best regards, Neri

[bango123]

Did you do some practical work and found that without this BCL nothing converged?
For grasping tasks (even with other robots like the baxter) I was unsuccessful. I think this is due to the large state space that needs to be explored.

Can you share some important things you learnt during the process?

1. Although obvious, ensure the environment is solvable.
2. The actual grasping code is very hard for a policy to learn/explore to. If possible, it would help to separate grasping from the other task trying to be accomplished.
3. I know orientation is not in these environments, but you may interested in them. Special care needs to be taken for neural networks to handle the quaternion constraint.

May you share the training code you used so that I may try to replicate it? How long did the training take?
I believe in the other comment I shared how to generate the examples (#15 (comment)). Outside of this I really just used baselines code and hyper parameters.

[nndei]

Thank you for the tips.
What do you mean exactly with solvable?
For now, I am sticking to the reach task, but I will consider your advice when trying the pick task.
Could you advise some resource to study this issue on the orientation?

[bango123]

By solvable I mean, you could craft a sequence of actions (not necessarily optimal) that actually accomplishes your goal. Also this will ensure that the task is "robust" to small inaccuracies in terms of grasping at different angles.

For the orientation, I do not have much experience. However, here is a paper I previously referenced that ensures the output of the neural network is normalized https://dl.acm.org/doi/pdf/10.1145/3123266.3123359
In addition, my colleague Ahmed developed neural networks for motion planning and has an example of planning in SE(3) where a quaternion is outputted https://arxiv.org/pdf/1907.06013