Question about PI05 post-training checkpoints and reproducibility

[whydahuzi]

Hi, thanks for releasing EventVLA and the RoboTwin-MeM codebase.

I would like to ask about the reproducibility of the PI05 post-training results.

Following the issues and suggested fixes discussed in Issue #3, I updated the relevant configurations and evaluation settings. I then performed PI05 PyTorch post-training from the released setup.

My current training setup is:

Policy: PI05 PyTorch
Post-training steps: 160k
Batch size: 32
Configuration: updated according to the fixes discussed in Issue #3

However, the resulting evaluation performance is still significantly lower than the results reported in the paper.

Since the released checkpoint and evaluation pipeline have already shown some reproducibility issues, I am wondering whether there are any additional training details or configurations that are not included in the released code/config files.

Questions
Is the PI05 checkpoint used to obtain the results reported in the paper trained with the same post-training procedure and configuration as the currently released code?
Are there any important post-training details that are not currently documented, such as learning rate, number of epochs/steps, data preprocessing, checkpoint initialization, or checkpoint selection?
Would it be possible to release the complete PI05 post-training checkpoints, or at least checkpoints from different stages of post-training?

In particular, releasing intermediate/final checkpoints would be very helpful for determining whether the performance gap comes from the post-training process, checkpoint selection, or the evaluation pipeline, and would make the reported results much easier for the community to reproduce.

Thank you for your work and for sharing the code. I would greatly appreciate any clarification regarding the PI05 post-training procedure and checkpoints.

[whydahuzi]

One additional question I am particularly curious about:

For the base model used for PI05 post-training, did you start from the official PI05 pretrained checkpoint, or from a checkpoint that had already been trained/fine-tuned using the task decomposition/subtask training procedure in your project?

I am asking because the choice of initialization checkpoint could potentially have a significant impact on the final post-training performance. It would be helpful if you could clarify which checkpoint was used in the experiments reported in the paper.