Default result only got 84 on MSRA

[Jarrome]

Great works! I've read it thoroughly and can easily run your code.

One issue is when I directly use the train-msr and finish train, the final acc is 84 and 83 for two repeat. I think they are with 16 frames. My repeats get lower scoring comparing to the paper.

I think it might because there are some parameter change in the script.
Would you be convenient to check out?

[hehefan]

Hi @Jarrome,

This is a bit weird. People usually get slightly better accuracies than the paper. Could you please send the log as well as the environment to me (crane.h.fan@gmail.com)? It could be much better if you could also provide the log with 24 frames. I would like to repeat and check the problem.

Many thanks.

Best regards.

[Jarrome]

Hi, @hehefan,

Thanks for your quick reply. I have emailed you the repo and piplist.
I will now try the 24 frames, and send you latter.

Best regards.

[vivonicole]

Hi @hehefan,

I also got a slightly lower accuracy 89.2 with 16 frames.

[vivonicole]

Hi @hehefan

I achieve a 92.68 with 24 frames. Great work.

[hehefan]

Hi @vivonicole,

The accuracy of 89.2 with 16 frames is reasonable. Tuning the radius might improve the performance. Thank you.

Best regards.

[Jarrome]

I close this issue as the problem might be from the compiling of sampling&grouping layer. Not from the authors' work.
It can be reopen, when there is any other find on the clustering step.

[sheshap]

Hi @hehefan

I achieve a 92.68 with 24 frames. Great work.

Can you please confirm if this result is for PSTNet or PSTNet++?

I got 90.24 for PSTNet with 24 frames.
Can you please share the logs?

[vivonicole]

Hi @hehefan
I achieve a 92.68 with 24 frames. Great work.

Can you please confirm if this result is for PSTNet or PSTNet++?

I got 90.24 for PSTNet with 24 frames. Can you please share the logs?

I'm sure it was PSTNet but I did not save the log..

[sheshap]

@hehefan

For 24 frames with a batch size of 24, using 4x2080Ti, I got 93.03% accuracy.

What could be the reason for so much fluctuation in accuracy despite setting a random seed for numpy and PyTorch?

[hehefan]

Hi @sheshap,

The fluctuation may be caused by the small scale of the MSR-Action3D dataset and the data augmentation. The accuracy is usually around 91.9±1%.

[sheshap]

Thank you for the reply.

In the case of MSR-Action3D, data augmentation is only scaling, right?

As per the datasets/msr.py file below lines:
if self.train:
# scale the points
scales = np.random.uniform(0.9, 1.1, size=3)
clip = clip * scales

Thanks

[hehefan]

Hi @sheshap,

You are right. But note that, because each frame has more than 2048 points. Sampling points is also a random source.

Best regards.