Feature request - option for not testing interaction in pairwise ttests

[samuelstjean]

Hello,

Might be a bit specific, but I was wondering if that was an idea. I'm currently using it as a cheap way to posthoc test everything and am not super interested in interaction effects as for some cases it would make no sense. However, it seems that doing these tests is

1a. memory hungry, because it probably checks for all interactions
1b. could maybe reuse / make better use of temporary variables, it currently blows up my 16 go of ram the instant it goes into the interaction effects for a dataset with around 300 000 lines (and 6 columns, for each group I'm checking).

[raphaelvallat]

Hi @samuelstjean ! Thanks for this!

1. A quick workaround is to call separately each factor that you are interested in, e.g.:

pg.pairwise_ttests(..., between="fac1")

followed by

pg.pairwise_ttests(..., between="fac2")

instead of pg.pairwise_ttests(..., between=["fac1", "fac2"])

I'll try to add an option to test only the interaction or main effect in future release.

2. I agree with you on this point; the current implementation is designed for flexibility, but not really for speed / memory efficiency (in my field of research we rarely apply T-tests on more than 1000 rows so memory has never been an issue). I'll try to look into that in future releases but please feel free to have a look at the code and suggest any ideas to improve this.

Thanks!
Raphael

[samuelstjean]

Ah yes, kind of did that, I cheated by taking parts of my panda frame (reusing the same memory array every time), calling the whole thing on it, and finally re-computing correct pvals for the final result, therefore avoiding the interaction effects.

I guess it's a bit too specific, but well for general purpose tests (and other fields) it may be helpful indeed. I'm afraid I'm a bit busy to have a shot at it myself before at least november though, thesis writing and everything.

[raphaelvallat]

• Added interaction argument to pairwise_ttests in cef2a0f
• Currently working on improving speed and memory

[raphaelvallat]

Minor improvements in speed and memory:

• 2d11acf
• 39975fb
• 36070fc

These changes are available on the develop branch and will be released in Pingouin 0.2.9 (sometimes in September).
Closing the issue for now but please feel free to reopen.
Thanks
Raphael