Effect size and Power Calculations

[gillianweir]

Hi Todd,

Hope you are well!
We have a PhD student here who is writing his dissertation proposal is thinking about using SPM in his analyses. Before he proposes it I just wanted to see how far off you guys are from having effect size calculations in the code?
Also do you have a power calculation method that you currently use/are developing for SPM? He is doing OSim and will be recruiting 10 male participants to compare different walking strategies (DVs: GRFs, lower limb kinematics). He also wanted to compares this to 3 chimpanzees but going off the feed with Johannes would you recommend to not do this analysis with SPM?

Many thanks in advance.

Gill

[0todd0000]

Hi Gill!

Before he proposes it I just wanted to see how far off you guys are from having effect size calculations in the code?

The code should be available later this year or early in 2018.

Also do you have a power calculation method that you currently use/are developing for SPM?

Yes, there is a Python package available here:
www.spm1d.org/power1d/
and a MATLAB version will be available by early 2018.
The package is described in the following article:
https://peerj.com/articles/cs-125/

...would you recommend to not do this analysis with SPM?

It depends what the goal is. If the goal is population-level inference then three individuals is likely too few. But that limitation is not specific to SPM. I don't know of any technique that can make solid population-level inferences from just three individuals. If you send more details like the goal of the study and the type of test he intends to use I'll try to answer a bit more clearly...

Todd

[gillianweir]

Hi Todd,

Thanks so much for that. Very excited for those new packages!

I certainly agree with that last point, however due to limitations with the lab that we had set up to collect chimp data n=3 is the current sample. The goal is to compare different walking postures (normal and different crouched postures) in 10 human participants. The second part is to compare those 4 postures in humans (n=10) to chimpanzee walking postures (n=3). I imagine the 10 human subjects comparing the 4 postures will be fine but humans vs chimps must use a descriptive approach.

Thanks again,

Gill

[0todd0000]

Hi Gill,
I agree that a descriptive approach might be best with a sample size of three. You can try making statistical inferences as well, but the critical threshold will be quite high due to small sample size, so the effects would need to be quite large to yield significance. If inference doesn't yield significance it may actually be a Type II error, so to avoid that it may indeed be best to conduct power analysis before conducting hypothesis testing to determine how powerfully the current sample sizes can detect a particular effect.
Todd

[gillianweir]

Todd many thanks again for your always valuable insight.

[burniel]

Hi, I have just been reading through the thread and was wondering if the effect size calculations code been realized yet?

[0todd0000]

Not yet unfortunately. Effect size calculations for simple experiments are complete but they're not yet ready for release. General effect size calculations for arbitrary experiments are still in development. If you have a particular design in mind (e.g. two-sample t test) I'd be happy to copy some code here.

As another option, the power1d package (www.spm1d.org/power1d/) cited above can be used to systematically explore arbitrary effects in arbitrary experiments, and in particular to calculate the probabilities of observing specific true effects.

[burniel]

Hi Todd, I am using paired t-tests (for joint moments and angles) and Hoteling’s paired T2 tests (for crank forces). So if you have matlab code available for those cases that would be great. Thanks Louise From: Todd Pataky [mailto:notifications@github.com] Sent: 22 March 2018 11:58 To: 0todd0000/spm1dmatlab Cc: Louise Burnie; Comment Subject: Re: [0todd0000/spm1dmatlab] Effect size and Power Calculations (#54) Not yet unfortunately. Effect size calculations for simple experiments are complete but they're not yet ready for release. General effect size calculations for arbitrary experiments are still in development. If you have a particular design in mind (e.g. two-sample t test) I'd be happy to copy some code here. As another option, the power1d package (www.spm1d.org/power1d/<http://www.spm1d.org/power1d/>) cited above can be used to systematically explore arbitrary effects in arbitrary experiments, and in particular to calculate the probabilities of observing specific true effects. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#54 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/Ad5MH4lLB71Ri4u2JuMnchBQj_6w7zh3ks5tg5G_gaJpZM4O7wSp>. ******************************************************************************************************************************************************** This message and any attachment has been sent in confidence. If you have received this message in error, please accept our apologies; we would ask that you notify the sender immediately and delete the message. If you are not the intended recipient please do not use, disclose, copy, print or rely on the message or its contents. ******************************************************************************************************************************************************** This message may contain personal views and opinions that are not necessarily those of the EIS unless specifically stated. We take our virus suppression responsibilities seriously, and this message confirms that this communication has been scanned for known viruses before transmission. However, as the Internet is an insecure medium we cannot be held responsible for any modifications that may be made during transmission, and we therefore suggest that you scan all incoming mail for viruses. Please note that we may use information about you contained in this message other than for monitoring, but only in accordance with the purposes for which we are registered under the Data Protection Act. Please note that we reserve the right to monitor incoming and outgoing communications, and so you should not expect them to be private in nature. ******************************************************************************************************************************************************** The EIS is a nationwide network of regional support teams offering world class services to our elite athletes. For up to date information on the EIS, please visit our website www.eis2win.co.uk. The English Institute of Sport is a company registered in the UK, whose office is The Manchester Institute of Health and Performance, 299 Alan Turing Way, Manchester, M11 3BS. Company registration number 4420052. ********************************************************************************************************************************************************