Unable to run fisher test due to LDSTP being too small

[thh32]

Currently getting the following issue when running the serial group comparisonl

Error in fisher.test(pre_table[, -3], alternative = "two.sided") : 
  FEXACT error 7(location). LDSTP=18630 is too small for this problem,
  (pastp=22.2575, ipn_0:=ipoin[itp=555]=2557, stp[ipn_0]=16.1017).
Increase workspace or consider using 'simulate.p.value=TRUE'
In addition: There were 30 warnings (use warnings() to see them)

The first OTU to pass the filtering used seems to run Ok as shown below, but none of the others are studied;

1	Richness	1	1
2	Shannon.effective	1	1
3	Simpson.effective	1	1
4	OTU_5697	1	1

I have tried both solutions posted on this overflow post without success (https://stackoverflow.com/questions/17052639/fisher-test-error-ldstp-is-too-small)

Is it possible to implement another version of the fisher test which works with larger datasets or remove the fisher test so only the Wilcoxon test is conducted and plotted as currently the Wilcoxon tests are done and printed to a file but the information is not plotted.

[SandraReitmeier]

Hi Thomas, I hope we will find a solution for your problem. What i'm always tried when I get this error is so increase the workspace fisher.test(input_table, workspace = 2e8) This always worked for me. If you still get error messages you can try to surrend the method by a trycatch function. I can implement it work you, if you are not used to programming. Let me know how it worked! Kind regards, Sandra

…

___________________________ Technische Universität München Sandra Fischer ZIEL - Institute for Food & Health ZIEL PhD Kolleg Core Facility Mikrobiom / NGS Weihenstephaner Berg 3 85354 Freising, Germany Tel: +49-(0)8161-715523<tel:%2B49-%280%298161-712371> sandra.fischer@tum.de www.ziel.tum.de

________________________________ Von: Thomas Hitch <notifications@github.com> Gesendet: Donnerstag, 29. November 2018 13:15:35 An: Lagkouvardos/Rhea Cc: Subscribed Betreff: [Lagkouvardos/Rhea] Unable to run fisher test due to LDSTP being too small (#17) Currently getting the following issue when running the serial group comparisonl FEXACT error 7(location). LDSTP=18630 is too small for this problem, (pastp=22.2575, ipn_0:=ipoin[itp=555]=2557, stp[ipn_0]=16.1017). Increase workspace or consider using 'simulate.p.value=TRUE' In addition: There were 30 warnings (use warnings() to see them) The first OTU to pass the filtering used seems to run Ok as shown below, but none of the others are studied; 1 Richness 1 1 2 Shannon.effective 1 1 3 Simpson.effective 1 1 4 OTU_5697 1 1``` I have tried both solutions posted on this overflow post without success (https://stackoverflow.com/questions/17052639/fisher-test-error-ldstp-is-too-small) Is it possible to implement another version of the fisher test which works with larger datasets or remove the fisher test so only the Wilcoxon test is conducted and plotted as currently the Wilcoxon tests are done and printed to a file but the information is not plotted. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#17>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP5YMVsozQ5hfsUBQ77E_lJ4MJxayYHpks5uz8_ngaJpZM4Y5jWm>.

[thh32]

Hi Sandra,

I tried increasing the workspace with increments of 2e8, 2e9, 2e11, 2e15, 2e100, 2e1000. However none of these seemed to solve the problem.

I havnt used a trycatch function in R before, could you perhaps explain how to go about this.

Thanks,
Tom

[SandraReitmeier]

Have your tried it with a lower workspace number as well? 2e7? For me 2e8 was the maxinum my computer could not handle 2e10 at all. You could as well send me your input file and I I'll try it.

…

___________________________ Technische Universität München Sandra Fischer ZIEL - Institute for Food & Health ZIEL PhD Kolleg Core Facility Mikrobiom / NGS Weihenstephaner Berg 3 85354 Freising, Germany Tel: +49-(0)8161-715523 sandra.fischer@tum.de www.ziel.tum.de

________________________________ Von: Thomas Hitch <notifications@github.com> Gesendet: Donnerstag, 29. November 2018 16:03:24 An: Lagkouvardos/Rhea Cc: Sandra Fischer; Comment Betreff: Re: [Lagkouvardos/Rhea] Unable to run fisher test due to LDSTP being too small (#17) Hi Sandra, I tried increasing the workspace with increments of 2e8, 2e9, 2e11, 2e15, 2e100, 2e1000. However none of these seemed to solve the problem. I havnt used a trycatch function in R before, could you perhaps explain how to go about this. Thanks, Tom — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#17 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP5YMRtct_kxEh9PlRNLwsq5de9hZNfxks5uz_c8gaJpZM4Y5jWm>.

[thh32]

I have just tried 2e7 and the error message seems to be the same.

Attached is the OTUsCombined.tab file I was trying to analyse.

OTUsCombined.tab.gz

Serial-Group-Comparisons.R.gz

[SandraReitmeier]

Hi Tom, I tried it and it worked fine. I took mouse type as independent variable name since this was the only one without missing values. It is important that your mapping_file is complete. If you do not have the information replace it with a value with referes to mssing. attached you find the output for mouse type #' Please give the file name generated by the Create_Input_Table.R script input_filename = "OTUsCombined.tab" #<--- CHANGE ACCORDINGLY !!! #' The name of the independant variable that the analysis will be performed on independant_variable_name <- "Mouse type" #' Please enter the position in the table (column number) where the dependant variable starts (e.g. Richness) #' Note: the first column containing sample names does not count! dependant_variables_start <- 20 #' Please enter the order of the group names #' If no group names are writting groups are ordered automatically group_order=c("") #' Please enter the position in the table (column number) where relative abundances of OTUs or taxonomic groups start #' Note: the first column containing sample names does not count! taxonomic_variables_start <- 23 Best Sandra

…

___________________________ Technische Universität München Sandra Fischer ZIEL - Institute for Food & Health ZIEL PhD Kolleg Core Facility Mikrobiom / NGS Weihenstephaner Berg 3 85354 Freising, Germany Tel: +49-(0)8161-715523<tel:%2B49-%280%298161-712371> sandra.fischer@tum.de www.ziel.tum.de

________________________________ Von: Thomas Hitch <notifications@github.com> Gesendet: Freitag, 30. November 2018 08:21:17 An: Lagkouvardos/Rhea Cc: Sandra Fischer; Comment Betreff: Re: [Lagkouvardos/Rhea] Unable to run fisher test due to LDSTP being too small (#17) I have just tried 2e7 and the error message seems to be the same. Attached is the OTUsCombined.tab file I was trying to analyse. OTUsCombined.tab.gz<https://github.com/Lagkouvardos/Rhea/files/2632384/OTUsCombined.tab.gz> — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#17 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP5YMYfiav8bKZF6EF2W8JTVdSArQLpnks5u0NxtgaJpZM4Y5jWm>.

[thh32]

OK I just tried using the IV of "Mouse type" and it worked fine as you said however the one I have been studying is "Timepoint_Type" which was manually added by me and has all the information so that shouldnt be an issue. Looking into it further I found that the "Timepoint_facility" IV also worked fine. The only difference I could see between the IVs that worked and those that dont is the number of groups, which made me wonder what would happen if I removed some of the samples from groups not really needing to be studied e.g. Dam-faeces and Dam-CC. Once they were removed the issue was gone and the analysis worked perfectly.

Due to this it seems as though the issue is that the fisher test can only work on 7 or less groups.

It might be worth adding an option at the top to prevent the use of the fisher test so that users with datasets including >7 groups can avoid this issue.

Thanks,
Tom

[SandraReitmeier]

Mhm this is strange, I tried it with !Timepoint_facility" with all groups and it worked, without removing any groups. Line: 460 Fishtest <- fisher.test(pre_table[,-3],alternative = "two.sided", workspace=2e7) Line 477 # Compute two-sided Fisher's test Line 477 fppval_res[i] <- fisher.test(pretrowbind,alternative = "two.sided",workspace=2e8)$p.value You added the workspace in both lines? Kind regards Sandra

…

___________________________ Technische Universität München Sandra Fischer ZIEL - Institute for Food & Health ZIEL PhD Kolleg Core Facility Mikrobiom / NGS Weihenstephaner Berg 3 85354 Freising, Germany Tel: +49-(0)8161-715523<tel:%2B49-%280%298161-712371> sandra.fischer@tum.de www.ziel.tum.de

________________________________ Von: Thomas Hitch <notifications@github.com> Gesendet: Freitag, 30. November 2018 12:27:58 An: Lagkouvardos/Rhea Cc: Sandra Fischer; Comment Betreff: Re: [Lagkouvardos/Rhea] Unable to run fisher test due to LDSTP being too small (#17) OK I just tried using the IV of "Mouse type" and it worked fine as you said however the one I have been studying is "Timepoint_Type" which was manually added by me and has all the information so that shouldnt be an issue. Looking into it further I found that the "Timepoint_facility" IV also worked fine. The only difference I could see between the IVs that worked and those that dont is the number of groups, which made me wonder what would happen if I removed some of the samples from groups not really needing to be studied e.g. Dam-faeces and Dam-CC. Once they were removed the issue was gone and the analysis worked perfectly. Due to this it seems as though the issue is that the fisher test can only work on 7 or less groups. It might be worth adding an option at the top to prevent the use of the fisher test so that users with datasets including >7 groups can avoid this issue. Thanks, Tom — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#17 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AP5YMWDYVTfTYkVHtaSFLM3O9mVlNrMwks5u0RY-gaJpZM4Y5jWm>.

[thh32]

SOLVED!

Thank you Sandra, turned out I had only edited the workspace on line 477, not line 460. Now it works perfectly!