Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when running fastqPairedFilter #212

Closed
lfaller opened this issue Mar 20, 2017 · 56 comments
Closed

Error when running fastqPairedFilter #212

lfaller opened this issue Mar 20, 2017 · 56 comments
Labels
Milestone

Comments

@lfaller
Copy link

lfaller commented Mar 20, 2017

Hi all,

First of all, thanks for making dada2 available! I've only played with the tutorials (the miseq PE-read walkthrough) but the features and capabilities look great so far!

I ran into a strange error withfastqPairedFilter() that seems to occur when I try to run the sample walkthrough code in knitr.


fastqPairedFilter(c(fnFs[i], fnRs[i]), c(filtFs[i], filtRs[i]),
                    truncLen=c(240,160),
                    maxN=0, maxEE=c(2,2), truncQ=2, rm.phix=TRUE,
                    compress=TRUE, verbose=TRUE)

Error in validObject(.Object) : 
  invalid class "SRFilterResult" object: superclass "Mnumeric" not defined in the environment of the object's class
Calls: <Anonymous> ... SRFilterResult -> new -> initialize -> initialize -> validObject

This did not occur when I ran the code manually on the command line however I can't imagine what would be different in the knitr environment (it sets up all the same libraries, etc). In fact, I set up my markdown document and copied and pasted into the command line from there while practicing. I am looking to automate basic processing of 16S data at some point, so being able to run it through knitr to produce a report that includes QC plots etc would be attractive.

Thanks for any suggestions or pointers to additional reading you might have!

@benjjneb
Copy link
Owner

Haven't seen that one before. Which package are you using to knit with, rmarkdown or knitr?

Can you report the output of the following:

packageVersion("knitr")
packageVersion("dada2")
packageVersion("ShortRead")
R.version

@lfaller
Copy link
Author

lfaller commented Mar 20, 2017

Yes, I should have included that information!

sessionInfo()
## R version 3.3.1 (2016-06-21)
## Platform: x86_64-apple-darwin13.4.0 (64-bit)
## Running under: OS X 10.11.6 (El Capitan)
## 
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
## 
## attached base packages:
## [1] stats4    parallel  stats     graphics  grDevices utils     datasets 
## [8] methods   base     
## 
## other attached packages:
##  [1] phyloseq_1.19.1            gridExtra_2.2.1           
##  [3] RColorBrewer_1.1-2         pheatmap_1.0.8            
##  [5] ggplot2_2.2.1              ShortRead_1.32.1          
##  [7] GenomicAlignments_1.10.1   SummarizedExperiment_1.4.0
##  [9] Biobase_2.34.0             Rsamtools_1.26.1          
## [11] GenomicRanges_1.26.4       GenomeInfoDb_1.10.3       
## [13] Biostrings_2.42.1          XVector_0.14.1            
## [15] IRanges_2.8.2              S4Vectors_0.12.2          
## [17] BiocParallel_1.8.1         BiocGenerics_0.20.0       
## [19] dada2_1.2.2                Rcpp_0.12.7               
## [21] knitr_1.14                
## 
## loaded via a namespace (and not attached):
##  [1] reshape2_1.4.2      splines_3.3.1       lattice_0.20-33    
##  [4] rhdf5_2.18.0        colorspace_1.3-2    htmltools_0.3.5    
##  [7] mgcv_1.8-12         yaml_2.1.13         survival_2.41-2    
## [10] foreach_1.4.3       plyr_1.8.4          stringr_1.1.0      
## [13] zlibbioc_1.20.0     munsell_0.4.3       gtable_0.2.0       
## [16] hwriter_1.3.2       codetools_0.2-14    evaluate_0.10      
## [19] labeling_0.3        latticeExtra_0.6-28 permute_0.9-4      
## [22] biomformat_1.2.0    scales_0.4.1        backports_1.0.5    
## [25] formatR_1.4         vegan_2.4-2         RcppParallel_4.3.20
## [28] jsonlite_1.1        digest_0.6.10       stringi_1.1.2      
## [31] grid_3.3.1          rprojroot_1.2       ade4_1.7-5         
## [34] tools_3.3.1         bitops_1.0-6        magrittr_1.5       
## [37] lazyeval_0.2.0      RCurl_1.95-4.8      tibble_1.2         
## [40] cluster_2.0.4       ape_4.1             MASS_7.3-45        
## [43] Matrix_1.2-6        data.table_1.10.4   assertthat_0.1     
## [46] rmarkdown_1.3       iterators_1.0.8     multtest_2.30.0    
## [49] igraph_1.0.1        nlme_3.1-128

@joey711
Copy link
Collaborator

joey711 commented Mar 20, 2017

Someone else has run into this same error with a different (QuasR) package:

https://support.bioconductor.org/p/92701/

No answer there, though, either. My guess is that it's propagating out from a change in one of the dependencies.

@lfaller
Copy link
Author

lfaller commented Mar 20, 2017

Ah that is a bummer :-( I will look into writing a regular R script to get around this. Thanks!

@joey711
Copy link
Collaborator

joey711 commented Mar 20, 2017

The solution may be as easy as fully loading the package at the beginning of your Rmd... e.g.

library("ShortRead")

and so on, until the error goes away. I can't guarantee it will work, but it is one of the things I'd try if Rmd/HTML was a high priority.

@lfaller
Copy link
Author

lfaller commented Mar 21, 2017

I did actually load it in the very beginning of my script.

@melindahiggins2000
Copy link

I'm getting this error as well, but on a Windows 7 operating system. At first, I thought this might be an RTools installation problem, but I've checked RTools and it is good to go. I'm also running the walkthrough demo successfully on a Windows 10 operating system. Is there a chance, this is an operating system specific error?

My code and error result:

> for (i in seq_along(fileNameForwards)) {
+   fastqPairedFilter(c(fileNameForwards[i], fileNameReverses[i]), 
+                     c(filtForwards[i], filtReverses[i]),
+                     truncLen=c(240,160),
+                     maxN = 0, maxEE = c(2,2),
+                     truncQ = 2, rm.phix=TRUE,
+                     compress=TRUE, verbose=TRUE)
+ }
 Show Traceback
 
 Rerun with Debug
 Error in validObject(.Object) : 
  invalid class “SRFilterResult” object: superclass "Mnumeric" not defined in the environment of the object's class 
> 

@lfaller
Copy link
Author

lfaller commented Mar 29, 2017

I am not sure if this error is actually caused by knitr, as I had previously assumed. I specifically unloaded the knitr package and I still encounter the same error:

> # Filter
> for(i in seq_along(fnFs)) {
+   fastqPairedFilter(c(fnFs[i], fnRs[i]), c(filtFs[i], filtRs[i]),
+                     truncLen=c(240,160),
+                     maxN=0, maxEE=c(2,2), truncQ=2, rm.phix=TRUE,
+                     compress=TRUE, verbose=TRUE)
+ }

 Error in validObject(.Object) : 
  invalid class “SRFilterResult” object: superclass "Mnumeric" not defined in the environment of the object's class 

> sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.11.6 (El Capitan)

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats4    parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] gridExtra_2.2.1            ggplot2_2.2.1              ShortRead_1.32.1           GenomicAlignments_1.10.1   SummarizedExperiment_1.4.0 Biobase_2.34.0            
 [7] Rsamtools_1.26.1           GenomicRanges_1.26.4       GenomeInfoDb_1.10.3        Biostrings_2.42.1          XVector_0.14.1             IRanges_2.8.2             
[13] S4Vectors_0.12.2           BiocParallel_1.8.1         BiocGenerics_0.20.0        dada2_1.2.2                Rcpp_0.12.7               

loaded via a namespace (and not attached):
 [1] magrittr_1.5        zlibbioc_1.20.0     munsell_0.4.3       lattice_0.20-33     colorspace_1.3-2    stringr_1.1.0       hwriter_1.3.2       plyr_1.8.4         
 [9] tools_3.3.1         grid_3.3.1          data.table_1.10.4   gtable_0.2.0        latticeExtra_0.6-28 lazyeval_0.2.0      RcppParallel_4.3.20 assertthat_0.1     
[17] tibble_1.2          Matrix_1.2-6        reshape2_1.4.2      RColorBrewer_1.1-2  bitops_1.0-6        RCurl_1.95-4.8      stringi_1.1.2       scales_0.4.1

@lfaller
Copy link
Author

lfaller commented Mar 29, 2017

The traceback for the error message:

8. stop(msg, ": ", errors, domain = NA) 
7. validObject(.Object) 
6. initialize(value, ...) 
5. initialize(value, ...) 
4. new("SRFilterResult", x, name = mkScalar(as.character(name)[length(name)]), 
    stats = data.frame(Name = as.character(name), Input = input, 
        Passing = passing, Op = op, stringsAsFactors = FALSE)) 
3. SRFilterResult(res, structure("MinQFilter", class = structure("ScalarCharacter", package = "Biobase"))) 
2. minQFilter(minQ[[1]])(fqF) 
1. fastqPairedFilter(c(fnFs[i], fnRs[i]), c(filtFs[i], filtRs[i]), 
    truncLen = c(240, 160), maxN = 0, maxEE = c(2, 2), truncQ = 2, 
    rm.phix = TRUE, compress = TRUE, verbose = TRUE) 

@benjjneb
Copy link
Owner

I strongly suspect this is an issue w/in the ShortRead package, and the fact that I haven't been able to reproduce it thus far makes things hard, but if multiple people are seeing it then we need to try to get to the bottom of it.

@melindahiggins2000 Can you also post the sessionInfo() output for your windows 7 system where the bug arises, and your windows 10 system where it doesn't?

@benjjneb benjjneb added the bug label Mar 29, 2017
@vhertzb
Copy link

vhertzb commented Mar 29, 2017

@melindahiggins2000 and I were teaching the dada2 pipeline today. We asked everybody to come prepared with everything installed and downloaded and had them do so as homework that they posted to their class github repo. We had at least one other Windows10 user who had the same issue. I will ask for her sessionInfo as well.

@melindahiggins2000
Copy link

Here is our sessionInfo() that is giving us the error on our lecturn computer:

> 
> for (i in seq_along(fileNameForwards)) {
+   fastqPairedFilter(c(fileNameForwards[i], fileNameReverses[i]), 
+                     c(filtForwards[i], filtReverses[i]),
+                     truncLen=c(240,160),
+                     maxN = 0, maxEE = c(2,2),
+                     truncQ = 2, rm.phix=TRUE,
+                     compress=TRUE, verbose=TRUE)
+ }
 Show Traceback
 
 Rerun with Debug
 Error in validObject(.Object) : 
  invalid class “SRFilterResult” object: superclass "Mnumeric" not defined in the environment of the object's class 

> sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats4    parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] ggplot2_2.2.1              ShortRead_1.32.1           GenomicAlignments_1.10.1  
 [4] SummarizedExperiment_1.4.0 Biobase_2.34.0             Rsamtools_1.26.1          
 [7] GenomicRanges_1.26.4       GenomeInfoDb_1.10.3        Biostrings_2.42.1         
[10] XVector_0.14.1             IRanges_2.8.2              S4Vectors_0.12.2          
[13] BiocParallel_1.8.1         BiocGenerics_0.20.0        dada2_1.2.2               
[16] Rcpp_0.12.10              

loaded via a namespace (and not attached):
 [1] magrittr_1.5        zlibbioc_1.20.0     munsell_0.4.3       lattice_0.20-34     colorspace_1.3-2   
 [6] stringr_1.2.0       hwriter_1.3.2       plyr_1.8.4          tools_3.3.2         grid_3.3.2         
[11] data.table_1.10.4   gtable_0.2.0        latticeExtra_0.6-28 lazyeval_0.2.0      RcppParallel_4.3.20
[16] assertthat_0.1      tibble_1.2          Matrix_1.2-7.1      reshape2_1.4.2      RColorBrewer_1.1-2 
[21] bitops_1.0-6        RCurl_1.95-4.8      labeling_0.3        stringi_1.1.3       scales_0.4.1       
> 

I did not run the traceback.

@benjjneb
Copy link
Owner

I cannot replicate this, but the srFilter function that is causing troubles is no longer used in the package. Thus it is very likely this bug is fixed in the current devel version (1.3.4+), and will be fixed in our next release (1.4, later this month).

Please reopen if this issue reappears in the newer versions of the package.

@caledonius
Copy link

This is still happening in dada2 version 1.2.2 and 1.5.0 on Win10 on both the command line and within markdown. Never happens with 1.2.1 on my Win10 laptop.

@benjjneb
Copy link
Owner

Grumble. I am confused.

@caledonius Can you report a sessionInfo() and traceback() for when you see this error in 1.5.0?

@benjjneb benjjneb reopened this Apr 26, 2017
@benjjneb benjjneb added this to the 1.6 milestone May 14, 2017
@benjjneb
Copy link
Owner

benjjneb commented Oct 5, 2017

Closing this in the absence of any further reports. I've never been able to reproduce this bug.

@benjjneb benjjneb closed this as completed Oct 5, 2017
@frannasc
Copy link

frannasc commented Jan 27, 2018

Hi guys,
I'm having the same problem when running:
filterAndTrim(fwd=file.path(pathF, fastqFs), filt=file.path(filtpathF, fastqFs),
rev=file.path(pathR, fastqRs), filt.rev=file.path(filtpathR, fastqRs),
truncLen=c(150,145),trimLeft=c(12, 12), maxEE=2, truncQ=2, maxN=0, rm.phix=TRUE,
compress=TRUE, verbose=TRUE)
Error in validObject(.Object) :
invalid class "SRFilterResult" object: superclass "Mnumeric" not defined in the environment of the object's class

packageVersion("knitr")
[1] '1.15.1'
packageVersion("dada2")
[1] '1.6.0'
packageVersion("ShortRead")
[1] '1.32.1'
R.version
_
platform x86_64-apple-darwin13.4.0
arch x86_64
os darwin13.4.0
system x86_64, darwin13.4.0
status
major 3
minor 3.2
year 2016
month 10
day 31
svn rev 71607
language R
version.string R version 3.3.2 (2016-10-31)
nickname Sincere Pumpkin Patch

Any help on this?
Francisco

@vhertzb
Copy link

vhertzb commented Jan 28, 2018

@frannasc Update your R to 3.4.2 and see if the same error occurs.

@SantanuMBLab
Copy link

Hello,
I am having this issue again
Error in validObject(.Object) :
invalid class “SRFilterResult” object: superclass "Mnumeric" not defined in the environment of the object's class
My session info

sessionInfo()
R version 3.3.0 (2016-05-03)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 17134)

locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages:
[1] stats4 parallel stats graphics grDevices utils datasets methods base

other attached packages:
[1] ggplot2_2.2.1 ShortRead_1.32.1 GenomicAlignments_1.10.1 SummarizedExperiment_1.4.0 Biobase_2.34.0 Rsamtools_1.26.2
[7] GenomicRanges_1.26.4 GenomeInfoDb_1.10.3 Biostrings_2.42.1 XVector_0.14.1 IRanges_2.8.2 S4Vectors_0.12.2
[13] BiocParallel_1.8.2 BiocGenerics_0.20.0 dada2_1.2.2 Rcpp_0.12.16

loaded via a namespace (and not attached):
[1] magrittr_1.5 zlibbioc_1.20.0 munsell_0.5.0 lattice_0.20-33 colorspace_1.3-2 rlang_0.2.0 stringr_1.3.1 hwriter_1.3.2 plyr_1.8.4
[10] tools_3.3.0 grid_3.3.0 data.table_1.10.4-3 gtable_0.2.0 latticeExtra_0.6-28 lazyeval_0.2.1 RcppParallel_4.4.0 tibble_1.4.2 Matrix_1.2-6
[19] reshape2_1.4.3 RColorBrewer_1.1-2 bitops_1.0-6 RCurl_1.95-4.10 labeling_0.3 stringi_1.1.7 pillar_1.2.3 scales_0.5.0

@benjjneb
Copy link
Owner

R version 3.3.0 (2016-05-03)
other attached packages:
[1] ggplot2_2.2.1 ShortRead_1.32.1 GenomicAlignments_1.10.1 SummarizedExperiment_1.4.0 Biobase_2.34.0 Rsamtools_1.26.2
[7] GenomicRanges_1.26.4 GenomeInfoDb_1.10.3 Biostrings_2.42.1 XVector_0.14.1 IRanges_2.8.2 S4Vectors_0.12.2
[13] BiocParallel_1.8.2 BiocGenerics_0.20.0 dada2_1.2.2 Rcpp_0.12.16

The software versions you have are pretty old (~2 years) at this point. If you update to current versions this error will probably be resolved.

I would recommend installing R 3.5, and then reinstalling dada2 via Bioconductor, which will update all the dependencies as well.

If you can't update R, you can try installing the current version of dada2 via github: library(devtools); devtools::install_github("benjjnbeb/dada2", ref="v1.8").

@SantanuMBLab
Copy link

Thank you for the kind suggestion.
Things are working fine with dada2 1.8 installed with R 3.5.1

@cpavloud
Copy link

I also have this issue

Filter

for(i in seq_along(fnFs)) {

  • fastqPairedFilter(c(fnFs[i], fnRs[i]), c(filtFs[i], filtRs[i]),
  •                 truncLen=c(290), matchIDs=TRUE,
    
  •                 maxN=0, truncQ=2, rm.phix=TRUE,
    
  •                 compress=FALSE, verbose=TRUE)
    
  • }
    Error in validObject(.Object) :
    invalid class "SRFilterResult" object: superclass "Mnumeric" not defined in the environment of the object's class
    Calls: fastqPairedFilter ... SRFilterResult -> new -> initialize -> initialize -> validObject
    Execution halted

R version 3.3.3 (2017-03-06) -- "Another Canoe"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

packageVersion("dada2"): '1.2.2'
packageVersion("ShortRead"): '1.32.0'
packageVersion("ggplot2"): '2.2.1'

@benjjneb
Copy link
Owner

@cpavloud

If you upgrade to a recent version of the dada2 R package (1.6 or later) this issue will go away. The 1.2 version is almost 2 years old now.

@bashirhamidi
Copy link

@benjjneb apologies for commenting on a closed issue. I'm encountering this same error when I run

filterAndTrim(fwd = fnFs, filt = filtFs,
                       rev = fnRs, filt.rev = filtRs) 

Error in validObject(.Object) : invalid class “SRFilterResult” object: superclass "Mnumeric" not defined in the environment of the object's class

My traceback() is as follows:

16: stop(msg, ": ", errors, domain = NA)
15: validObject(.Object)
14: initialize(value, ...)
13: initialize(value, ...)
12: new("SRFilterResult", x, name = mkScalar(as.character(name)[length(name)]), 
        stats = data.frame(Name = as.character(name), Input = input, 
            Passing = passing, Op = op, stringsAsFactors = FALSE))
11: SRFilterResult(res, new("ScalarCharacter", .Data = "CleanNFilter"))
10: nFilter(maxN[[1]])(fqF)
9: withCallingHandlers(expr, warning = function(w) if (inherits(w, 
       classes)) tryInvokeRestart("muffleWarning"))
8: suppressWarnings(keep <- nFilter(maxN[[1]])(fqF) & nFilter(maxN[[2]])(fqR))
7: (function (fn, fout, maxN = c(0, 0), truncQ = c(2, 2), truncLen = c(0, 
       0), maxLen = c(Inf, Inf), minLen = c(20, 20), trimLeft = c(0, 
       0), trimRight = c(0, 0), minQ = c(0, 0), maxEE = c(Inf, Inf), 
       rm.phix = c(TRUE, TRUE), rm.lowcomplex = c(0, 0), matchIDs = FALSE, 
       orient.fwd = NULL, id.sep = "\\s", id.field = NULL, n = 1e+06, 
       OMP = TRUE, qualityType = "Auto", compress = TRUE, verbose = FALSE, 
       ...) 
6: .mapply(FUN, dots, MoreArgs)
5: FUN(X[[i]], ...)
4: lapply(X = X, FUN = FUN, ...)
3: mclapply(seq_len(n), do_one, mc.preschedule = mc.preschedule, 
       mc.set.seed = mc.set.seed, mc.silent = mc.silent, mc.cores = mc.cores, 
       mc.cleanup = mc.cleanup, affinity.list = affinity.list)
2: mcmapply(fastqPairedFilter, mapply(c, fwd, rev, SIMPLIFY = FALSE), 
       mapply(c, filt, filt.rev, SIMPLIFY = FALSE), MoreArgs = list(truncQ = truncQ, 
           truncLen = truncLen, trimLeft = trimLeft, trimRight = trimRight, 
           maxLen = maxLen, minLen = minLen, maxN = maxN, minQ = minQ, 
           maxEE = maxEE, rm.phix = rm.phix, rm.lowcomplex = rm.lowcomplex, 
           orient.fwd = orient.fwd, matchIDs = matchIDs, id.sep = id.sep, 
           id.field = id.field, n = n, OMP = OMP, qualityType = qualityType, 
           compress = compress, verbose = verbose), mc.cores = ncores, 
       mc.silent = TRUE)
1: filterAndTrim(fwd = fnFs, filt = filtFs, rev = fnRs, filt.rev = filtRs)

What I have tried:

  1. Explicitly loading the full libraries using library()
  2. Removing and reinstalling dada2, ShortRead, and Biostrings from Bioconductor, followed by a fresh session.
if (!requireNamespace("BiocManager", quietly = TRUE))
    install.packages("BiocManager")
BiocManager::install("Biostrings")
BiocManager::install("ShortRead")
BiocManager::install("dada2")```
  1. Reinstalling ShortRead from source + fresh session

BiocManager::install("ShortRead", type="source")

  1. Reinstalling ALL Bioconductor packages + fresh session
if (!requireNamespace("BiocManager", quietly = TRUE))
    install.packages("BiocManager")
pkgs <- rownames(installed.packages())
BiocManager::install(pkgs, type = "source", checkBuilt = TRUE)```
  1. Reinstalling dada2 from github + fresh session

devtools::install_github("benjjneb/dada2", force = TRUE)

  1. Reinstalling/updating Xcode, XQuartz, R, restart machine
  2. Banging head against the wall + fresh R session

Notable package versions are as follows:

> packageVersion("knitr")
[1] ‘1.33’
> packageVersion("dada2")
[1] ‘1.18.0’
> packageVersion("ShortRead")
[1] ‘1.48.0’
> R.version
platform       x86_64-apple-darwin17.0     
arch           x86_64                      
os             darwin17.0                  
system         x86_64, darwin17.0          
status                                     
major          4                           
minor          0.5                         
year           2021                        
month          03                          
day            31                          
svn rev        80133                       
language       R                           
version.string R version 4.0.5 (2021-03-31)
nickname       Shake and Throw   

Lastly, my SessionInfo is as follows:

R version 4.0.5 (2021-03-31)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Big Sur 10.16

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats4    parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] ShortRead_1.48.0            GenomicAlignments_1.26.0    SummarizedExperiment_1.20.0 Biobase_2.50.0             
 [5] MatrixGenerics_1.2.1        matrixStats_0.58.0          Rsamtools_2.6.0             GenomicRanges_1.42.0       
 [9] GenomeInfoDb_1.26.7         BiocParallel_1.24.1         Biostrings_2.58.0           XVector_0.30.0             
[13] IRanges_2.24.1              S4Vectors_0.28.1            BiocGenerics_0.36.1         dada2_1.18.0               
[17] Rcpp_1.0.6                 

loaded via a namespace (and not attached):
 [1] lattice_0.20-44        png_0.1-7              assertthat_0.2.1       utf8_1.2.1            
 [5] plyr_1.8.6             R6_2.5.0               ggplot2_3.3.3          pillar_1.6.0          
 [9] zlibbioc_1.36.0        rlang_0.4.11           rstudioapi_0.13        Matrix_1.3-3          
[13] stringr_1.4.0          RCurl_1.98-1.3         munsell_0.5.0          DelayedArray_0.16.3   
[17] compiler_4.0.5         xfun_0.22              pkgconfig_2.0.3        tidyselect_1.1.1      
[21] tibble_3.1.1           GenomeInfoDbData_1.2.4 fansi_0.4.2            crayon_1.4.1          
[25] dplyr_1.0.5            withr_2.4.2            bitops_1.0-7           grid_4.0.5            
[29] gtable_0.3.0           lifecycle_1.0.0        DBI_1.1.1              magrittr_2.0.1        
[33] scales_1.1.1           RcppParallel_5.1.4     stringi_1.5.3          cli_2.5.0             
[37] hwriter_1.3.2          reshape2_1.4.4         latticeExtra_0.6-29    ellipsis_0.3.2        
[41] generics_0.1.0         vctrs_0.3.8            RColorBrewer_1.1-2     tools_4.0.5           
[45] glue_1.4.2             purrr_0.3.4            jpeg_0.1-8.1           yaml_2.2.1            
[49] colorspace_2.0-1       sessioninfo_1.1.1      knitr_1.33       

@kendyhoangucsc
Copy link

Thank you guys! was running into the same issue and resolved by matrix version issue!

@benjjneb
Copy link
Owner

This should be fixed in the latest commit to the devel (master) branch. That is, it should work with Matrix 1.3.3.

@yinglu1
Copy link

yinglu1 commented May 13, 2021

Downgraded Matrix to 1.3-2 and the issue was resolved. Thanks!!

@emankhalaf
Copy link

Updating everyone that downgrading Matrix package to 1.3-2 has resolved the issue. Here's your steps to to get it resolved:

  1. remove.packages("Matrix")
  2. Clear workspace
  3. Restart session
  4. Install the previous version devtools::install_version("Matrix", version = "1.3.2", repos = "http://cran.us.r-project.org")
  5. Verify your package version is 1.3-2 using packageVersion("Matrix")
  6. Run your workflow as you would normally

Here's my sessionInfo() for reference:

sessionInfo()
## R version 4.0.5 (2021-03-31)
## Platform: x86_64-apple-darwin17.0 (64-bit)
## Running under: macOS Big Sur 10.16
## 
## Matrix products: default
## BLAS:   /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRblas.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib
## 
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
## 
## attached base packages:
## [1] stats4    parallel  stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] dada2_1.18.0                Rcpp_1.0.6                  ShortRead_1.48.0            GenomicAlignments_1.26.0    SummarizedExperiment_1.20.0
##  [6] Biobase_2.50.0              MatrixGenerics_1.2.1        matrixStats_0.58.0          Rsamtools_2.6.0             GenomicRanges_1.42.0       
## [11] GenomeInfoDb_1.26.7         Biostrings_2.58.0           XVector_0.30.0              IRanges_2.24.1              S4Vectors_0.28.1           
## [16] BiocParallel_1.24.1         BiocGenerics_0.36.1         Matrix_1.3-2               
## 
## loaded via a namespace (and not attached):
##  [1] lattice_0.20-44        png_0.1-7              assertthat_0.2.1       digest_0.6.27          utf8_1.2.1             plyr_1.8.6            
##  [7] R6_2.5.0               evaluate_0.14          highr_0.9              ggplot2_3.3.3          pillar_1.6.0           zlibbioc_1.36.0       
## [13] rlang_0.4.11           jquerylib_0.1.4        rmarkdown_2.8          labeling_0.4.2         stringr_1.4.0          RCurl_1.98-1.3        
## [19] munsell_0.5.0          DelayedArray_0.16.3    compiler_4.0.5         xfun_0.22              pkgconfig_2.0.3        htmltools_0.5.1.1     
## [25] tidyselect_1.1.1       tibble_3.1.1           GenomeInfoDbData_1.2.4 fansi_0.4.2            crayon_1.4.1           dplyr_1.0.6           
## [31] bitops_1.0-7           grid_4.0.5             DBI_1.1.1              jsonlite_1.7.2         gtable_0.3.0           lifecycle_1.0.0       
## [37] magrittr_2.0.1         scales_1.1.1           RcppParallel_5.1.4     stringi_1.6.1          farver_2.1.0           reshape2_1.4.4        
## [43] hwriter_1.3.2          latticeExtra_0.6-29    bslib_0.2.4            ellipsis_0.3.2         vctrs_0.3.8            generics_0.1.0        
## [49] RColorBrewer_1.1-2     tools_4.0.5            glue_1.4.2             purrr_0.3.4            jpeg_0.1-8.1           yaml_2.2.1            
## [55] colorspace_2.0-1       BiocManager_1.30.14    knitr_1.33             sass_0.3.1

Hi,
I have the same error, so, I installed the older version first (by mistake since I should remove the 1.3-3 first), and now I have 2 versions of matrix package showing in the right bottom corner of Rstudio. So, I wanted to redo the steps correctly, and uninstalled matrix package which removed the older version 1.3-2 but 1.3-3 is still there. So, I did the following and got these errors:
remove.packages("Matrix")
Removing package from ‘/home/eman/R/x86_64-pc-linux-gnu-library/3.6’
(as ‘lib’ is unspecified)
Error in remove.packages : there is no package called ‘Matrix’

library(Matrix, lib.loc = "/usr/lib/R/library")
remove.packages("Matrix", lib="/usr/lib/R/library")

Then I clicked on the adjacent X icon to Matrix package in the right bottom corner to remove it, and got this error:

Updating HTML index of packages in '.Library'
Warning in remove.packages :
cannot create file '/usr/share/R/doc/html/packages.html', reason 'Permission denied'
Warning in remove.packages :
cannot update HTML package index

Any recommendations please?
Thanks

@bashirhamidi
Copy link

@emankhalaf
Clear your workspace and start R. Then see if you actually have the Matrix package installed.

  1. Clear workspace
  2. Restart R session
  3. Run packageVersion("Matrix")

@emankhalaf
Copy link

@emankhalaf
Clear your workspace and start R. Then see if you actually have the Matrix package installed.

  1. Clear workspace
  2. Restart R session
  3. Run packageVersion("Matrix")

Much thanks for your swift response. I did the following:

packageVersion("Matrix")
[1] ‘1.3.3’
remove.packages("Matrix")
Removing package from ‘/home/eman/R/x86_64-pc-linux-gnu-library/3.6’
(as ‘lib’ is unspecified)
Error in remove.packages : there is no package called ‘Matrix’

So, how could I figure it out?

@bashirhamidi
Copy link

It seems you have multiple locations (paths) for your R libraries. You need to figure out where you have installed it.

  1. Run .libPaths() and it should give you a list of where R looks for your libraries.

At the very least it should have

/home/eman/R/x86_64-pc-linux-gnu-library/3.6

and one other location

  1. Then run remove.packages(pkgs = "Matrix", lib = "location of the second lib path"). Replace the text in quotations with the actual path.

@emankhalaf
Copy link

  1. remove.packages(pkgs = "Matrix", lib = "location of the second lib path")

I did this:

.libPaths()
[1] "/home/eman/R/x86_64-pc-linux-gnu-library/3.6" "/usr/local/lib/R/site-library"
[3] "/usr/lib/R/site-library" "/usr/lib/R/library"
Then,
remove.packages(pkgs = "Matrix", lib = "/home/eman/R/x86_64-pc-linux-gnu-library/3.6")
Error in remove.packages : there is no package called ‘Matrix’

and when ran

remove.packages(pkgs = "Matrix", lib = "/home/eman/R/x86_64-pc-linux-gnu-library/3.6" "/usr/local/lib/R/site-library")
Error: unexpected string constant in "remove.packages(pkgs = "Matrix", lib = "/home/eman/R/x86_64-pc-linux-gnu-library/3.6" "/usr/local/lib/R/site-library""

I have no idea how to solve it. Actually, I ran the entire script last week successfully, but when I re-ran it today, I got this error.

@bashirhamidi
Copy link

@emankhalaf When you input multiple paths to the remove.packages() function, it has no idea what to do with it.

Try These lines one at a time

  1. remove.packages(pkgs = "Matrix", lib = "/usr/local/lib/R/site-library")
  2. remove.packages(pkgs = "Matrix", lib = "/usr/lib/R/site-library")
  3. remove.packages(pkgs = "Matrix", lib = "/usr/lib/R/library")

@emankhalaf
Copy link

3. remove.packages(pkgs = "Matrix", lib = "/usr/lib/R/library")

Yes, for sure. I tried each and got these:

remove.packages(pkgs = "Matrix", lib = "/usr/local/lib/R/site-library")
Error in remove.packages : there is no package called ‘Matrix’

remove.packages(pkgs = "Matrix", lib = "/usr/lib/R/site-library")
Error in remove.packages : there is no package called ‘Matrix’

remove.packages(pkgs = "Matrix", lib = "/usr/lib/R/library")
Updating HTML index of packages in '.Library'
Warning in remove.packages :
cannot create file '/usr/share/R/doc/html/packages.html', reason 'Permission denied'
Warning in remove.packages :
cannot update HTML package index

So, I am not sure what should I do with HTML package?

@benjjneb
Copy link
Owner

benjjneb commented May 15, 2021

@emankhalaf I would consider installing the latest devel version of the dada2 R package as a fix here instead. It might be simpler, and solves the error a different way that is independent of the Matrix package version.

library(devtools)
devtools::install_github("benjjneb/dada2")

This latest devel version (1.19.2) will be the next release version of the package, and that release is imminent -- it should be out via BioConductor in a week or two.

@emankhalaf
Copy link

Hello,
I found this version (1.2.2) in Bioconductor
https://mirrors.nju.edu.cn/bioconductor/packages/3.4/bioc/html/dada2.html
Is it the most recent version?

@benjjneb
Copy link
Owner

@emankhalaf No, that is version 1.2.2 and 3+ years old at this point.
The most current version is 1.19.2. You can install it as described above.

@emankhalaf
Copy link

Hello,
I found this version (1.2.2) in Bioconductor
https://mirrors.nju.edu.cn/bioconductor/packages/3.4/bioc/html/dada2.html
Is it the most recent version?

@emankhalaf No, that is version 1.2.2 and 3+ years old at this point.
The most current version is 1.19.2. You can install it as described above.

Thanks! My current installed version is 1.19.1, however, I will be waiting for the latest as you recommend.

@kalen-rasmussen
Copy link

kalen-rasmussen commented May 26, 2021

Hey All,
Thanks for this tread and working so hard to troubleshoot and fix it. I wanted to quickly share what I ended up doing to resolve the issue in the hopes that it may help someone else. I originally encountered the issue while running dada2 1.18.0, ShortRead 1.48.0, and Matrix 1.3.3. I found this thread and updated dada2 to version 1.19.2 in the hopes that it would work with Matrix 1.3.3, but still encountered the same error while trying to perform the filterAndTrim step of the pipeline. I was able to resolve the issue by doing either of the following; 1. detaching ShortRead from my R environment, then running the filterAndTrim command, and reloading ShortRead when I next needed it, or 2. Loading the packages Matrix, ShortRead, and dada2 in that order. Either of these worked for me without having to downgrade to Matrix 1.3.2.
Hopefully this helps someone else, cheers.

Here is my info session for reference:

sessionInfo()
R version 4.0.5 (2021-03-31)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Big Sur 10.16

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats4    parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] ShortRead_1.48.0            GenomicAlignments_1.26.0    SummarizedExperiment_1.20.0 Biobase_2.50.0              MatrixGenerics_1.2.1       
 [6] matrixStats_0.58.0          Rsamtools_2.6.0             GenomicRanges_1.42.0        GenomeInfoDb_1.26.7         Biostrings_2.58.0          
[11] XVector_0.30.0              IRanges_2.24.1              S4Vectors_0.28.1            BiocParallel_1.24.1         BiocGenerics_0.36.1        
[16] Matrix_1.3-3                dada2_1.19.2                Rcpp_1.0.6                  phyloseq_1.34.0            

loaded via a namespace (and not attached):
 [1] jsonlite_1.7.2         splines_4.0.5          foreach_1.5.1          RcppParallel_5.1.4     assertthat_0.2.1       latticeExtra_0.6-29   
 [7] GenomeInfoDbData_1.2.4 progress_1.2.2         pillar_1.6.1           lattice_0.20-44        glue_1.4.2             RColorBrewer_1.1-2    
[13] colorspace_2.0-1       plyr_1.8.6             pkgconfig_2.0.3        zlibbioc_1.36.0        purrr_0.3.4            scales_1.1.1          
[19] jpeg_0.1-8.1           tibble_3.1.2           mgcv_1.8-35            generics_0.1.0         ggplot2_3.3.3          ellipsis_0.3.2        
[25] survival_3.2-11        magrittr_2.0.1         crayon_1.4.1           fansi_0.5.0            nlme_3.1-152           MASS_7.3-54           
[31] hwriter_1.3.2          vegan_2.5-7            tools_4.0.5            data.table_1.14.0      prettyunits_1.1.1      hms_1.1.0             
[37] lifecycle_1.0.0        stringr_1.4.0          Rhdf5lib_1.12.1        munsell_0.5.0          cluster_2.1.2          DelayedArray_0.16.3   
[43] ade4_1.7-16            compiler_4.0.5         tinytex_0.31           rlang_0.4.11           rhdf5_2.34.0           grid_4.0.5            
[49] RCurl_1.98-1.3         iterators_1.0.13       rhdf5filters_1.2.1     biomformat_1.18.0      rstudioapi_0.13        igraph_1.2.6          
[55] bitops_1.0-7           gtable_0.3.0           codetools_0.2-18       multtest_2.46.0        DBI_1.1.1              reshape2_1.4.4        
[61] R6_2.5.0               knitr_1.33             dplyr_1.0.6            utf8_1.2.1             permute_0.9-5          ape_5.5               
[67] stringi_1.6.2          png_0.1-7              vctrs_0.3.8            tidyselect_1.1.1       xfun_0.23 

@telatin
Copy link

telatin commented May 27, 2021

Hello,
I'm also having this issue. I use exclusively tools from BioConda, and now have a minimal script that fails with a freshly created environment.

The script

library(dada2); packageVersion("dada2")
args <- commandArgs(TRUE)
path <- args[[1]]

# Forward and reverse fastq filenames have format: SAMPLENAME_R1_001.fastq and SAMPLENAME_R2_001.fastq
fnFs <- sort(list.files(path, pattern="_R1_001.fastq", full.names = TRUE))
fnRs <- sort(list.files(path, pattern="_R2_001.fastq", full.names = TRUE))

# Extract sample names, assuming filenames have format: SAMPLENAME_XXX.fastq
sample.names <- sapply(strsplit(basename(fnFs), "_"), `[`, 1)

# Place filtered files in filtered/ subdirectory
filtFs <- file.path(path, "filtered", paste0(sample.names, "_F_filt.fastq.gz"))
filtRs <- file.path(path, "filtered", paste0(sample.names, "_R_filt.fastq.gz"))
out <- filterAndTrim(fnFs, filtFs, fnRs, filtRs, truncLen=c(240,160),
              maxN=0, maxEE=c(2,2), truncQ=2, rm.phix=TRUE,
              compress=TRUE, multithread=TRUE)

System info

I initially had the problem in a Ubuntu box, but replicated it locally on my Mac (macOS 11.3.1) so I print here
the differences between a freshly created environment and the last working environment.

NEW NON WORKING ENV.            LAST WORKING ENVIRONMENT
==============================================================                

 * Checking library:  dada2 	 * Checking library:  dada2 
 
Packages              version   Packages              version
dada2                  1.18.0   dada2                  1.18.0
Rcpp                    1.0.6   Rcpp                    1.0.6

Loaded.               version.  Loaded                version
Biobase                2.50.0	Biobase                2.50.0
BiocGenerics           0.36.0	BiocGenerics           0.36.0
BiocParallel           1.24.1	BiocParallel           1.24.1
Biostrings             2.58.0	Biostrings             2.58.0
bitops                  1.0-7	bitops                  1.0-7
colorspace              2.0-1	colorspace              2.0-0 *
compiler                4.0.5	compiler                4.0.3 *
crayon                  1.4.1	crayon                  1.4.1
DelayedArray           0.16.3	DelayedArray           0.16.3
dplyr                   1.0.6	dplyr                   1.0.5 *
ellipsis                0.3.2	ellipsis                0.3.1 *
fansi                   0.5.0	fansi                   0.4.2 **
generics                0.1.0	generics                0.1.0
GenomeInfoDb           1.26.4	GenomeInfoDb           1.26.4
GenomeInfoDbData        1.2.4	GenomeInfoDbData        1.2.4
GenomicAlignments      1.26.0	GenomicAlignments      1.26.0
GenomicRanges          1.42.0	GenomicRanges          1.42.0
ggplot2                 3.3.3	ggplot2                 3.3.3
glue                    1.4.2	glue                    1.4.2
grid                    4.0.5	grid                    4.0.3 *
gtable                  0.3.0	gtable                  0.3.0
hwriter                 1.3.2	hwriter                 1.3.2
IRanges                2.24.1	IRanges                2.24.1
jpeg                  0.1-8.1	jpeg                  0.1-8.1
lattice               0.20-44	lattice               0.20-41 *
latticeExtra           0.6-29	latticeExtra           0.6-29
lifecycle               1.0.0	lifecycle               1.0.0
magrittr                2.0.1	magrittr                2.0.1
Matrix                  1.3-3	Matrix                  1.3-2 *
MatrixGenerics          1.2.1	MatrixGenerics          1.2.1
matrixStats            0.58.0	matrixStats            0.58.0
munsell                 0.5.0	munsell                 0.5.0
parallel                4.0.5	parallel                4.0.3 *
pillar                  1.6.1	pillar                  1.6.0 *
pkgconfig               2.0.3	pkgconfig               2.0.3
plyr                    1.8.6	plyr                    1.8.6
png                     0.1-7	png                     0.1-7
purrr                   0.3.4	purrr                   0.3.4
R6                      2.5.0	R6                      2.5.0
RColorBrewer            1.1-2	RColorBrewer            1.1-2
RcppParallel            5.1.4	RcppParallel            5.1.2 *
RCurl                1.98-1.3	RCurl                1.98-1.3
reshape2                1.4.4	reshape2                1.4.4
rlang                  0.4.11	rlang                  0.4.10 *
Rsamtools               2.6.0	Rsamtools               2.6.0
rstudioapi               0.13	rstudioapi               0.13
S4Vectors              0.28.1	S4Vectors              0.28.1
scales                  1.1.1	scales                  1.1.1
ShortRead              1.48.0	ShortRead              1.48.0
stats4                  4.0.5	stats4                  4.0.3 *
stringi                 1.6.2	stringi                 1.5.3 *
stringr                 1.4.0	stringr                 1.4.0
SummarizedExperiment   1.20.0	SummarizedExperiment   1.20.0
tibble                  3.1.2	tibble                  3.1.1 *
tidyselect              1.1.1	tidyselect              1.1.0 *
tools                   4.0.5	tools                   4.0.3
utf8                    1.2.1	utf8                    1.2.1
vctrs                   0.3.8	vctrs                   0.3.7 *
XVector                0.30.0	XVector                0.30.0
zlibbioc               1.36.0	zlibbioc               1.36.0

Update: How it was fixed

I tried to recreate the "new nonworking environment" by just adding 'r-matrix=1.3-2' to the dependency list; it worked, while previously autoselecting r-matrix=1.3-3 it was not.

I'm shocked by this instability in the dep tree! :)

@bashirhamidi
Copy link

I'm glad people have been able to get their pipelines working again :)

@benjjneb #212 (comment) above from @kalen-rasmussen notes updated dada2 which should have bypassed the Matrix function issue but seemingly is still present... Just want to make you aware of that.

Matrix_1.3-3
dada2_1.19.2

@benjjneb
Copy link
Owner

Update: Fixed
I tried to recreate the "new non working environment" just adding 'r-matrix=1.3-2' and it worked, while r-matrix=1.3-2 was not. I'm shocked by this instability :)

Glad it got fixed! Yeah, it is the same issue with the same fix. Unfortunate that this might be propagated to some Bioconda builds though, which we don't really have control over.

@emankhalaf
Copy link

dada2_1.19.2

I believe I should uninstall dada2_1.19.1 (the one I have currently) first, then install the new version (1.19.2), correct?

@emankhalaf
Copy link

It is working now! Thanks @benjjneb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests