veg_distance not available?

[fjlicht]

Recently updated R and now am unable to run previous code. Not sure if it is a vegan issue or an R issue?

> df = as(sample_data(ITS), "data.frame")
> d = distance(ITS, "bray")
Error in .C("veg_distance", x = as.double(x), nr = N, nc = ncol(x), d = double(N *  : 
  "veg_distance" not available for .C() for package "vegan"
> sessionInfo()
R version 3.4.4 (2018-03-15)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.4

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] phyloseq_1.22.3

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.16        compiler_3.4.4      pillar_1.2.1        plyr_1.8.4         
 [5] XVector_0.18.0      iterators_1.0.9     tools_3.4.4         zlibbioc_1.24.0    
 [9] jsonlite_1.5        tibble_1.4.2        nlme_3.1-137        rhdf5_2.22.0       
[13] gtable_0.2.0        lattice_0.20-35     mgcv_1.8-23         pkgconfig_2.0.1    
[17] rlang_0.2.0         Matrix_1.2-14       foreach_1.4.4       igraph_1.2.1       
[21] parallel_3.4.4      stringr_1.3.0       cluster_2.0.7-1     Biostrings_2.46.0  
[25] S4Vectors_0.16.0    IRanges_2.12.0      multtest_2.34.0     stats4_3.4.4       
[29] ade4_1.7-11         grid_3.4.4          Biobase_2.38.0      data.table_1.10.4-3
[33] survival_2.41-3     reshape2_1.4.3      ggplot2_2.2.1       magrittr_1.5       
[37] splines_3.4.4       scales_0.5.0        codetools_0.2-15    MASS_7.3-49        
[41] BiocGenerics_0.24.0 biomformat_1.6.0    permute_0.9-4       ape_5.1            
[45] colorspace_1.3-2    stringi_1.1.7       lazyeval_0.2.1      munsell_0.4.3      
[49] vegan_2.5-1        

[jarioksa]

It looks like being an issue in the package where you have function distance(). We do not have that function in vegan. It also seems that the package having distance() function calls directly compiled code in vegan which it should not have done.

There is currently a similar issue in StackOverflow, and a common denominator is phyloseq package. Perhaps that is where you get the error? If so, please contact phyloseq developers and ask them to upgrade their package.

We tested vegan against dependent packages, and CRAN ran similar tests before accepting vegan, but this problem did not appear there. We would have communicated with that other package had we seen this problem before, but we did not catch this in our tests.

[jarioksa]

Please note that if ITS is a data frame and you want to have dissimilarities for its rows in standard R form, you can use directly vegdist(ITS, "bray") instead of using distance() function from some other package to call vegan function.

[jarioksa]

@joey711 : is this an issue with phyloseq (we now use .Call() interface in vegan for most compiled code, including distances).

[fjlicht]

Thank you Jari, you are correct and I believe it must be a phyloseq issue since it has not been updated through CRAN. I saw the StackOverflow issue and the solution of reloading the vegan package did not resolve the issue. I will try your suggestion of vegdist() to circumvent my issue.

[jarioksa]

This is also reported as an issue in phyloseq as joey711/phyloseq#918

[joey711]

Actually, @jarioksa , phyloseq is just wrapping vegan::vegdist()

https://github.com/joey711/phyloseq/blob/077f904823a6bcd94d0b7607f621d9097230c4a4/R/distance-methods.R#L138

Did something change with vegan::vegdist() ? I'm seeing references to veg_distance, as in the title of this ticket?

[jarioksa]

Now it looks like this is a mismatch between vegan R code and its compiled libraries: for some reason (and I don't know which) you seem to use vegan::vegdist() of version 2.4-x, but your compiled libraries are for version 2.5-1. The compiled code is upgraded to 2.5-1, but the R code comes still from 2.4-x. What happens if you say rm(vegdist)?

[jarioksa]

@joey711 all three current reports have the same feature: they upgraded to vegan_2.5-1 and after that phyloseq commands fail. In two cases they seem to be distance methods, and in one case make_network. What is common here is that compiled library was upgraded to vegan 2.5-1, but R code was not upgraded and 2.4-x code was still used. There was a mismatch: different versions of vegan were used for compiled and R code.

The make_network() case looks simpler to debug: it has this definition:

R/network-methods.R:89:#' ii <- make_network(enterotype, max.dist=0.3, distance=function(x){vegan::vegdist(x, "jaccard")})

Here the distance is defined as a call to vegan::vegdist(), but this call goes to pre-upgrade vegan.

There are similar copies of vegan function in https://github.com/joey711/phyloseq/blob/master/R/distance-methods.R#L110
After this a local copy (dfun) of vegdist() is used instead of updated vegdist(). This copy is made when loading phyloseq, and will not change when vegan changes. There are also two setMethod("vegdist",...) calls in extend_vegan.R, and I have no idea what is the version of vegdist they use if vegan is upgraded during a session.

If this is the case, probably the easiest way of refreshing phyloseq is a fresh start: close R, start a new session and go. Would this work @fjlicht @okayama1 ?

[okayama1]

Thanks a lot for looking into this issue. I have tried that and it doesn't work for me :-(

…

On Wed, Apr 18, 2018 at 3:38 PM, Jari Oksanen ***@***.***> wrote: @joey711 <https://github.com/joey711> all three current reports have the same feature: they upgraded to *vegan*_2.5-1 and after that *phyloseq* commands fail. In two cases they seem to be distance methods, and in one case make_network. What is common here is that compiled library was upgraded to *vegan* 2.5-1, but *R* code was not upgraded, but 2.4-x code was still used. The make_network() case looks simpler to debug: it has this definition: R/network-methods.R:89:#' ii <- make_network(enterotype, max.dist=0.3, distance=function(x){vegan::vegdist(x, "jaccard")}) Here the distance is defined as a call to vegan::vegdist(), but this call goes to pre-upgrade *vegan*. There are similar copies of *vegan* function in https://github.com/joey711/phyloseq/blob/master/R/distance-methods.R#L110 After this a local copy (dfun) of vegdist() is used instead of updated vegdist(). This copy is made when loading *phyloseq*, and will not change when *vegan* changes. There are also two setMethod("vegdist",...) calls in extend_vegan.R, and I have no idea what is the version of vegdist they are used if *vegan* is upgraded during a session. If this is the case, probably the easiest way of refreshing *phyloseq* is a fresh start: close *R*, start a new session and go. Would this work @fjlicht <https://github.com/fjlicht> @okayama1 <https://github.com/okayama1> ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#272 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGbowQxZu1BCA7eyRzlBY-5M8hxO8OiOks5tptFbgaJpZM4TY5eG> .

[gavinsimpson]

@okayama1 Just to confirm; close all R sessions, including those in RStudio or other IDEs you are using. Start a clean session; update/reinstall vegan and phyloseq; load the packages and try to run the code causing the problem. Be sure to start clean R sessions — ideally invoke R with --vanilla.

[okayama1]

I have now tried to do the same from the console using R --vanilla and it is not working not sure why ...

…

On Wed., 18 Apr. 2018, 4:17 pm Gavin Simpson, ***@***.***> wrote: @okayama1 <https://github.com/okayama1> Just to confirm; close all R sessions, including those in RStudio or other IDEs you are using. Start a clean session; update/reinstall *vegan* and *phyloseq*; load the packages and try to run the code causing the problem. Be sure to start clean R sessions — ideally invoke R with --vanilla. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#272 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGbowXXpbFGh1MIpIreKIhrUpizMujwGks5tptpugaJpZM4TY5eG> .

[jarioksa]

I tried this at work where I have phyloseq installed on my desktop. Funnily, there all @okayama1 examples worked with vegan 2.5-1 (and newer). However, when I installed vegan 2.4-6, the very same examples failed wit message

> bray_dist <- phyloseq::distance(gp, "bray")
Error in vegdist(c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 27, 100, 130, 1,  : 
  object 'do_vegdist' not found

Looking at the path to this error we see:

> traceback()
... [whole lotta rubbish] ...
6: do.call(dfun, fun.args)
5: do.call(dfun, fun.args)
4: distance(otu_table(physeq), method, type, ...)
3: distance(otu_table(physeq), method, type, ...)
2: phyloseq::distance(gp, "bray")
1: phyloseq::distance(gp, "bray")

So it is the dfun setting of distance-methods.R line 110, and it seems to set the contents of dfun when installing phyloseq: in my case I had installed phyloseq with vegan 2.5-0 and it worked there, but failed when I downgraded to vegan 2.4-6. Further, because dependence checking re-installs checked packages, dfun will have correct contents for checks. However, the contents of dfun will not be updated when vegan is updated.

Conclusion: re-install phyloseq after updating vegan to 2.5-1.

This will work if you install phyloseq from the source. I have no idea what happens in Windows or macOS if you install phyloseq from binary package. It is possible that they may have the version of vegdist of their build time. If binary builds have hard-coded vegan 2.4-x version of vegdist, you just must use vegan 2.4-6 and switch to 2.5-1 as soon as these binary packages switch to vegan 2.5-1. Curious to know how it works.

[jarioksa]

The phyloseq package must be used with the same version of vegan that was used when phyloseq was build and installed. If vegan is version 2.5-x and phyloseq was built for older version 2.4-x, you get an error message

Error in .C("veg_distance", x = as.double(x), nr = N, nc = ncol(x), d = double(N *  : 
  "veg_distance" not available for .C() for package "vegan"

If vegan is version 2.4-x and phyloseq was built for newer version 2.5-x, you get an error message

Error in vegdist(c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 27, 100, 130, 1,  : 
  object 'do_vegdist' not found

You must take care that phyloseq and vegan versions are consistent. You can either try with re-installing phyloseq or installing a suitable vegan version (which means either upgrading to 2.5-x or downgrading to 2.4-6 depending on your phyloseq). The vegan version used by phyloseq is fixed when a binary version of phyloseq was built. If you are installing phyloseq from source files, re-installing phyloseq will always use the installed version of vegan. If you installing phyloseq from binary packages (typically in Windows and macOS), you are stuck with the vegan version that was built in phyloseq.

Please note that the problems occur only when you use phyloseq commands -- vegan commands will work normally.

This is a transitional problem which will appear when only one of phyloseq or vegan is updated, and will disappear with time when both of these packages are updated.

[jarioksa]

See also joey711/phyloseq#918 and joey711/phyloseq#921 for this issue.