New CRAN release

[jarioksa]

Current CRAN release fails in R-devel (to be R 4.0.0), and CRAN maintainers have requested us to release a new version of Hmsc by December 18, 2019. I have fixed the R-devel issues, and from that point of view, Hmsc is ready for the release. However, some fine tuning is needed, and there also is a possibility of some quick last minute fixes. However, I think we should move early next week to have some buffer for the CRAN deadline.

Primary problem: Class of matrix

There were numerous places where the code assumed that an object belongs to only one class, but it can have an inheritance sequence of several classes. In the he future version of R, the class of matrix object will have such an inheritance and have two classes c("matrix", "array"). In general, a class should not be queried using if(class(x) == "foo"), but using if(inherits(x, "foo")). I have changed all class() queries to that idiom, or used specific query functions is.matrix(), is.data.frame(), is.factor(). Most of the CRAN failures come from constructions switch(class(x), "matrix" = {...}, "data.frame" = {...}) which fails if x is of class c("matrix", "array"). I have made the minimal change: explicitly use only the first class: switch(class(x)[1L], ...). I think most of these cases would be better as if ... else if ... constructs, because there are usually only two alternatives (and no error catching default or else): switch() is great for multiple choices, but for two choices if ... else if ... is more readable. Specifically, I would like the construct

if (is.matrix(x)) {...}
else if (inherits(x, "list") {...} # or provide specific test for a *list of matrices*
else {stop("coding error: contact developers")}

However, switch() is correct, and I have not fixed it because it was not broken.

NEWS

I have added a file inst/NEWS.md for user-visible changes in release versions. This should be checked and updated (and approved by you).

Running time of CRAN tests

The first CRAN release had some delays because it was put under manual inspection. While this is normal for new packages, and things may be easier for updated versions, there was one point that was caught in CRAN tests: running example(sampleMcmc) took nearly 10 sec, and we had to explain that it is needed. We had bad luck, and a slow computer was used in CRAN test farm: the test took ca. 8 sec, but even in my tiny MacBook Air it runs in 6 sec, but even that is above the 5 sec limit of passed test. So we need to cut down to samples=50 even if that feels bad (with a warning comment) – setting nParallell=2, nChains=2 could also help (and even verbose = FALSE helps a bit).

Alternative README for the CRAN release

The current README.md mainly serves the github server. However, it will also be copied to the README in CRAN, but there some features are less useful or non-functional (such as download statistics, or instructions of github installs). If we provide an alternative README in the inst/ folder, it will copied over the current README in the package and in CRAN, and we could have different versions for CRAN and github. That new inst/README.md is minimally the current one stripped of github specifics. I think it might be better to have something similar as @MelindadeJonge already had, and that was largely incorporated in the package help (man/Hmsc-package.Rd).

Version number

The first release version was 3.0-2: we submitted first 3.0-0, but we had to increase the patch level at each resubmission. The current github version is 3.0-4, and I think we should aim at releasing at that number, and the documentation should reflect this. Personally, I don't see much reason to change a version number without a release (releases have meaningful numbers, and they are tagged in git, but for other changes the version numbers have little use), but I don't see large problem in having discontinuous numbering in CRAN either – as long as we make clear to users, that there are no release between those numbers.

[dvrbts]

If at all possible it would be advisable to reduce the number of packages required. I have been running Hmsc on a server for a week now on multiple data sets when gelman.diag generated an error. In response I tried to install the latest version from GitHub, which failed due to the lack of libMagick. Installing from CRAN gives the same result. I do not have permission to install libraries on this server, so I cannot install a new version of Hmsc. In addition, the install.packages('HMSC-master',repos=NULL) command uninstalled the old version so now I cannot run any version of Hmsc on this machine. I will have to try and remember how I shoehorned the code last time to try and reinstall it. If you could move magick to "suggests" it would be a big benefit to many of I am guessing.

[dvrbts]

FWIW, eliminating all references to phytools in DESCRIPTION and NAMESPACE allowed me to reinstall Hmsc. SInce I don't need phytools for my data sets, all is good (I hope).

[jarioksa]

@dvrbts I agree, I think there are far too many dependencies with their chained depencies. Indeed, also in my own system, the real pain was phytools because it depends on animation and that needs external software that needs be found and installed by some other means (ImageMagick, libMagick etc depending on your system). I also spent pretty long time with finding and installing ImageMagick libraries. The problem with these chains is that while some function in phytools needs animation, we do not need that function in Hmsc. and could quite well do without. However, pruning dependencies needs more time than we have for this release.

[dvrbts]

Well, based on my experience so far you could move phytools to "suggests" with a warning to users that want phylogenetic analysis that they need Imagemagick (which is a wonderful piece of software in its own right). Then, those users just wanting environmental, spatial, and latent vectors could proceed. Obviously the timing is quite short, so I fully understand, but maybe in the future some elements of Hmsc could be elective instead of monolithic.

…

On 12/12/19 4:28 PM, Jari Oksanen wrote: @dvrbts <https://github.com/dvrbts> I agree, I think there are far too many dependencies with their chained depencies. Indeed, also in my own system, the real pain was *phytools* because it depends on *animation* and that needs external software that needs be found and installed by some other means (ImageMagick, libMagick etc depending on your system). I also spent pretty long time with finding and installing ImageMagick libraries. The problem with these chains is that while some function in *phytools* needs *animation*, we do not need that function in *Hmsc*. and could quite well do without. However, pruning dependencies needs more time than we have for this release. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#34?email_source=notifications&email_token=ADZXMMXYITLF7H5T2TLCT33QYJKCFA5CNFSM4JZNDBJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGXA3IY#issuecomment-565054883>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADZXMMTGN6DDIDT4OGQOWS3QYJKCFANCNFSM4JZNDBJQ>.

-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ David W. Roberts Professor Department of Ecology, Montana State University On sabbatical at: Swiss Federal Research Institute WSL, Zuercherstrasse 111 CH-8903 Birmensdorf, Switzerland

[dvrbts]

The Hmsc output object is already quite complex, but maybe "Call" could be added to the attributes?

[jarioksa]

I agree with @dvrbts : the advantage of having a Call attribute is that you can easily modify an Hmsc object using update(<oldobject>, <newarguments>). This is also a safe thing to add, because it changes nothing of the old usage (but allows new usage with update()).

[jarioksa]

I have now finished tweaking the package for the release. Deadline was Wed 18-Dec-2019, and I think we could see if we are ready to submit on Monday to have some time buffer.

We have now done all those things that I outlined in the initial posting. The only major thing was handling the matrix class issue of R-devel. Also the smaller things were implemented: there is now a separate inst/README.md for CRAN, and example(sampleMcmc) was cut down to 20 samples to avoid CRAN NOTEs. There are some regular bug fixes which you may check in inst/NEWS.md, and a couple of ideas that cropped out in this thread:

@taddallas made pull request #36 to drop phytools and I merged it in this version. Most importantly, this removes the need of the external ImageMagick software (which also has been a problem in some travis continuous integration checks).

@dvrbts suggested adding call argument in Hmsc. This is now done. It only adds one scalar element in the object, and if you do not need it, it does no harm. However, it allows easier modification of Hmsc models. The example of Hmsc could now be written alternatively as:

     # Creating a Hmsc object without phylogeny, trait data or random levels
     m = Hmsc(Y=TD$Y, XData=TD$X, XFormula=~x1+x2)
     
     # Creating a Hmsc object with phylogeny and traits
     m = update(m, TrData=TD$Tr, TrFormula=~T1+T2, phyloTree=TD$phylo)

I did not modify the example, though.

Please check what we get now so that we can see if we are ready to release on Monday.

[jarioksa]

Version 3.0-4 is now in CRAN.

We did not schedule this release, but we had to do it because of issues in CRAN tests with R-devel. Therefore several issues of our own were left for later releases that we hopefully can do with our own timetable.