replacing dimnames with identical values demotes >2-dim DelayedArray derivatives into DelayedArray

[Liubuntu]

Hi @hpages ,

To continue with the slack conversion, I'm opening an issue here for better track.

In my case of GDSArray, if already a 2-dimensional GDSMatrix, replacing the dimnames with identical values is expected as no-op. However, for >2-dimensional GDSArray, replacing dimnames[1:2] with identical values demotes the object into DelayedDimnames.

library(SeqArray)
library(GDSArray)

gds <- SeqArray::seqExampleFileName("gds")
ga1 <- GDSArray(gds, "genotype/data")
ga2 <- GDSArray(gds, "phase/data")

names(dimnames(ga2))
## [1] "variant.id" "sample.id"                                                                                                                                                                                                                              
names(dimnames(ga1))
## [1] "variant.id" "sample.id"  "ploidy.id"                                                                                                                                                                                                                 
identical(dimnames(ga1)[1:2], dimnames(ga2))
## [1] TRUE                                                                                                                                                                                                                                                  

dnames <- dimnames(ga2)
dimnames(ga2) <- dnames
is(ga2, "GDSMatrix")
## [1] TRUE                                                                                                                                                                                                                                                  
str(ga2)
## no-op                                                                                                                                                                                                                                                     

dimnames(ga1)[1:2] <- dnames  ## demoted as "DelayedDimnames", not a no-op.                                                                                                                                                                                  
is(ga1, "GDSArray")
## [1] FALSE                                                                                                                                                                                                                                                 
str(ga1)
## Demoted into "DelayedDimnames"                                                                                                                                                                                                                            

I was mainly curious about this performance. In your development of DelayedArray, was it something expected? Or are you also expecting no-op for this kind of operation?

The question comes from the assay,SummarizedExperiment method, when I put the 3-dimensional GDSArray into the @assay slot, each time calling of assays(se) will demotes the object inside @assays into DelayedArray (internally as DelayedDimnames).

!> selectMethod(assays, "SummarizedExperiment")
 Method Definition:

 function (x, ..., withDimnames = TRUE)
 {
     assays <- as(x@assays, "SimpleList")
     if (withDimnames) {
         assays <- endoapply(assays, function(assay) {
             dimnames(assay)[1:2] <- dimnames(x)
             assay
         })
     }
     assays
 }

For the source of this question from my package VariantExperiment, It doesn't really matter to check is(, "GDSArray") or is(, "DelayedArray"), and I also think your idea for an updated show method would be nice. Like your example in slack:

<6 x 2 x 90354753> DelayedArray object of type "list" with a seed of class GDSArraySeed: 

Thanks,
Qian

[hpages]

Ok I see now what the problem is. Thanks for the detailed report. Note that even something as basic as dimnames(ga1) <- dimnames(ga1) is not a no-op, which is not right. I think the issue is related to the fact that the dimensions are named i.e. that names(dimnames(ga1)) is not NULL. Will work on this.

[hpages]

This should be fixed in DelayedArray 0.11.8.

Note that the following is quite unexpected:

library(GDSArray)
gds <- SeqArray::seqExampleFileName("gds")
ga1 <- GDSArray(gds, "genotype/data")
sapply(dimnames(ga1), class)
#  variant.id   sample.id   ploidy.id 
# "character" "character"   "integer" 

A very strong expectation is that the dimnames along the dimensions of any array-like object are either a NULL or a character vector. Not sticking to that is risky and has the potential to break code that relies on it.

Actually this is what was preventing dimnames(ga1) <- dimnames(ga1) from being a no-op so I accommodated the code in DelayedArray to be robust against this non-standard behavior of dimnames(). However I would still strongly encourage you to fix this in the GDSArray package.

[Liubuntu]

Thanks @hpages for the quick fix. With your update on DelayedArray, the above example performs as expected no-op for replacing with identical dimnames.

to follow your guidelines in having dimnames for DelayedArray derivatives, I have just updated GDSArray to have all dimnames in character vectors. Bioconductor/GDSArray@a9b5d78

The VariantExperiment package building error was also automatically fixed. Thanks!

[hpages]

Excellent! I've also tried to improve a little bit the doc in DelayedArray to emphasize the importance for dimnames() methods to return the dimnames in standard form:

DelayedArray/vignettes/02-Implementing_a_backend.Rmd

Lines 142 to 152 in 81f5de8

	If the ADS format allows storage of the dimnames, make sure that `dimnames()`
	always returns them in the _standard form_, that is:
	- The dimnames must be returned as a `NULL` (if the dataset has no dimnames)
	or as an ordinary list with one list element per dimension in the dataset.
	- Each element in the returned list is either `NULL` or a character vector
	of length the extend of the dataset along the corresponding dimension.
	It is particularly important to make sure that the vectors in the list
	returned by `dimnames()` are character vectors. Other types like factors
	or integer vectors are not allowed and _will_ break downstream code.

We can close this now.