as.data.table.matrix keeps row names for null data tables

[mllg]

Consider this example:

M = matrix(1:3, nrow = 3)
M = M[, integer(0)]

DT = as.data.table(M) # null data table

The resulting object DT is a "Null data.table" with dimensions c(0,0). This deviates from as.data.frame, but this is documented.

> nrow(DT)
[1] 0
> dim(DT)
[1] 0 0

Confusingly, the null data.table still has rownames:

rownames(DT)
[1] "1" "2" "3"

This is problematic, because the dimensions of a data.frame is calculated using the rownames:

> base::dim.data.frame(DT)
[1] 3 0

Of course there is a S3 method dim.data.table which returns c(0, 0). However, you get the wrong dimensions in C/C++ code.

I wonder if there is any advantage in keeping the row names. I first thought they are stored for the conversion of data.tables to data.frames (as.data.frame / setDF), but the row names seem to be ignored here, too.

[sritchie73]

I can't replicate this behavior:

> library(data.table)
data.table 1.11.8  Latest news: r-datatable.com
> M = matrix(1:3, nrow = 3)
> M = M[, integer(0)]
> 
> DT = as.data.table(M) # null data table
> M
    
[1,]
[2,]
[3,]
> dim(M)
[1] 3 0
> as.data.table(M)
Null data.table (0 rows and 0 cols)
> dim(as.data.table(M))
[1] 0 0
> rownames(M)
NULL
> rownames(M) <- 1:3
> rownames(M)
[1] "1" "2" "3"
> dim(as.data.table(M))
[1] 0 0
> nrow(as.data.table(M))
[1] 0

[jaapwalhout]

@sritchie73 You are doing something slightly different then @mllg imo

With data.table 1.11.8 i can replicate this:

> packageVersion("data.table")
[1] ‘1.11.8’

> M  <- matrix(1:3, nrow = 3)
> M <- M[, integer(0)]
> M
    
[1,]
[2,]
[3,]
> rownames(M)
NULL
> 
> DT <- as.data.table(M)
> dim(DT)
[1] 0 0
> rownames(DT)
[1] "1" "2" "3"
> base::dim.data.frame(DT)
[1] 3 0

I think the problem stems from the line of the code of the as.data.table.matrix-method (retreived with getAnywhere(as.data.table.matrix)):

function (x, keep.rownames = FALSE, ...) 
{
    if (!identical(keep.rownames, FALSE)) {
        ans = data.table(rn = rownames(x), x, keep.rownames = FALSE)
        if (is.character(keep.rownames)) 
            setnames(ans, "rn", keep.rownames[1L])
        return(ans)
    }
    d <- dim(x)
    nrows <- d[1L]
    ncols <- d[2L]
    ic <- seq_len(ncols)
    value <- vector("list", ncols)
    if (mode(x) == "character") {
        for (i in ic) value[[i]] <- x[, i]
    }
    else {
        for (i in ic) value[[i]] <- as.vector(x[, i])
    }
    col_labels <- dimnames(x)[[2L]]
    if (length(col_labels) == ncols) {
        if (any(empty <- !nzchar(col_labels))) 
            col_labels[empty] <- paste0("V", ic[empty])
        setattr(value, "names", col_labels)
    }
    else {
        setattr(value, "names", paste0("V", ic))
    }
    setattr(value, "row.names", .set_row_names(nrows))
    setattr(value, "class", c("data.table", "data.frame"))
    alloc.col(value)
}

As you can see, in the third line from the end the rownames are set with the number of rows of the matrix M (which is 3 in this example).

A possible solution might be to set the nrows-parameter in the as.data.table.matrix-method with nrows <- d[1L] * (d[2L] > 0L) instead of just nrows <- d[1L].

[sritchie73]

Wouldn't it make more sense to just grab the number of rows from the new data.table? I.e. change the fourth last line to:

setattr(value, "row.names", .set_row_names(nrow(value)))

[MichaelChirico]

interesting that M does indeed have a shape...

dput(M)
# structure(integer(0), .Dim = c(3L, 0L))

But as data.table is a columnar object, we can tell at this line:

value <- vector("list", ncols)

that the end result is going to be data.table(NULL).

I'm not 100% sure what the "right" result is, but if it's going to be a NULL data.table, might as well just do this:

ncols <- d[2L]
if (!ncols) return(null.data.table())
# ...

[jangorecki]

@MichaelChirico suggestion was the simplest and well addressing the issue. I pushed it in PR and added few more tests for other edge cases so we are more future proof.

[sritchie73]

This problem also arises with as.data.table.data.frame:

> library(data.table)
> mtcars[,0]
data frame with 0 columns and 32 rows
> DF <- mtcars[,0]
> DT <- as.data.table(DF)
> DT
Null data.table (0 rows and 0 cols)
> rownames(DT)
 [1] "1"  "2"  "3"  "4"  "5"  "6"  "7"  "8"  "9"  "10" "11" "12" "13" "14" "15" "16" "17" "18" "19" "20" "21" "22" "23" "24" "25" "26"
[27] "27" "28" "29" "30" "31" "32"