setDT error when list is too big

[chuk-yong]

First I created a list of 100 elements with each element containing 3 words:
`> w100 <- topfeat_first10M[1:100]

str(w100)
List of 100
$ : chr [1:3] "thanks" "for" "the"
$ : chr [1:3] "one" "of" "the"
$ : chr [1:3] "a" "lot" "of"
$ : chr [1:3] "i" "want" "to"
$ : chr [1:3] "to" "be" "a"
$ : chr [1:3] "going" "to" "be"`

then I converted this list using setDT. It returned a dt with 3 columns which was exactly what I wanted:
`> w100DT <- transpose(setDT(w100))

str(w100DT)
Classes ‘data.table’ and 'data.frame': 100 obs. of 3 variables:
$ V1: chr "thanks" "one" "a" "i" ...
$ V2: chr "for" "of" "lot" "want" ...
$ V3: chr "the" "the" "of" "to" ...

• attr(*, ".internal.selfref")= `

Next, I created a list with 1million elements and ran the same instruction, but I got this:
`> w1M <- topfeat_first10M[1:1000000]

w1MDT <- transpose(setDT(w1M))
Error in setDT(w1M) :
All elements in argument 'x' to 'setDT' must be of same length`

What's happening? Memory issue? Anyway to get around it?

[MichaelChirico]

look at table(lengths(w1M))

[chuk-yong]

`> table(lengths(w1M))

 3      4      5      6      9 

998543 1365 87 4 1 `

thanks! and I wonder why I ended up with elements with length other than 3.