mutate with multiple list columns cause crashes, a selection of errors

[richierocks]

Calling mutate_ with multiple list columns fails using dplyr 0.4.2.

To reproduce:

library(dplyr)

get_random_letters <- function(n)
{
  replicate(n, sample(letters, 100, replace = TRUE), simplify = FALSE)
}

n <- 1e3
d <- mutate_(
  data.frame(id = seq_len(n)),
  x = ~ get_random_letters(n),
  y = ~ get_random_letters(n),
  z = ~ get_random_letters(n) 
)

# OK to here; next line fails

d <- mutate_(
  d,
  x = ~ lapply(x, sort),
  y = ~ lapply(x, sort),
  z = ~ lapply(x, sort)
)

I get a variety of responses, seemingly randomly. In decreasing order of frequency:

• R crashes
• Error: data_frames can only contain 1d atomic vectors and lists
• Error in as.list.data.frame(X) :
  unimplemented type (29) in 'lazy_duplicate'
• As above, but with 11, 12 or 28 as the unimplemented type number.
• Error: Columns are not all same length
• Error in structure(.Call(C_objectSize, x), class = "object_size") : unimplemented type (29) in 'object.size'

Tested under Windows and Linux. It worked fine when only one or two columns were mutated.

Possibly related to #1228

[hadley]

@romainfrancois can you take a look at this too please?

[john-sandall]

I'm seeing a similar problem, I posted on SO (http://stackoverflow.com/questions/31010713/combining-dplyrmutate-with-lubridateymd-hms-in-r-randomly-causes-segfault) but if it's definitely this same bug I'll close the SO post and reference this issue.

I'm seeing the same variety of crashes as @richierocks mentioned, in that same order of frequency, although my issue is occurring with the standard evaluating mutate() function and only when I'm calling ymd_hms from lubridate within the mutation. Sample code to replicate on SO or in this gist: https://gist.github.com/john-sandall/05c3abb24fc738ddc2ad

Update: Spurred by this thread, I uninstalled dplyr 0.4.2 & installed 0.4.1, issue is resolved. Definitely seems to be some kind of bug only present in 0.4.2

[HarlanH]

I'm also seeing hard crashes and weird behavior in 0.4.2, including factors getting converted to numeric. No simple test cases yet...

[amin04]

I second that. Hard crashes & weird behaviour with columns. Had to switch back to 0.4.1.

[daltonhance]

Chiming in here. I'm also getting hard crashes and having Rmarkdown fail to compile when using mutate with a call to lubridate.

%>% mutate(Mark_Time_Value = ymd_hms(Mark_Time_Value),
     Release_Time_Value =  ymd_hms(Release_Time_Value),
     Obs_Time_Value =  ymd_hms(Obs_Time_Value)) 
%>% mutate(TravelTime = difftime(Obs_Time_Value, Release_Time_Value, units = "days" ))

The error also occurred when I performed all these steps in a single mutate call (I changed to see if that would fix it). I thought it was lubridate before finding this thread, because otherwise this pipe and other pipes with mutate calls work just fine.

I'll try switching back to 0.4.1 to see if that fixes.

EDIT: Yep, switching back fixed it.

If it helps I did get this error message on one attempted run of Rmarkdown: unimplemented type (31) in 'duplicate'

[hadley]

No need to add additional examples, we're aware of the problem.

[romainfrancois]

This should be fixed now. @richierocks can you try it please. This was a sneaky protection problem.

[romainfrancois]

@john-sandall, @daltonhance @amin04 this should also fix the things you mention. Please test.

[daltonhance]

@romainfrancois The development version seems to work for me. But I first checked the version available on CRAN as of this morning and ran into some of the same problems.

[richierocks]

@romainfrancois The problem seems to be solved. My code that was broken is now working again, and I haven't been able to reproduce those errors at all.

[romainfrancois]

Nice. Thanks for the heads up. That's a close then. 😎

[lgautier]

That's a pretty serious issue. Is there an ETA for a bugfix release (for example dplyr 0.4.3) ?