Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segfault when writing dataframes with two or more factor columns #47

Closed
LTLA opened this issue Nov 20, 2019 · 2 comments
Closed

Segfault when writing dataframes with two or more factor columns #47

LTLA opened this issue Nov 20, 2019 · 2 comments
Assignees

Comments

@LTLA
Copy link
Contributor

LTLA commented Nov 20, 2019

set.seed(0)
library(rhdf5)
Y <- data.frame(
    X=sample(LETTERS, 20000, replace=TRUE),
    Y=sample(LETTERS, 20000, replace=TRUE)
)
unlink('stuff.h5')
h5write(Y, file="stuff.h5", name='data', DataFrameAsCompound=FALSE)

Running it locally on my mac indicates that memory usage climbs inexorably - as of time of writing, I'm at 3 GB usage in an R session that's stuck on h5write. Running it on Unix on an interactive job with an 8GB memory limit eventually causes a segfault, presumably because all memory is allocated. valgrind indicates that the error occurs in _H5Dwrite.

No problems are observed if there is only one factor column (i.e., either column is converted to a character or integer), the command above completes instantly.

Session information
R version 3.6.1 Patched (2019-10-31 r77366)
Platform: x86_64-apple-darwin17.7.0 (64-bit)
Running under: macOS High Sierra 10.13.6

Matrix products: default
BLAS:   /Users/luna/Software/R/R-3-6-branch/lib/libRblas.dylib
LAPACK: /Users/luna/Software/R/R-3-6-branch/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] rhdf5_2.30.0

loaded via a namespace (and not attached):
[1] compiler_3.6.1 Rhdf5lib_1.8.0
@grimbough grimbough self-assigned this Nov 21, 2019
@grimbough
Copy link
Owner

grimbough commented Nov 21, 2019

Thanks for the report, can confirm I see the same issue on my machine. It also happens if both are converted to character, which is what is happening prior to the write anyway. - I forgot the default for data.frame() is stringsAsFactors = TRUE.

@grimbough
Copy link
Owner

This should now be fixed in both release (2.30.1) and devel (2.31.1). Let me know if it's still causing issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants