Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BoundsError: attempt to access 0-element Array in DataFrames.dropmissing! with file read with CSV #690

Closed
anj00 opened this issue Jul 16, 2020 · 2 comments · Fixed by JuliaData/SentinelArrays.jl#21
Closed

BoundsError: attempt to access 0-element Array in DataFrames.dropmissing! with file read with CSV #690

anj00 opened this issue Jul 16, 2020 · 2 comments · Fixed by JuliaData/SentinelArrays.jl#21

Comments

@anj00
Copy link

anj00 commented Jul 16, 2020 

using CSV, DataFrames
x = DataFrame(CSV.File(IOBuffer("x,y\n,3\n,4")))
dropmissing!(x, :y)

gives error ERROR: BoundsError: attempt to access 0-element Array{Int64,1} at index [1] . Note, not in place dropmissing works fine.

I am running

versioninfo()
Julia Version 1.4.2
Commit 44fa15b150* (2020-05-23 18:35 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)
......
  [336ed68f] CSV v0.7.3
  [a93c6f00] DataFrames v0.21.4

also reading CSV the old style CSV.read(IOBuffer("x,y\n,3\n,4"), copycols = true) causes same error

Is it me missing some option or CSV (or DataFrames) issue?
@anj00
Copy link
Author

anj00 commented Jul 17, 2020

This code makes things work for me. Basically getting rid of vector types CSV is using now and casting it to a Vector (I understand it goes against the idea of latest release, just a workaround for now),

csv_file = CSV.File(IOBuffer("x,y\n,3\n,4"))
newcols = AbstractVector[]
for col in getfield(csv_file, :columns)
	push!(newcols, Vector{eltype(col)}(col))
end
x = DataFrame!(newcols, getfield(csv_file, :names))
dropmissing!(x, :y)

quinnj added a commit to JuliaData/SentinelArrays.jl that referenced this issue Jul 17, 2020
@quinnj
Fix deleteat! with empty inds array
7fc4b32 
Fixes JuliaData/CSV.jl#690. The issue here is we weren't
testing or accounting for the case when the `inds` arguement to deleteat! is empty.
This comes up in practice from `dropmissing!` in DataFrames.jl when there are no missing
values in a column, it still calls `deleteat!` with the empty array.
The fix is pretty straightforward: we can just remove the bounds checks here because
we're doing the bounds check in the underlying `::Integer` method anyway.
@quinnj quinnj mentioned this issue Jul 17, 2020
Merged
@quinnj
Copy link
Member

quinnj commented Jul 17, 2020

Sorry about this and the delay; I have a fix here: JuliaData/SentinelArrays.jl#21

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix deleteat! with empty inds array JuliaData/SentinelArrays.jl
2 participants
@anj00 @quinnj