Finalizers cause exception_access_violation while using HDF5 from a thread. #990
Comments
|
I guess the solution here is to close all properties objects when they are done? |
- Clarifies and adds docs - Closes properties after completion (helps JuliaIO#990) - Removes custom exception for create_group (fixes JuliaIO#829)
- Clarifies and adds docs - Closes properties after completion (helps JuliaIO#990) - Removes custom exception for create_group (fixes JuliaIO#829)
- Clarifies and adds docs - Closes properties after completion (helps JuliaIO#990) - Removes custom exception for create_group (fixes JuliaIO#829)
- Clarifies and adds docs - Closes properties after completion (helps JuliaIO#990) - Removes custom exception for create_group (fixes JuliaIO#829)
* Cleanup group code - Clarifies and adds docs - Closes properties after completion (helps #990) - Removes custom exception for create_group (fixes #829) * Update src/groups.jl Co-authored-by: Mark Kittisopikul <mkitti@users.noreply.github.com> * add deprecation Co-authored-by: Mark Kittisopikul <mkitti@users.noreply.github.com>
|
Following up on #1013 (comment), how about we define a function function with(fn, args...)
try
fn(args...)
finally
for arg in args
finalize(arg)
end
end
endso e.g. Lines 24 to 35 in 6022be0 could be written as function create_group(parent::Union{File,Group}, path::AbstractString; pv...)
with(_link_properties(path), GroupCreateProperties()) do lcpl, gcpl
pv = setproperties!(lcpl, gcpl; pv...)
isempty(pv) || error("invalid keyword options $pv")
return create_group(parent, path, lcpl, gcpl)
end
endor perhaps a macro to do the same, e.g.: function create_group(parent::Union{File,Group}, path::AbstractString; pv...)
@with lcpl=_link_properties(path) gcpl=GroupCreateProperties() begin
pv = setproperties!(lcpl, gcpl; pv...)
isempty(pv) || error("invalid keyword options $pv")
return create_group(parent, path, lcpl, gcpl)
end
end |
|
I was thinking something similar, but calling |
|
@simonbyrne that sounds a lot better than the current setup and remind me of something similar in the D language for finalizers. The function form seems simpler than going with a macro, unless there is an intrinsic benefit of the macro. |
|
Does calling finalize run the finalizers on the current task/thread or does it schedule it on another task/thread? |
|
I'm pretty sure it runs on the current task (i.e. it calls it immediately). julia> mutable struct Foo
end
julia> close(f::Foo) = println("done")
close (generic function with 1 method)
julia> f = Foo()
Foo()
julia> finalizer(close, f)
Foo()
julia> close(f)
done
julia> f = nothing; GC.gc() # close called again
done
julia> f = Foo()
Foo()
julia> finalizer(close, f)
Foo()
julia> finalize(f)
done
julia> f = nothing; GC.gc() # close not called again |
|
|
This seems more general than HDF5.jl. While I'm happy to experiment with it here, perhaps we should consider putting it in a small package at some point? Maybe we should incubate it as a subdirectory package or at least put it within it's own module. The |
|
Closed by #1021 |
When running long jobs, I'm occasionally running into this traces like this. What I think is happening is that the finalizer is running to close out a property while another thread is trying to access the HDF5 library. This can be be mitigated by running
GC.gc()when between jobs where threads are used.Note I'm doing this while using HDF5.jl on a single spawned thread.
The text was updated successfully, but these errors were encountered: