universal in-place decompression interface #132
Comments
Moelf
changed the title
|
I've drafted a PR around Buffers to support faster decompression when loading Arrow files. Would that support your use case as well? Or would you need the sink to be ByteData (instead of buffer)? |
mkitti
added a commit
that referenced
this issue
Apr 9, 2023
…transcode (#136) * allow users to optionally provide an output buffer when calling transcode * add tests and refactor docstring * nicer formating Co-authored-by: Mark Kittisopikul <mkitti@users.noreply.github.com> * new copydata! method * reassociate docstring with correct method * Expand ByteData Co-authored-by: Mark Kittisopikul <mkitti@users.noreply.github.com> * Expand ByteData Co-authored-by: Mark Kittisopikul <mkitti@users.noreply.github.com> * Expand ByteData Co-authored-by: Mark Kittisopikul <mkitti@users.noreply.github.com> * clarify formatting * formatting * Generic args Co-authored-by: Joao Aparicio <jpmn.aparicio@gmail.com> * Generic args Co-authored-by: Joao Aparicio <jpmn.aparicio@gmail.com> * small fixes * simplify * fix buffer * fix * formatting * fix buffer * Address PR comment #136 (comment) * Address PR comment #136 (comment) * Address PR review #136 (comment) #136 (comment) * dont use isnothing * add doc-string * replace keepbytes with marginpos * fix typeo --------- Co-authored-by: Mark Kittisopikul <mkitti@users.noreply.github.com> Co-authored-by: Joao Aparicio <jpmn.aparicio@gmail.com>
|
Closed by #132 and released as 0.9.12 |
bauglir
added a commit
to bauglir/TranscodingStreams.jl
that referenced
this issue
Apr 14, 2023
The examples in many of the codec packages rely on being able to call `transcode` providing a `Codec` type and a string[^1][^2]. JuliaIO#132 removed type annotations from the trailing arguments for `transcode(::Type{C}, ...) where {C<:Codec}` causing a method ambiguity with `transcode(T, src::String)` in Julia `Base`[^3]. This adds an additional method to `Base.transcode` to resolve this ambiguity. Fixes JuliaIO#139. [^1]: https://github.com/JuliaIO/CodecZlib.jl/tree/f9fddaa28c093c590a7a93358709df2945306bc7#usage [^2]: https://github.com/JuliaIO/CodecZstd.jl/tree/6327ffa9a3a12fc46d465dcfc8b30bed91cf284b#usage [^3]: https://github.com/JuliaLang/julia/blob/ff7b8eb00bf887f20bf57fb7e53be0070a242c07/base/c.jl#L306
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
by "in-place" I mean the user would pre-allocate a
Vector{UInt8}or something as the sinksometimes decompression is needed for low-level stuff, such as handling "buffers" in a file spec, and multiple decompression together assemble the entire data blob.
it would be nice if there's a in-place interface that support multiple algorithms through this central package.
The text was updated successfully, but these errors were encountered: