-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore(core): growable DirectByteSink in native code via byte_sink.h #3864
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not all binaries are updated.
dc6025c
to
1eddb0e
Compare
core/src/test/java/io/questdb/test/std/str/DirectByteCharSinkTest.java
Outdated
Show resolved
Hide resolved
this does not look like a user facing "feature", PR title should reflect that (chore) |
…ctByteSink and introduced JavaCritical_ JNI method variants. Also fixed MemoryTag.
…nk. Changed ByteSequence size from long to int.
While I don't have a use for it yet, we might want to be able to transfer ownership of the sink to/from Java and native code by leaking/acquiring the impl pointer. E.g.: Transfering ownership from Java to native code. DirectByteSink dbs = ..;
long ptr = dbs.leak(); // pointer can be handed over to native code. Transfering ownership from native code to Java. // some JNI call that constructed the buffer earlier, maybe another thread.
long ptr = ...
DirectByteSink dbs = new DirectByteSink();
dbs.of(ptr); Something to revisit if a need arises :-) |
[PR Coverage check]😍 pass : 113 / 116 (97.41%) file detail
|
Technical enhancements to enable populating a byte sink from native code.
DirectByteCharSink
intoio.questdb.std.bytes.DirectByteSink
.DirectByteSink
implementation by adding a layer of indirection and exposing its core as a C struct calledquestdb_byte_sink
(seebyte_sink.h
).Additionally, this PR comes with:
malloc
minimum).realloc
, a more consistent new capacity calculation that finds the next power of two (see https://stackoverflow.com/questions/64734694/why-do-dynamic-arrays-have-to-geometrically-increase-their-capacity-to-gain-o1 on why this is valuable).io.questdb.std.bytes
package and inside it:interface ByteSequence
byte byteAt(int index)
,int size();
io.questdb.std.str.ByteSequence
, so this name collision is temporary.interface DirectByteSequence extends ByteSequence
long lo()
,long hi()
,long ptr()
interface NativeByteSink extends QuietCloseable
ptr()
returning the aforementionedquestdb_byte_sink*
close()
method to ensure correct memory bookeeping after use.interface BorrowableAsNativeByteSink
NativeByteSink borrowDirectByteSink();
NativeByteSink
class DirectByteSink
(as aforementioned), containing the core logic forDirectByteCharSink
, void of anyCharSink
logic.{Direct}Utf8Sequence
interfaces on top of the new{Direct}ByteSequence
.N.B.:
DirectByteCharSink
is also going to be removed and replaced by another utf-8 sink type in a future PR by @puzpuzpuz.N.B.:
ByteSequence
, for now - as a compromise, has anint
-based length rather thanlong
. This is to allow for easier migration to these new interfaces.