Support shared memory reads #316
Conversation
|
This pull request has been linked to Shortcut Story #11819: support shmem buffer reads (as used by the REST server). |
|
Thanks @eddelbuettel ! I'll pull this locally & build & validate with a local REST-server checkout. |
|
I removed the 'draft' label. The earlier reds in CI can be ignored as they come from macOS and Windows which are, effectively, not used in this code -- and earlier changes simply didn't condition as cleanly on Linux-only. |
eddelbuettel
force-pushed
the
de/sc-11819/shmem_reads
branch
from
ddf594e
to
294d9c1
Compare
eddelbuettel
force-pushed
the
de/sc-11819/shmem_reads
branch
from
294d9c1
to
1213a3c
Compare
src/libtiledb.cpp
Outdated
| @@ -158,6 +156,46 @@ tiledb_datatype_t _string_to_tiledb_datatype(std::string typestr) { | |||
| } | |||
| } | |||
|
|
|||
| int32_t _tiledb_datatype_to_sizeof(tiledb_datatype_t dtype) { | |||
There was a problem hiding this comment.
FYI, this is available in the core C API as tiledb_datatype_size - https://github.com/TileDB-Inc/TileDB/blob/a7e1d7cde0fe07c1dbf8fb18243e91cf3b83fcbf/tiledb/sm/c_api/tiledb.cc#L314
(technically not exposed in the C++ API as far as I can tell, but still usable)
There was a problem hiding this comment.
No need to change this now, but just for future reference/refactoring.
There was a problem hiding this comment.
💯
I'll switch to that,
There was a problem hiding this comment.
Done in 95fcb52
|
Thanks @eddelbuettel! |
This PR enables accessing shared memory buffers as e.g. provided by the TileDB Cloud backend. This is not of general use for the TileDB R package which will usually call TileDB Embedded in process but supports a usage pattern, and performance enhancement, in our deployment for TileDB Cloud.
The 'list of buffers' proposal is working fine as we can use simple regular expressions thanks to C++17 to go from (required) data buffers to (optional) offsets and validity_map buffers.