Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
Allowing to modify Buffers in place #6538
Original bug ID: 6538
We currently have "mutable strings with fixed size" (Bytes) and "expandable but otherwise immutable strings" (Buffer). It would be useful to expose a "mutable and expandable string" data structure to users, and the simplest way to do it is probably to extend the Buffer API with some more "blit" functions (from strings, bytes, and perhaps another buffer). Or is there any conceptual reason to enforce that Buffer is append-only?
Comment author: gerd
In Ocamlnet I'm using a derivative of Buffer called Netbuffer, allowing in-place modification, even with direct access to the underlying string (NB. this module is not yet ported to bytes):
In practice, it turns out that Netbuffer.delete is by far the most important addition (for the purposes in Ocamlnet). Usually data at the beginning of the buffer is deleted, which is handy for stream-based data processors that append new data at the end of the buffer and remove processed data at the beginning. (Note, however, that there are better data structures for this use case, in particular page-aligned buffers, e.g. http://projects.camlcity.org/projects/dl/ocamlnet-4.0.4/doc/html-main/Netpagebuffer.html).
Netbuffer also tries to address one of the peculiarities of Buffer, namely that Buffer defines I/O functions. The point is that you want direct access to the underlying string for I/O, as this avoids another copy of the data. In Netbuffer I added a function add_inplace exposing the data buffer, e.g. you can do
Netbuffer.add_inplace buf Pervasives.input
This way buffering and I/O are again orthogonal in the API. This price to pay is that the buffer details are exposed in the API (and you are bound to it forever).
My experience is that inserting and overwriting strings is not really important. There is usually a good alternative that rebuilds the whole buffer in the way needed, and this alternative is often not slower. I'd consider this more as comfort feature.
Comment author: @c-cube
I like the idea of making Buffer more powerful. It should be a more powerful variant of Bytes.t, with indexing, concatenation, etc. and, indeed, some access to the underlying array for efficient IO.
An alternative is to give a set of low-level access primitives (say, export the type in a module BufferInternals or something, and let libraries write their own overlay). Or, provide
This issue has been open one year with no activity. Consequently, it is being marked with the "stale" label. What this means is that the issue will be automatically closed in 30 days unless more comments are added or the "stale" label is removed. Comments that provide new information on the issue are especially welcome: is it still reproducible? did it appear in other contexts? how critical is it? etc.