Tasklist: IO Subsystem
Clone this wiki locally
This page details all the necessary improvements to Parrot's I/O subsystem that need to be made. If you work on any of these tasks, make sure you remove it from the list when you're done.
Here are some details of a major refactor proposed by Whiteknight which will probably be pursued soon:
- Refactor IO-related codepaths to use a new
io_vtablefor dispatching operations to the subsystem API
- Refactor out a buffering API, which can take vtable function pointers for type-agnostic buffering on separate input/output streams.
- Start separating out Pipe and FileHandle logic, though do not modify the user-facing interfaces at all.
- Possible create a new Pipe PMC, to expose the new pipe interface without removing those features from FileHandle (yet).
Related TO-DO items
These are some older TODO items which will be covered by this refactor:
- Make it possible to use separate input and output buffering. Input buffering is needed for
peek(), but sockets or pipes usually don't use output buffering.
- Separate pipe-related logic out of
FileHandle. Create a
PipePMC type. Alternatively, we could deprecate opening pipes to external commands via
- Move buffering logic from
Handle. Be able to inherit it from all other PMC types. Include buffering-related
METHOD_buffer_size(). We should keep the buffering logic at a C-level. All that is needed for now is to pass the buffering functions a pair of read/write callbacks for files and sockets (e.g.
send()). (Also, maybe add a flag to the buffer_type and buffer_size methods to specify whether this is for input-only or output-only or both, (default)).
- Modify I/O API so that it doesn't call
PCCINVOKE(). Method calls on I/O PMC's should call the API functions, not the other way around.
- Unify logic wherever possible so multiple I/O types can share.
- Unify the codepaths for
Pipeinto the I/O API. Refactor the I/O API to be more unified.
- Add externally-visible socket functions src/io/socket_*.c to src/io/api.c.
- Extract a sane buffering API.
Parrot_io_putps(). Unify where possible and remove duplicate code.
- Add src/io/pipe.c and src/io/stringhandle.c files for type-specific logic similar to src/io/filehandle.c.
- Make sure the functions in src/io/buffer.c will work with all buffer-enabled PMC's.
Unnecessary TODO Items
Here are a list of previously-mentioned TODO items which become unnecessary in this refactor:
from theopen()` method of each
- Error handling: figure out what happens if an unsuitable PMC is passed to
GET_ATTRmacros in src/io/filehandle.c so all functions will be able to handle subclasses.
- Move encoding logic (
ATTR_encoding(), etc.) from
Handle. Be able to inherit it from all other PMC types. The encoding logic is encapsulated in the string code quite well now, so we should have the
StringHandlePMC handle encodings on its own. Only the encoding attribute should be shared.
Parrot_io_make_offset_pmc(). If they do not need to be in src/io/api.c, move them elsewhere, possibly src/io/filehandle.c.
say()stringify the same way (see http://rt.perl.org/rt3/Ticket/Display.html?id=55196).
%sconversion specification in the
printf()-like functions do not handle null C strings well.
PARROT_NETWORKING_SUPPORT(see TT #534).
Following are various I/O related RT tickets:
- Create a
StreamBufferPMC to abstract away buffering details. This would allow
SocketPMC's to be subclassed more easily and give all I/O types easy access to buffering.
Parrot_io_printf()should output to Parrot's standard output PMC, not
Parrot_io_eprintf()should output to Parrot's standard error PMC, not
- Deprecate the current pipe API and create an
OSProcessPMC that works like this (see IPC::Open3).
proc = new 'OSProcess' # The 'flags' argument is something like EXEC_STDIN | EXEC_STDOUT | EXEC_STDERR proc.'exec'(command, args, flags) # Get stdin of exec()'ed process for writing w_handle = proc.'stdin'() # Get stdout of exec()'ed process for reading r_handle = proc.'stdout'()
- Created an abstract
- Abstracted away relevant API code from
StringHandleto be a proper subclass of
- Removed the layers structures and macros after the migration is complete.
- Removed src/io/io_mmap.c as it's unused and not useful.
- Converted I/O layers to I/O objects.
src/io/io_unix.c is the guts of most I/O on most platforms. isrc/io/io_win32.c is Windows. src/io/io_stdio.c is
STDERR. These three need to be ported to the new system.
src/io/io_utf8.c is really the wrong way to go about implementing UTF-8 encoding.
FilehandlePMC's should be marked with their character set and encoding, similar to strings.
- Created a
FileHandlePMC as a core filehandle object which can be subclassed by various HLL's.
- Continued to support different I/O operations on different platforms, using the
#ifdefdirective on platform-specific sections.
- Renamed all
PIO_*functions to 'Parrot_io_*'. Since the implementation is completely changing, it's better to create new functions with the new names than to change the names of existing functions.
- Removed the deprecated
pioctlopcode and fixed related documentation (see http://rt.perl.org/rt3/Ticket/Display.html?id=48589).
- Decided if we plan to use AIO before the 1.0 release (see http://rt.perl.org/rt3/Ticket/Display.html?id=57920).
- Removed src/io/io_passdown.c and src/io/io_layers.c since they're purely implementation artifacts of the I/O layers implementation.
- Changed src/io/io_string.c to a subclass of
FileHandlethat provides the same interface but to a string instead of a filehandle.
- Changed all I/O PMC's to inherit from
- Added a
close() anddisconnect() method to the
- Added an improved file-like API to the
readline(), etc). Internally, these methods should direct to similar functions in src/io/api.c.
Handleso that it can be inherited by
- Create an
Socketcan inherit from (but not
- Create a
SelectPMC, maybe have a look at libevent. (It's a dynpmc)
StreamDescriptorPMC to abstract away system-dependent I/O descriptors. This would allow
SocketPMC's to be subclassed more easily. (Mostly done as 'Handle' PMC)