Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Fetching contributors…

Cannot retrieve contributors at this time

1119 lines (704 sloc) 28.403 kB

Title

DRAFT: Synopsis 32: Setting Library - IO

Version

 Author:        Rod Adams <rod@rodadams.net>, the authors of the related Perl 5 docs.
 Maintainer:    Larry Wall <larry@wall.org>
 Contributions: Aaron Sherman <ajs@ajs.com>
                Mark Stosberg <mark@summersault.com>
                Carl Mäsak <cmasak@gmail.com>
                Moritz Lenz <moritz@faui2k3.org>
                Tim Nelson <wayland@wayland.id.au>
                Daniel Ruoso <daniel@ruoso.com>
 Date:          19 Feb 2009 extracted from S29-functions.pod; added stuff from S16-IO later
 Last Modified: 23 Feb 2009
 Version:       2

The document is a draft.

If you read the HTML version, it is generated from the pod in the pugs repository under /docs/Perl6/Spec/S32-setting-library/IO.pod so edit it there in the SVN repository if you would like to make changes.

IO

getc
    our Bool method getc (IO $self: *@LIST)

See below for details.

print
    our Bool method print (IO $self: *@LIST)
    our Bool multi print (*@LIST)
    our Bool method print (Str $self: IO $io)

See below for details.

say
    our Bool method say (IO $self: *@LIST)
    our Bool multi say (*@LIST)
    our Bool method say (Str $self: IO $io)

See below for details.

printf
    our Bool method printf (IO $self: Str $fmt, *@LIST)
    our Bool multi printf (Str $fmt, *@LIST)

See below for details.

uri
    IO::Streamable method uri(Str $uri);

Returns an appropriate IO::Streamable descendant, with the type depending on the uri passed in. Here's the mapping:

 URI type IO type
 ======== =======
 file:    IO::File or IO::Directory
 ftp:     IO::Socket::TCP (data channel)
 http:    IO::Socket::TCP

IO::Streamable method uri(Str $uri) { $uri.match(/^<alnum>+\:/); return(&$PROTOCOLS{$1}($uri)); }

%PROTOCOLS global variable

For each protocol, stores a subroutine reference that returns the appropriate object when the URI is passed in.

Roles

The functionality of IO objects is broken down into several roles, which should identify the features each object supports.

IO

The base role only tags that this is an IO object for more generic purposes. It doesn't specify any methods or attributes.

IO::Readable

This role provides unbuffered read access to the data stream.

role IO::Readable { has $.isReadable;

        method Int read($buf is rw, Int $bytes)
}

When the $.isReadable is set, it tries to change the readability of the filehandle. This is not always possible, but can be done in a number of cases. IO::Socket can remove readability by calling shutdown(), for example.

method Int read($buf is rw, Int $bytes)

Tries to read $bytes bytes and store in $buf. The contents of $buf are replaced and the actual number of bytes read is returned. A return of 0 means end of file. It might return unthrown failures, to be specified by each IO implementation.

It is important to realize that this is "raw" read. You're going to have plain octets stored in $buf, if this is actually encoded data, you're going to need to encode it later, or use "getc" or other IO::Readable::Encoded methods.

IO::Writeable

This role provides unbuffered write access to the data stream.

role IO::Writeable { has $.isWriteable;

        method Int write($buf, Int $bytes)
}

When the $.isWriteable is set, it tries to change the writeability of the filehandle. This is not always possible, but can be done in a number of cases. IO::Socket can remove writeability by calling shutdown(), for example.

method Int write($buf, Int $bytes)

Tries to write $bytes bytes of $buf. The actual number of bytes written is returned. It might return unthrown failures, to be specified by each IO implementation.

It is important to realize that this is "raw" write. $buf should contain plain octets that are going to be sent. If $buf contains encoded data, you should decode it first, or use "print" or other IO::Writeable::Encoded methods.

IO::Seekable

method Bool eoi()

EOI = End Of Input -- equivalent to End Of File, but applies to other kinds of sockets as well.

Returns true if it's the end of the input (ie. end of file or whatever), returns false if not, returns undef if we can't say for certain.

method Bool seek(Int $position)

Position this stream into $position. The meaning of this position is always in "octets".

method Int tell()

Returns the current raw position in the stream in number of "octets".

IO::Buffered

Indicates that this object performs buffering. The management of the buffer is completely implementation specific.

method Bool flush()

Flushes the buffers associated with this object.

method Bool autoflush() is rw

Forces this object to keep its buffers empty

If set to nonzero, forces a flush right away and after every write or print on the currently selected output channel. Default is 0 (regardless of whether the channel is really buffered by the system or not; $OUT_FH.autoflush tells you only whether you've asked Perl explicitly to flush after each write). $*OUT will typically be line buffered if output is to the terminal and block buffered otherwise. Setting this variable is useful primarily when you are outputting to a pipe or socket, such as when you are running a Perl program under rsh and want to see the output as it's happening. This has no effect on input buffering.

IO::Streamable

This role represents objects that depend on some external resource, which means that data might not be available at request.

new()
 method IO::Streamable new(
        Bool :$NoOpen,
 );

Unless the NoOpen option is passed, an open will be done on the IO object when it is created.

method Bool blocking() is rw

This allows the user to control wether this object should do a blocking wait or immediatly return in the case of not having data available.

uri
 method IO::Streamable uri(Str $uri) {...}

This should be callable on the class, and act like a kind of "new()" function. When given a URI, it returns an IO::Streamable of the appropriate type, and throws an error when an inappropriate type is passed in. For example, calling IO::File.uri('http://....') will throw an error (but will suggest using just uri('http://...') instead).

IO::Encoded

This is a generic role for encoded data streams.

method Str encoding() is rw
method Str locale() is rw

Encoding and locale are required for sane conversions.

IO::Readable::Encoded

This role provides encoded access to a readable data stream, implies IO::Encoded. Might imply IO::Buffered, but that's not a requirement.

method Int input_record_count()

Returns a count of the number of records (lines?) that have been input. Now with cleaned-up localization usage.

method Str input_record_separator() is rw

This regulates how "readline" behaves.

The input record separator, newline by default. This influences Perl's idea of what a ``line'' is. Works like awk's RS variable, including treating empty lines as a terminator if set to the null string. (An empty line cannot contain any spaces or tabs.) You may set it to a multi-character string to match a multi-character terminator, or to undef to read through the end of file. Setting it to "\n\n" means something slightly different than setting to "", if the file contains consecutive empty lines. Setting to "" will treat two or more consecutive empty lines as a single empty line. Setting to "\n\n" will blindly assume that the next input character belongs to the next paragraph, even if it's a newline.

Remember: the value of input_record_separator is a string, not a regex. awk has to be better for something. :-)

method Str input_field_separator() is rw

This regulates how "readfield" behaves.

method Str input_escape() is rw

This allows the definition of a escape character, which should be used by readline and readfield.

method Str readline()

Reads the stream before it finds a $.input_record_separator and returns it (including the separator). If $.input_escape is set, it should pay attention to that.

method Str readfield()

Reads the stream before it finds a $.input_field_separator and returns it (including the separator). If a readfield finds a $.input_record_separator it consumes the record separator, but returns undef. If $.input_escape is set, it should pay attention to that.

method Str getc(Int $char? = 1)

Reads the next $char character in the set $.encoding according to the $.locale, or the undefined value at end of file, or if there was an error (in the latter case $! is set).

IO::Writeable::Encoded

This role provides encoded access to a writeable data stream, implies IO::Encoded. Might imply IO::Buffered, but that's not a requirement.

If these are called in their non-object form, they operate on $*OUT, except in the case of warn(), which operates on $*ERR. The form with leading dot prints $_ to the appropriate handle unless $_ happens to be a filehandle.

method Int output_record_count()

Returns a count of the number of records (lines?) that have been output.

method Str output_record_separator() is rw

This regulates how say and print(%hash) behaves.

method Str output_field_separator() is rw

This regulates how print(@arr), say(@arr), print(%hash) and say(%hash) behave.

method Str output_escape() is rw

This allows the definition of a escape character, which should be used by say and print to preserve the record/field semantics.

method Bool print(Str $str)
method Bool say(Str $str)

Sends $str to the data stream doing proper encoding conversions. Say sends an additional $.output_record_separator. This should also convert "\n" to the desired $.output_record_separator.

method Bool print(Array @arr)
method Bool say(Array @arr)

Sends each element of @arr separated by $.output_field_separator. Say should add an additional $.output_record_separator. If an element contains the $.output_record_separator or the $.output_field_seaparator and a $.output_escape is defined, it should do the escaping.

method Bool print(Hash %hash)
method Bool say(Hash %hash)

Sends each pair of the hash separated by $.output_record_separator, with key and value separated by $.output_field_separator. If one of those contains a $.output_record_separator or a $.output_field_seaparator and $.output_escape is set, it should do the escaping.

our Bool method print (IO $self: *@LIST)
our Bool multi print (*@LIST)
our Bool method print (Str $self: IO $io)

Prints a string or a list of strings. Returns Bool::True if successful, Failure otherwise. The IO handle, if supplied, must be an object that supports I/O. Indirect objects in Perl 6 must always be followed by a colon, and any indirect object more complicated than a variable should be put into parentheses.

It is a compiler error to use a bare print without arguments. (However, it's fine if you have an explicit argument list that evaluates to the empty list at runtime.)

say our Bool method say (IO $self: *@LIST)
our Bool multi say (*@LIST)
our Bool method say (Str $self: IO $io)

This is identical to print() except that it auto-appends a newline after the final argument.

    Was:    print "Hello, world!\n";
    Now:    say   "Hello, world!";

As with print, it is a compiler error to use a bare say without arguments.

our Bool method printf (IO $self: Str $fmt, *@LIST)
our Bool multi printf (Str $fmt, *@LIST)

The function form works as in Perl 5 and always prints to $*OUT. The method form uses IO handles, not formats, as objects.

IO::Closeable

This role indicates that this object can be closed.

method Bool close()

Closes the file or pipe associated with the object.

Returns True on success, but might return an unthrown failure. Returns true only if IO buffers are successfully flushed and closes the system file descriptor.

Unlike in Perl 5, an IO object is not a special symbol table entry neither this object is available magically anywhere else. But as in Perl 5, unless stated otherwise, IO::Closeable objects always close themselves during destruction

IO::Socket

role IO::Socket { has %.options; ... }

Accessing the %.options would on Unix be done with getsockopt/setsockopt.

pair
    our List of IO method pair(Int $domain, Int $type, Int $protocol)

A wrapper for socketpair(2), returns a pair of IO objects representing the reader and writer ends of the socket.

   use Socket;
   ($r, $w) = Socket.pair(AF_UNIX, SOCK_STREAM, PF_UNSPEC);

IO::Listening

open
 method open()

 Does a bind() and a listen().  
accept
 method IO::Socket accept()

IO::FileDescriptor

This role indicates that this object actually represents an open file descriptor in the os level.

method int fileno()

File descriptors are always native integers, conforming to C89.

Classes

IO::File

This does file input and output.

class IO::File does IO::Streamable { ... }

new
 method new(
        FSNode :$FSNode, 
        Str :$Filename,
        :$fd
        Bool :$NoOpen, 
        :$Writeable,
        :$Readable
 );

The FSNode, Filename and fd options are mutually exclusive. If "use portable" is in effect, the Filename option throws an error; use an FSNode instead.

NoOpen is passed to IO::Streamable.new()

Examples:

 # Read -- throws errors with 'use portable'
 $fobj = new IO::File(Filename => $filename);

 # Write -- works with 'use portable'
 $fobj = new IO::File(
        FSNode => IO::FSNode.new(type => 'Unix', Filename => $filename), 
        Writeable => 1
 );

 # Read using file descriptor
 $fobj = new IO::File(fd => $fd);

This final example associates an IO object with an already-open file descriptor, presumably passed in from the parent process.

open()

This function opens a file that had the "NoOpen" option passed to the new() method.

IO.truncate
IO.fcntl

Available only as a handle method.

IO::FileSystem

This represents the filesystem.

class IO::FileSystem does IO::Streamable does Tree { has Str $.fstype; # ext3, ntfs, vfat, reiserfs, etc has Str $.illegal_chars; # ie. /\x0 has Int $.max_path; ... }

It inherits $cwn and $root from Tree.

glob

Returns FSNode objects

find

Returns FSNode objects

rename

IO::FSNode

class IO::FSNode does Tree::Node { has Array of IO::FSNodeACL @.ACLs; has Hash of %.times; ... }

The %times has keys that can be eg. ctime, Modification, and Access (and maybe others on other operating systems), and the values are all Temporal::Instant objects.

When .path() is implemented, it should return the path that this was opened with.

IO ~~ :X
EXPR ~~ :X
  $file.:X
  $file ~~ :X

A file test, where X is one of the letters listed below. This unary operator takes one argument, either a filename or a filehandle, and tests the associated file to see if something is true about it.

A Pair used as a pattern is treated as a file test.

    :r  File is readable by effective uid/gid.
    :w  File is writable by effective uid/gid.
    :x  File is executable by effective uid/gid.
    :o  File is owned by effective uid.

    :R  File is readable by real uid/gid.
    :W  File is writable by real uid/gid.
    :X  File is executable by real uid/gid.
    :O  File is owned by real uid.

    :e  File exists.
    :z  File has zero size (is empty).
    :s  File has nonzero size (returns size in bytes).

    :f  File is a plain file.
    :d  File is a directory.
    :l  File is a symbolic link.
    :p  File is a named pipe (FIFO), or Filehandle is a pipe.
    :S  File is a socket.
    :b  File is a block special file.
    :c  File is a character special file.
    :t  Filehandle is opened to a tty.

    :u  File has setuid bit set.
    :g  File has setgid bit set.
    :k  File has sticky bit set.

    :T  File is an ASCII text file (heuristic guess).
    :B  File is a "binary" file (opposite of :T).

    :M  Script start time minus file modification time, in days.
    :A  Same for access time.
    :C  Same for inode change time (Unix, may differ for other platforms)

The interpretation of the file permission operators :r, :R, :w, :W, :x, and :X is by default based on:

  • The mode of the file and the uids and gids of the user
  • ACLs (access control lists)
  • read-only filesystems

There may be other reasons you can't actually read, write, or execute the file. Such reasons may be for example network filesystem access controls and unrecognized executable formats.

Also note that, for the superuser on the local filesystems, the :r, :R, :w, and :W tests always return 1, and :x and :X return 1 if any execute bit is set in the mode. Scripts run by the superuser may thus need to do a stat() to determine the actual mode of the file, or temporarily set their effective uid to something else.

The :T and :B switches work as follows. The first block or so of the file is examined for odd characters such as strange control codes or characters with the high bit set. If too many strange characters (>30%) are found, it's a :B file; otherwise it's a :T file. Also, any file containing null in the first block is considered a binary file. If :T or :B is used on a filehandle, the current IO buffer is examined rather than the first block. Both :T and :B return true on a null file, or a file at EOF when testing a filehandle. Because you have to read a file to do the :T test, on most occasions you want to use a :f against the file first, as in next unless $file ~~ :f && $file ~~ :T .

You can test multiple features using junctions:

  if -$filename ~~ :r & :w & :x  {...}

Or pass multiple tests together in OO style:

  if $filename.TEST(:e,:x) {...}
realpath
 method Str realpath();

Gets the real path to the object, resolving softlinks/shortcuts, etc

=== operator
 method infix:<===>(Str $filename);

Test whether the specified filename is the same file as this file. On a Unix system, this would presumably be done by comparing inode numbers or something.

new

This is called automatically on object creation.

multi method new(Array of Str :@PathElements); multi method new(Str :$Type, Str :$Path, Str :$Create); multi method new(Str :$Path);

This last throws an error if "use portable" pragma is used.

If the "Create" option is passed in, and the node doesn't exist in the filesystem, it attempts to create the node; this can be used for "mkdir", "link", and similar functionality.

Examples:

 $fsnode = new IO::FSNode(PathElements => ['home', 'wayland']);
 $fsnode = new IO::FSNode(Type => 'Unix', Path => '/home/wayland');
 $fsnode = new IO::FSNode(Path => '/home/wayland'); # portability error
delete

This deletes the FSNode from the filesystem. If the node has children, it throws an error unless the "Recursive" option is specified. Returns the number of nodes deleted.

IO::FSNodeACL

This is a basic abstraction; for better control, use the operating-system specific interfaces, over which this is a thin veneer.

class IO::FSNodeACL { has Str $.type; # "User", "Group", "Everyone", ??? has Str $.id; # username or groupname; unused for $type eq "Everyone" has %.permissions; # Unsupported values may (or may not) throw # UnsupportedPermission when set or read has IO::FSNode $.owningObject; ... }

The permissions used in %permissions are:

Readable

Should be supported by all filesystems as an item to read from the hash for the group "Everyone".

Writeable

Should be supported by all filesystems as an item to read from the hash for the group "Everyone".

Executeable

Supported on most Unix systems, anyway. Windows should be able to guess when this is read, and throw an exception if written to.

Default

An ACL of User,fred,Default sets the user "fred" to be the owner of the file. This can be done with groups too. Work on Unix, at least.

The $.owningObject attribute of FSNodeACL shows what the ACL is set on. On a Windows system, this can be a parent directory, as permissions are inherited.

IO::FileNode

 role   IO::FileNode does IO::FSNode {
...
 }
our List multi method lines (IO $handle:) is export;
our List multi lines (Str $filename);

Returns all the lines of a file as a (lazy) List regardless of context. See also slurp.

our Item multi method slurp (IO $handle: *%opts) is export;
our Item multi slurp (Str $filename, *%opts);

Slurps the entire file into a Str or Buf regardless of context. (See also lines.) Whether a Str or Buf is returned depends on the options.

IO::DirectoryNode

 role   IO::DirectoryNode does IO::FSNode {
...
 }
open
  $dir.open();

Opens a directory for processing, if the new() method was passed the NoOpen option. Makes the directory looks like a list of autochomped lines, so just use ordinary IO operators after the open.

rmdir FILENAME
rmdir

Deletes the directory specified by FILENAME if that directory is empty. If it succeeds it returns true, otherwise it returns false and sets $! (errno). If FILENAME is omitted, uses $_.

IO::LinkNode

 role   IO::LinkNode does IO::FSNode {
...
 }
link
readlink
symlink

IO::Socket::TCP

class IO::Socket::TCP does IO::Socket does IO::Streamable { ... }

has $.RemoteHost
has $.RemotePort
has $.LocalHost
has $.LocalPort
new
 method IO::Socket::TCP new(
        Str :$RemoteHost, Str :$RemotePort, 
        Str :$LocalHost, Str :$LocalPort, 
        Bool :$Blocking, 
        Bool :$NoOpen
 );

The NoOpen option is passed to IO::Streamable.new()

IPv6 is supported.

open
 method open()

If it's not an IO::Listening, it does a connect().

It's intended for the case where the creation of the object didn't do one.

method Int read($buf is rw, Int $bytes)

Does a recv().

method Int write($buf, Int $bytes)

Does a send().

IO.getpeername
/[get|set][host|net|proto|serv|sock].*/

IO::Pipe

class IO::Pipe does IO::Streamable { ... }

May also do IO::Readable and IO::Writable, depending on opening method.

close()

If the file handle came from a piped open, close will additionally return false if one of the other system calls involved fails, or if the program exits with non-zero status. (If the only problem was that the program exited non-zero, $! will be set to 0.) Closing a pipe also waits for the process executing on the pipe to complete, in case you want to look at the output of the pipe afterwards, and implicitly puts the exit status value of that command into $!.

Pipe.to
    our IO method to(Str $command, *%opts)

Opens a one-way pipe writing to $command. IO redirection for stderr is specified with :err(IO) or :err<Str>. Other IO redirection is done with feed operators. XXX how to specify "2>&1"?

Pipe.from
    our IO method from(Str $command, *%opts)

Opens a one-way pipe reading from $command. IO redirection for stderr is specified with :err(IO) or :err<Str>. Other IO redirection is done with feed operators. XXX how to specify "2>&1"?

Pipe.pair
    our List of IO method pair()

A wrapper for pipe(2), returns a pair of IO objects representing the reader and writer ends of the pipe.

   ($r, $w) = Pipe.pair;

Calls that operate on the default IO handle

close()
open()

...

OS-specific classes

Unix

IO::FSNode::Unix

chown
    our Int multi chown ($uid = -1, $gid = -1, *@files)

Changes the owner (and group) of a list of files. The first two elements of the list must be the numeric uid and gid, in that order. A value of -1 in either position is interpreted by most systems to leave that value unchanged. Returns the number of files successfully changed.

    $count = chown $uid, $gid, ’foo’, ’bar’;
    chown $uid, $gid, @filenames;

On systems that support fchown, you might pass file handles among the files. On systems that don’t support fchown, passing file handles produces a fatal error at run time.

Here’s an example that looks up nonnumeric uids in the passwd file:

   $user = prompt "User: ";
   $pattern = prompt "Files: ";

   ($login,$pass,$uid,$gid) = getpwnam($user)
       or die "$user not in passwd file";

   @ary = glob($pattern);      # expand filenames
   chown $uid, $gid, @ary;

On most systems, you are not allowed to change the ownership of the file unless you’re the superuser, although you should be able to change the group to any of your secondary groups. On insecure systems, these restrictions may be relaxed, but this is not a portable assumption. On POSIX systems, you can detect this condition this way:

    use POSIX qw(sysconf _PC_CHOWN_RESTRICTED);
    $can_chown_giveaway = not sysconf(_PC_CHOWN_RESTRICTED);
chmod LIST

Changes the permissions of a list of files. The first element of the list must be the numerical mode, which should probably be an octal number, and which definitely should not be a string of octal digits: 0o644 is okay, 0644 is not. Returns the number of files successfully changed.

    $cnt = chmod 0o755, 'foo', 'bar';
    chmod 0o755, @executables;
    $mode = '0644'; chmod $mode, 'foo';      # !!! sets mode to --w----r-T
    $mode = '0o644'; chmod $mode, 'foo';     # this is better
    $mode = 0o644;   chmod $mode, 'foo';     # this is best
lstat

Returns a stat buffer. If the lstat succeeds, the stat buffer evaluates to true, and additional file tests may be performed on the value. If the stat fails, all subsequent tests on the stat buffer also evaluate to false.

stat
IO.stat

Returns a stat buffer. If the lstat succeeds, the stat buffer evaluates to true, and additional file tests may be performed on the value. If the stat fails, all subsequent tests on the stat buffer also evaluate to false.

IO::POSIX

Indicates that this object can perform standard posix IO operations. It implies IO::Readable and IO::Writeable.

method IO dup()
has Bool $.blocking is rw
method Bool flock(:$r,:$w)
method Bool funlock()
...

IO::File::Windows

role IO::File::Windows does IO::File { method open(Bool :$BinaryMode) {...} }

open()

Takes the BinaryMode option to open the file in binary mode.

Unfiled

IO.ioctl

Available only as a handle method.

alarm
prompt
    our Str prompt (Str $prompt)

    Should there be an IO::Interactive role?  
Str.readpipe
sysopen
IO.sysseek
umask

Removed functions

IO.eof

Gone, see IO::Endable

IO.fileno

See IO::FileDescriptor

IO.name

Changed to .path(), but we haven't gotten around to specifying this on all of them.

The .name method returns the name of the file/socket/uri the handle was opened with, if known. Returns undef otherwise. There is no corresponding name() function.

pipe

Gone, see Pipe.pair

select(both)

Gone. (Note: for subsecond sleep, just use sleep with a fractional argument.)

IO.shutdown()

Gone, see IO::Socket.close(), $IO::Readable.isReadable, and $IO::Writeable.isWriteable

socketpair

Gone, see Socket.pair

IO.sysread

Gone, see IO::Readable.read()

IO.syswrite

Gone, see IO::Writeable.read()

utime

Gone, see %IO::FSNode.times.

Additions

Please post errors and feedback to perl6-language. If you are making a general laundry list, please separate messages by topic.

POD ERRORS

Hey! The above document had some coding errors, which are explained below:

Around line 464:

'=item' outside of any '=over'

Around line 474:

You forgot a '=back' before '=head2'

Around line 793:

'=item' outside of any '=over'

Around line 810:

You forgot a '=back' before '=head2'

Around line 816:

'=item' outside of any '=over'

Around line 822:

You forgot a '=back' before '=head2'

Around line 938:

'=item' outside of any '=over'

Around line 1008:

You forgot a '=back' before '=head2'

Around line 1033:

'=item' outside of any '=over'

Around line 1037:

You forgot a '=back' before '=head1'

Jump to Line
Something went wrong with that request. Please try again.