Containers.pod

TITLE

DRAFT: Synopsis 32: Setting Library - Containers.pod

AUTHORS

Rod Adams <rod@rodadams.net>
Larry Wall <larry@wall.org>
Aaron Sherman <ajs@ajs.com>
Mark Stosberg <mark@summersault.com>
Carl Mäsak <cmasak@gmail.com>
Moritz Lenz <moritz@faui2k3.org>
Tim Nelson <wayland@wayland.id.au>

VERSION

Created: 19 Feb 2009 extracted from S29-functions.pod

Last Modified: 2 Mar 2012
Version: 30

The document is a draft.

If you read the HTML version, it is generated from the Pod in the specs repository under https://github.com/perl6/specs/blob/master/S32-setting-library/Containers.pod so edit it there in the git repository if you would like to make changes.

Function Roles

Positional

role Positional {...}

The Positional role implies the ability to support postcircumfix:<[ ]>.

Associative

role Associative {...}

The Associative role implies the ability to support postcircumfix:<{ }>.

cat

multi cat( **@list --> Cat )

cat reads arrays serially rather than in parallel as zip does. It returns all of the elements of the containers that were passed to it like so:

cat(@a;@b;@c);

Typically, you could just write (@a,@b,@c), but sometimes it's nice to be explicit about that:

@foo := [[1,2,3],[4,5,6]]; say cat(||@foo); # 1,2,3,4,5,6

In addition, a Cat in item context emulates the Str interface lazily.

[Conjecture: Cats should maybe just do the lazy strings, and leave flattening to other operators.]

roundrobin

multi roundrobin( **@list --> Parcel )

roundrobin is very similar to zip. The difference is that roundrobin will not stop on lists that run out of elements but simply skip any undefined value:

my @a = 1;
my @b = 1..2;
my @c = 1..3;
for roundrobin( @a; @b; @c ) -> $x { ... }

will get the following values for $x: 1, 1, 1, 2, 2, 3

zip

multi zip ( **@list --> Parcel of Parcel )
multi infix:<Z> ( **@list --> Parcel of Parcel )

zip takes any number of arrays and returns one tuple for every index. This is easier to read in an example:

for zip(@a;@b;@c) -> $nth_a, $nth_b, $nth_c {
  ...
}

Mnemonic: the input arrays are "zipped" up like a zipper.

The zip function defaults to stopping as soon as any of its lists is exhausted. This behavior may be modified by conceptually extending any short list using *, which replicates the final element.

If all lists are potentially infinite, an evaluation in eager context will automatically fail as soon as it can be known that all sublists in the control of iterators of infinite extent, such as indefinite ranges or arbitrary replication. If it can be known at compile time, a compile-time error results.

Z is an infix equivalent for zip:

for @a Z @b Z @c -> $a, $b, $c {...}

In lol context a list of List is returned instead of a flat list.

List

The following are defined in the List class:

class List does Positional {...}

new

multi method new(*@args --> List )

Constructs an List that can visit all iterable elements of all the arguments.

list

sub list(*@args --> List )

Constructs an List that can visit all iterable elements of all the arguments.

cat

multi cat ( @values --> Cat )

Returns a Cat object, a concatenated version of the list that does the Str interface, but generates the string lazily to the extent permitted by the pattern of access to the string. Its two primary uses are matching against an array of strings and doing the equivalent of a join(''), except that join is always eager. However, a Cat in an interpolative context is also effectively eager, since the interpolator needs to know the string length. List context is lazy, though, so a cat of a cat is also lazy, and in fact, you just get a flat cat because cat in a list context is a no-op. The Cat interface also lets you interrogate the object at a particular string position without actually stringifying the element; the regex engine can make use of this to match a tree node, for instance, without serializing the entire subtree.

Accessing a filehandle as both a filehandle and as a Cat is undefined, because lazy objects are not required to be as lazy as possible, but may instead choose to precalculate values in semi-eager batches to maximize cache hits.

classify

multi method classify ( @values: &mapper --> Hash )
multi method classify ( @values: %mapper --> Hash )
multi method classify ( @values: @mapper --> Hash )
multi classify ( &mapper, *@values --> Hash )
multi classify ( %mapper, *@values --> Hash )
multi classify ( @mapper, *@values --> Hash )

classify transforms a list or array of values into a hash representing the classification of those values according to a mapper; each hash key represents the classification for one or more of the incoming list values, and the corresponding hash value contains an array of those list values classified by the mapper into the category of the associated key. For example:

@list = (1, 2, 3, 4);
(:@even, :@odd) := |classify { $_ % 2 ?? 'odd' !! 'even' }, @list;

In this example, @even will contain all even numbers from @list and @odd will contain all odd numbers from @list.

To simply transform a list into a hash of arrays:

%cars_by_color := classify { .color }, @cars;
red_car_owners(%cars_by_color<red>.map:{.owner});

A mapper maybe any unary function, hash, or array. Values that have no mapping will be classified under an undefined failure key (whatever is returned by the mapper in use). [Conjecture: if a hash comes with a default value, it can map anything.]

classify always assumes that the mapper returns a single value. To categorize values into multiple categories, see categorize.

categorize

multi method categorize ( @values: &mapper --> Hash )
multi method categorize ( @values: %mapper --> Hash )
multi method categorize ( @values: @mapper --> Hash )
multi categorize ( &mapper, *@values --> Hash )
multi categorize ( %mapper, *@values --> Hash )
multi categorize ( @mapper, *@values --> Hash )

Like classify, categorize transforms a list or array of values into a hash representing the categorizations of those values according to a mapper; each hash key represents one possible categorization for one or more of the incoming list values, and the corresponding hash value contains an array of those list values categorized by the mapper into the category of the associated key.

Unlike classify, categorize always assumes that the return value of the mapper is a list of categories that are appropriate to the current value. Hence, if the mapper returns Nil, the value is discarded rather than being returned under a Nil key. On the other hand, if the mapper returns a list of categories, the value in question will be pushed into multiple hash locations (while classify would merely make a key out of the list returned from the mapper).

grep

multi method grep ( @values: Matcher $test --> List )
multi grep ( Matcher $test, *@values --> List )

grep takes a list or array of values and returns a lazily evaluated list comprised of all of the values from the original list for which the $test smart-matches as true.

Here is an example of its use:

@friends = grep { .is_friend }, @coworkers;

This takes the array @coworkers, checks every element to see which ones return true for the .is_friend method, and returns the resulting list to store into @friends.

Note that, unlike in Perl 5, a comma is required after the Matcher in the multi form.

Note that grep is an implicit loop, so next and last without an argument will iterate or terminate the grep itself, not some loop surrounding the statement containing the grep. Use a label if you mean the other thing.

first

multi method first ( @values: Matcher $test --> Parcel )
multi first ( Matcher $test, *@values --> Parcel )

first searches exactly like grep but returns only the first matching value.

pick

multi method pick ( @values: Int $num = 1 --> Any )
multi method pick ( @values: Whatever --> Any )
multi pick ( Int $num, *@values --> Any )
multi pick ( Whatever, *@values --> Any )

pick takes a list or array of values and returns a random selection of elements from the list (without replacement; see roll for dice roll aka replacement semantics). If * is specified as the number (or if the number of elements in the list is less than the specified number), all the available elements are returned in random order:

@team = @volunteers.pick(5);
@shuffled = @deck.pick(*);

Due to the normal semantics of returning a Parcel, a pick of a single element may be used as an item without requiring .[0].

roll

multi method roll ( @values: Int $num = 1 --> Any )
multi method roll ( @values: Whatever --> Any )
multi roll ( Int $num, *@values --> Any )
multi roll ( Whatever, *@values --> Any )

roll takes a list or array of values and returns a random selection of elements from the list, like rolling N independent dice, where each list element corresponds to one side of your dice. This is also known as "pick with replacement", that is, like pulling one marble out of bag and putting it back, and doing this N times; see pick for the non-replacement semantics. If * is specified as the number, * provides an infinite list of random choices from @values:

@byte = (0,1).roll(8);                   # 8d2
for (1..20).roll(*) -> $die_roll { ... } # 1d20 xx *

Due to the normal semantics of returning a Parcel, a roll of a single element may be used as an item without requiring .[0].

Note that roll 1, 6 is not the same as a 1d6, but always returns 6 as a degenerate one-sided die. Use roll 1, 1..6 to simulate a six-sided die.

join

multi method join ( @values: Str $separator = '' --> Str )
multi join ( Str $separator = '', *@values --> Str )

join returns a single string comprised of all of the elements of @values, separated by $separator.

Given an empty list, join returns the empty string.

The separator defaults to the null string. To join with space, just coerce to Str.

map

multi method map ( @values: Code *&expression --> List of Parcel )
multi map ( Code $expression, *@values --> List of Parcel )

map returns a lazily evaluated list which is comprised of the return value of the expression, evaluated once for every one of the @values that are passed in.

If the expression returns no values or multiple values, then the resulting list may not be the same length as the number of values that were passed.

The actual return value is a multislice containing one slice per map iteration. In most contexts these slices are flattened into a single list.

Note that map is an implicit loop, so next and last without an argument will iterate or terminate the map itself, not some loop surrounding the statement containing the map. Use a label if you mean the other thing.

reduce

multi method reduce ( @values: Code *&expression --> Item )
multi reduce ( Code $expression ;; *@values --> Item ) {
  my $res;
  for @values -> $cur {
    FIRST {$res = $cur; next;}
    $res = &$expression($res, $cur);
  }
  $res;
}

Note that reduce is an implicit loop, so next and last without an argument will iterate or terminate the reduce itself, not some loop surrounding the statement containing the reduce. Use a label if you mean the other thing.

reverse

multi method reverse ( @values: --> List ) is export
multi reverse ( *@values --> List ) {
   gather {
       1 while take pop @values;
   }
}

Note that strings are now reversed with flip.

rotate

multi method rotate ( @values is copy: Int $n = 1 --> List ) is export

Produces a new list with the same elements as the old list, rotated by the specified amount. See Array::rotate for more info.

sort

multi method sort( @values: *&by --> Array )
multi method sort( @values: Ordering @by --> Array )
multi method sort( @values: Ordering $by = &infix:<cmp> --> Array )

multi sort( Ordering @by,  *@values --> List )
multi sort( Ordering $by, *@values --> List )

Returns @values sorted, using criteria $by or @by for comparisons. @by differs from $by in that each criterion is applied, in order, until a non-zero (tie) result is achieved.

If $by is a code object of arity zero or one, it is applied on each item of @values, and @values is sorted by comparing the result values with &infix:<cmp> (Schwartzian Transform).

Ordering is as described in "Type Declarations" in S29. Any Ordering may receive either or both of the mixins descending and canon(Code $how) to reverse the order of sort, or to adjust the case, sign, or other order sensitivity of cmp. (Mixins are applied to values using but.) If a Signature is used as an Ordering then sort-specific traits such as is canon($how) are allowed on the positional elements.

If all criteria are exhausted when comparing two elements, sort should return them in the same relative order they had in @values.

To sort an array in place use the .=sort mutator form.

See http://www.nntp.perl.org/group/perl.perl6.language/16578 for more details and examples (with is insensitive meaning is canonicalized(&lc).)

min

multi method min( @values: *&by --> Any )
multi method min( @values: Ordering @by --> Any )
multi method min( @values: Ordering $by = &infix:<cmp> --> Any )

multi min( *@values, Ordering :@by --> Any )
multi min( *@values, Ordering :$by --> Any )

Returns the earliest (i.e., lowest index) minimum element of @values , using criteria $by or @by for comparisons. @by differs from $by in that each criterion is applied, in order, until a non-zero (tie) result is achieved.

Ordering is as described in "Type Declarations" in S29. Any Ordering may receive the mixin canonicalized(Code $how) to adjust the case, sign, or other order sensitivity of cmp. (Mixins are applied to values using but.) If a Signature is used as an Ordering then sort-specific traits such as is canonicalized($how) are allowed on the positional elements.

For a min function that does not require an ordering, see the [min] reduction operator.

max

multi method max( @values: *&by --> Any )
multi method max( @values: Ordering @by --> Any )
multi method max( @values: Ordering $by = &infix:<cmp> --> Any )

multi max(*@values, Ordering :@by, --> Any )
multi max(*@values, Ordering :$by, --> Any )

Returns the earliest (i.e., lowest index) maximum element of @values , using criteria $by or @by for comparisons. @by differs from $by in that each criterion is applied, in order, until a non-zero (tie) result is achieved.

Ordering is as described in "Type Declarations" in S29. Any Ordering may receive the mixin canonicalized(Code $how) to adjust the case, sign, or other order sensitivity of cmp. (Mixins are applied to values using but.) If a Signature is used as an Ordering then sort-specific traits such as is canonicalized($how) are allowed on the positional elements.

For a max function that does not require an ordering, see the [max] reduction operator.

minmax

multi method minmax( @values: *&by --> Any )
multi method minmax( @values: Ordering @by --> Any )
multi method minmax( @values: Ordering $by = &infix:<cmp> --> Any )

multi minmax( *@values, Ordering :@by --> Any )
multi minmax( *@values, Ordering :$by --> Any )

Returns a Range from the minimum element of @values to the maximum element, using criteria $by or @by for comparisons. @by differs from $by in that each criterion is applied, in order, until a non-zero (tie) result is achieved. Range elements in @values are treated as if their minimum and maximum values were passed individually, except that if the corresponding excludes flag is set in Range, the excludes flag is also set in the returned Range.

Ordering is as described in "Type Declarations" in S29. Any Ordering may receive the mixin canonicalized(Code $how) to adjust the case, sign, or other order sensitivity of cmp. (Mixins are applied to values using but.) If a Signature is used as an Ordering then sort-specific traits such as is canonicalized($how) are allowed on the positional elements.

For a minmax function that does not require an ordering, see the [minmax] reduction operator.

any

multi method any( @values: --> Junction )
multi any( *@values --> Junction ) is export

Returns a junction with all the values of the list |-ed together. The junction will only match against another value if at least one of the values in the list matches.

all

multi method all( @values: --> Junction )
multi all( *@values --> Junction ) is export

Returns a junction with all the values of the list &-ed together. The junction will only match against another value if all of the values in the list match.

one

multi method one( @values: --> Junction )
multi one( *@values --> Junction ) is export

Returns a junction with all the values of the list ^-ed together. The junction will only match against another value if exactly one of the values in the list matches.

none

multi method none( @values: --> Junction )
multi none( *@values --> Junction ) is export

Returns a junction which will only match against another value if none of the values in the list matches.

comb

multi method comb ( Regex $matcher, Int $limit = * )

Treats the list as a string (by simply joining them together), and calls .comb on that string with the same parameters. See Str::comb.

Array

All these methods are defined in the Array role/class.

role Array does List {...}

new

multi method new(*@args --> Array )

Constructs a new Array containing the arguments passed to new.

shape

method shape (@array: --> Parcel ) is export

Returns the declared shape of the array, as described in S09.

end

method end (@array: --> Any ) is export

Returns the final subscript of the first dimension; for a one-dimensional array this simply the index of the final element. For fixed dimensions this is the declared maximum subscript. For non-fixed dimensions (undeclared or explicitly declared with *), the index of the actual last element is used. So that the degenerate case of an empty range works correctly, -1 is returned if the array is empty. (If you actually try to index with -1, you'll get a failure.)

elems

method elems (@array: --> Int ) is export

Returns the length of the array counted in elements.

delete

method delete (@array : *@indices --> List ) is export

Sets elements specified by @indices in the invocant to a non-existent state, as if they never had a value. Deleted elements at the end of an Array shorten the length of the Array, unless doing so would violate an is shape() definition.

The interpretation of @indices is subject to change. The normal way to delete is by applying a :delete adverb to any subscripting operation.

Returns the value(s) previously held in deleted locations.

:delete

This adverb may be applied to any subscripting operation. The operation returns the elements normally, but reads them out destructively.

exists

method exists (@array: Int $index --> Bool )

True if the specified Array element has been assigned to. This is not the same as being defined.

Supplying a different number of indices than invocant has dimensions is an error.

The normal way to test for existence is to apply the :exists adverb to a subscripting operation.

:exists

This adverb may be applied to any subscripting operation. The operation returns true if specified element exists. If a slice is specified by the subscript, a Parcel of Bool is returned, which can be processed using junctions.

pop

multi method pop ( @array: --> Scalar ) is export

Remove the last element of @array and return it. If @array is empty returns a failure.

push

multi method push ( @array: *@values --> Array ) is export

Adds all the values to the end of @array eagerly. Returns the modified array.

plan

multi method plan ( @array: *@list --> Array ) is export

Adds the list to the end of @array lazily as a kind of "lazy push". (That is, the reified part of the array is not modified, but the list is appended to the not-yet-reified specifications for how to extend the array on demand, if it is subscripted or shifted beyond the currently reified elements.) Returns the modified array.

Note that the semantics of these are different:

@ro := (0,1,*+*...*);
@rw.plan(0,1,*+*...*);

In the first case, @ro is bound directly to the list, so its values are memoized but not considered mutable. In contrast, @rw allows modification of any reified element; new elements are merely initialized to the fibonacci sequence. If you try to modify a non-reified element, the array will be reified to that point before the modification is attempted.

For all external purposes, the array considers that its plan is part of the array. If you ask for .elems, for instance, it will try to reify the entire array, which might take a long time in the case of infinite arrays (though a list may return Inf if it can determine this). Methods such as .splice can refer to the rest of the list in the abstract, and need only reify those elements necessary to perform the operation in question. (Hence, there is no need for an .unplan method, since the plan of an array may be thrown away by using splice without the risk of memory exhaustion.)

rotate

multi method rotate ( @array is copy: Int $n = 1, Int *@n --> Array ) is export

Produces a new array with the same elements as the old array, rotated by the specified amount. A positive rotation of 1 is defined as:

@array.push(@array.shift);

A negative rotation of 1 is defined as:

@array.unshift(@array.pop);

If the magnitude of $n is larger than the number of elements, the rotational semantics must be equivalent to doing:

@array.rotate(sign($n)) for ^abs($n)

The new array to be returned contains nothing but aliases for the old array's elements; however, you can use this to get any of three different semantic behaviors:

@a.=rotate       # @a is rotated in place
@b = @a.rotate   # @b contains copied elements of rotated @a
@b := @a.rotate  # @b contains aliased elements of rotated @a

If additional rotations are specified via the slurpy, they are applied to subdimensions of multidimensional arrays. (To perform a flat rotation on a shaped array requires flattening to a list and rotating that instead.)

shift

multi method shift ( @array: --> Scalar ) is export

Remove the first element from @array and return it. If @array is empty returns a failure.

splice

multi method splice( @array is rw: $offset = 0, $size = Inf, *@values --> Any ) is export

splice fills many niches in array-management, but its fundamental behavior is to remove zero or more elements from an array and replace them with a new (and potentially empty) list. This operation can shorten or lengthen the target array.

$offset is the index of the array element to start with. A WhateverCode whose argument is the number of elements in the @array may also be used.

$size is the number of elements to remove from @array. A WhateverCode similar to $offset may be used instead (note that this specifies a position, not an actual number of elements to remove).

The slurpy list of values (if any) is then inserted at $offset.

Calling splice with a traditional parameter list, you must define $offset and $size if you wish to pass a replacement list of values. To avoid having to pass these otherwise optional parameters, use the piping operator(s):

splice(@array,10) <== 1..*;

which replaces @array[10] and all subsequent elements with an infinite sequence starting at 1.

If @array is multidimensional, splice operates only on the first dimension, and works with Array References.

splice returns a Parcel of the deleted elements, which behaves as expected in either list or item context.

$offset and $size will be treated as Int. The function fails if either parameter is negative, or undefined.

Either of $offset or $size may be specified relative to the length of the array using a WhateverCode whose argument will be the number of elements in the array.

While it is illegal for the offset to be larger than the size of the array, it is allowed for the final position to be off the end of the array; in this case the entire rest of the array is spliced, whatever is available.

unshift

multi method unshift ( @array: *@values --> Array ) is export

unshift adds the values onto the start of the @array and returns the modified array.

keys

kv

pairs

values

multi method keys   ( @array: ) is export
multi method kv     ( @array: ) is export
multi method pairs  ( @array: ) is export
multi method enums  ( @array: ) is export
multi method values ( @array: ) is export

Iterates the elements of @array, in order.

What is returned at each element of the iteration varies with function. values returns the value of the associated element; kv returns a 2 element list in (index, value) order, pairs a Pair(index, value). With pairs the values are references back to the original containers, while with enums a snapshot of those values is taken. That is, .pairs returns a PairMap while enums returns an EnumMap.

If @array is declared to be multi-dimensional, the keys returned may in fact be slice lists. (Arrays that are effectively multi-dimensional by containing other arrays or hashes are treat with their declared dimensionality, not their effective dimensionality.)

Note that kv @array returns the same as zip(^@array; @array)

Hash

The following are defined in the Hash close.

class Hash is EnumMap {...}

:delete

This adverb may be applied to any subscripting operation. The operation returns the elements normally, but reads them out destructively.

:exists

This adverb may be applied to any subscripting operation. The operation returns true if specified element exists. If a slice is specified by the subscript, a Parcel of Bool is returned, which can be processed using junctions.

keys

kv

pairs

values

multi method keys   ( %hash: ) is export
multi method kv     ( %hash: ) is export
multi method pairs  ( %hash: ) is export
multi method enums  ( %hash: ) is export
multi method values ( %hash: ) is export

Iterates the elements of %hash. The order is implementation dependent and arbitrary, but will be the same between successive calls to these functions, as long as %hash doesn't change.

What is returned at each element of the iteration varies with function. keys only returns the key; values the value; kv returns both as a 2 element list in (key, value) order, pairs a Pair(key, value). With pairs the values are references back to the original containers, while with enums a snapshot of those values is taken. That is, .pairs returns a PairMap while enums returns an EnumMap.

Note that kv %hash returns the same as zip(keys %hash; values %hash)

The lvalue form of keys is no longer supported. Use the .buckets property instead.

any

multi method any( %hash: --> Junction ) is export

Returns a junction with all the keys of the hash |-ed together. The junction will only match against another value if at least one of the keys in the hash matches.

all

multi method all( %hash: --> Junction ) is export

Returns a junction with all the keys of the hash &-ed together. The junction will only match against another value if all of the keys in the hash match.

one

multi method one( %hash: --> Junction ) is export

Returns a junction with all the keys of the hash ^-ed together. The junction will only match against another value if exactly one of the keys in the hash matches.

none

multi method none( %hash: --> Junction ) is export

Returns a junction which will only match against another value if none of the keys in the hash matches.

invert

multi method invert ( %hash: --> List ) is export {
    map -> $k, $v { $v X=> $k }, %hash.kv;
}

Produces a backmapping of values to keys, expanding list values into multiple pairs. (The X=> expands $v if it is a list.) [NB: this may need refinement to handle keys that do Positional.]

push

multi method push ( %hash: *@values --> Hash ) is export

Like hash assignment insofar as it accepts either Pair objects or alternating keys and values; also like in that it returns the new hash. However, unlike assignment, when a duplicate key is detected, push coerces the colliding entry's value to an array and pushes the Pair's value onto that array. Hence to invert a hash containing duplicate values without losing (associative) information, say:

%out.push(%in.invert)

Note that when reading the values of such a hash, you must not assume that all the elements are arrays, since the first instance of a key merely sets the value without turning it into an array. (Fortunately, most list operators create a list of one element when used on an object that is not a list.)

The intent is that reversing a hash twice produces a hash equivalent to the original hash.

Classes and Roles

This documents Buf, List, Range, Set, Bag, Junction, Array, Hash, KeyHash, KeySet, KeyBag, Pair, and PairMap.

Range

class Range does Positional does Iterable {
    method from() {...}
    method to() {...}
    method min() {...}
    method max() {...}
    method List minmax() {...}
}

Buf

class Buf does Positional does Stringy {...}

A mutable container for an array of integer values in contiguous memory. The default constructor takes a single array parameter of integers, the largest of which determines the actual type. So

Buf.new([:16<c3>, :16<B6>]) # or
Buf.new([195, 182])         # which is exactly the same

returns a buf8 containing two uint8 items, while

Buf.new([256])

returns a buf16 which consists of a single uint16.

To explicit request a Buf of a specific size, you can use

Buf.new([127], :size(16))     # returns a buf16
Buf.new([1024], :size(8))     # dies, because 1024 >= 2**8

Subtypes with additional constraints like utf8 (which only allows valid UTF-8 byte sequences) exist and provide similar constructors. See "Built-In Data Types" in S02.

The array in the constructor used to be slurpy rather than positional, but the former was deemed to be too inefficient (during signature construction) for arrays of many elements.

Methods

decode

method decode($encoding = $?ENC, $nf = $?NF --> Str )

Decode the Buf into a Str. For subtypes that know their encoding (like utf8, utf16) the $encoding parameter defaults to their intrinsic encoding instead.

Buf Operators

Two Buf objects of the same bit size can be compared with the same operators as strings (in particular eq, lt, le, gt, ge, ne and leg), but actually compares the stored integers, not characters. Concatenating two compatible Bufs produces an object of the most specific type possible, for example buf8.new() ~ utf8.new() results in a buf8 object.

Comparing or concatenating two Buf objects of different bit sizes, or a Buf that doesn't know its encoding with a Str throws an exception.

Likewise, coercing an encoding-unaware Buf to Str dies.

[Conjecture: The behaviour of encoding-aware Buf objects on string operators is intentionally not defined yet, because I have no idea what implications on speed and usage they might have --moritz].

Enum and Pair

class Enum does Associative {...}
class Pair does Associative {...}

A value formed by associating a single key with a single value. In an Enum, both key and value are immutable. In a Pair, the key is immutable but the value mutable. Enum works identically to Pair unless you try to modify it.)

invert

multi method invert ( $pair: --> List ) is export {
 $pair.value X=> $pair.key
}

EnumMap

class EnumMap does Associative does Positional {...}

An immutable hash value, viewable either as a (constant) hash or as a sequence of Enums. The keys may not contain duplicates, while the values may. The implementation of EnumMap associativity is not guaranteed to be as efficient as a Hash lookup. (A brute force linear scan for the matching key is allowed. Optimization is also allowed.)

PairMap

class PairMap does Associative does Positional {...}

A hash value that is mutable only in values, differing from a normal hash insofar as the key set is frozen. It may be accessed either as a frozen hash or as a sequence of Pairs. The keys may not contain duplicates, while the values may. The implementation of PairMap associativity is not guaranteed to be as efficient as a Hash lookup. (A brute force linear scan for the matching key is allowed. Optimization to something like an ordered hash is also allowed.)

Set

class Set does Associative {...}

A set of unique values. When used as a hash always treats the set's values as the keys of the hash, returning True for set elements. See KeySet for a container that can represent different sets as keys are added or deleted. A Set responds to hash operators as if it were a Hash of True.

pick

multi method pick ( $set: Int $num = 1 --> Any )
multi method pick ( $set: Whatever --> Any )

Works like an ordinary list pick.

roll

multi method roll ( $set: Int $num = 1 --> Any )
multi method roll ( $set: Whatever --> Any )

Works like an ordinary list roll.

KeySet

A mutable Set container, represented as KeyHash of Bool.

Bag

class Bag does Associative {...}

A collection of values that need not be unique, represented by an associative mapping from each key value to its replication number. The .elems method returns the sum of all replication values.

pick

multi method pick ( $bag: Int $num = 1 --> Any )
multi method pick ( $bag: Whatever --> Any )
multi method pickpairs ( $bag: Int $num = 1 --> Any )
multi method pickpairs ( $bag: Whatever --> Any )

Like an ordinary list pick, but returns keys of the bag weighted by values, as if the keys were replicated the number of times indicated by the corresponding value and then list pick used. KeyBag is the mutable form of Bag. A Bag responds to hash operators as if it were a Hash of UInt.

The underlying metaphor for picking is that you're pulling colored marbles out a bag. (For "picking with replacement" see roll instead.) Picking require tracking the temporary state, so the immutable Bag is copied to a temporary private KeyBag, and the picks are made from that using the corresponding .grab or .grabpairs method (see below).

Each .pick invocation maintains its own private state and has no effect on subsequent .pick invocations.

roll

multi method roll ( $bag: Int $num = 1 --> Any )
multi method roll ( $bag: Whatever --> Any )

Like an ordinary list roll, but returns keys of the bag weighted by values, as if the keys were replicated the number of times indicated by the corresponding value and then list roll used. The underlying metaphor for rolling is that you're throwing $num dice that are independent of each other, which (in bag terms) is equivalent to picking a colored marble out your bag and then putting it back, and doing this $num times. In dice terms, the number of marbles corresponds to the number of sides, and the number of marbles of the same color corresponds to number of sides with the same color. (For "picking without replacement" see pick instead.) Rolling requires no temporary state.

KeyBag

A mutable Bag container, represented as KeyHash of UInt.

KeyHash

role KeyHash[::T, $default = Any] does Associative {...}

A KeyHash represents a mutable set of values, represented as the keys of a Hash. When asked to behave as a list it ignores its values and returns only .keys. KeySet and KeyBag are derived from this type, but constrain their values to be Bool and UInt, respectively. A KeyHash automatically deletes any key whose value goes false. For any KeyHash, the .elems methods returns the current sum of the values, which the KeyHash must either track or compute on demand. Tracking is preferable for efficient implementation of .pick and .grab.

All KeyHash containers have a default value that is false (such as 0 or '' or Nil or Bool::False), and keep around only those entries with non-default values, deleting any entry if its value goes to false.

grab

multi method grab ( $bag: Int $num = 1 --> Any )
multi method grab ( $bag: Whatever --> Any )
multi method grabpairs ( $bag: Int $num = 1 --> Any )
multi method grabpairs ( $bag: Whatever --> Any )

Like pick, a grab returns a random selection of elements, weighted by the values corresponding to each key. Since a KeyHash, unlike a Set or Bag, is mutable, .grab works directly on the KeyHash, modifying it in place. (You may use .pick to treat the KeyHash as immutable; this will copy it and grab only from the temporary copy.)

Grabbing decrements the grabbed key's replication value by one (deleting the key when it goes to 0). By definition, .elems of the bag also decreases by one, so the probabilities stay consistent through subsequent grab operations.

With the .grabpairs version, the replication value of the picked key is forced immediately to 0, removing all marbles of that color from the bag, as it were. Instead of returning keys, returns the grabbed values as a list of Pair objects, whose keys are the deleted keys and whose values are the deleted replication values.

KeyWeight

A KeyHash of FatRat; like a KeyBag but may have non-integral weights for use in weighted picking. Keys with fractional weights are deleted if they go to 0. Negative weights are not deleted, but the implementation may complain if it notices you attempting to use such a weight.

Junction

All method calls on junctions autothread because the type provides no public methods.

!eigenstates

method !eigenstates (Junction $j: --> Parcel)

Returns an unordered list of the values that constitute the junction (formerly called .values). It flattens nested junctions of the same type, so (1|(2|3)).eigenstates returns an arbitrary permutation of the list 1, 2, 3.

Note this is a private method; you must arrange for yourself to be trusted by the junctions class in order to call it, which probably involves evil MONKEY_TYPING.

Alternately, the values may be explicitly converted to a set value using .Set or Set(). Note, however, that the set of eigenstates for a none junction defines the values that are *not* there, so the resulting Set will be the complement to the universe of values recognized by the junction! Also note that any and all junctions represent sets of sets of items, not sets of items. Since set items are by definition unique, only the one junction can be unthinkingly mapped to and from a given set. (This is why we discourage casual use of the eigenstates as a set.)

Additions

Please post errors and feedback to perl6-language. If you are making a general laundry list, please separate messages by topic.