1.1This document describes metadata for functions/methods. This specification is part of Rinci. Please do a read up on it first, if you have not already done so.
Result envelope. Function should return an enveloped result to express error code/message as well as actual result. The envelope can be produced by the function itself, or added by a wrapper tool. Result envelope is modeled after HTTP or PSGI response, it is an array in the following format:
[STATUS, MESSAGE, RESULT, META]STATUS is a 3-digit integer, much like HTTP response status code and is explained further in "Envelope status codes". MESSAGE is a string containing error message. RESULT (or PAYLOAD) is the actual content to be returned and can be omitted or set to undef if the function does not need to return anything. META is called result metadata, a hash containing extra data, analogous to HTTP response headers. Result metadata is specified further in Rinci::result.
Some example of an enveloped results:
[200, "OK", 42]
[404, "Not found"]
[500, "Can't delete foo: permission denied", {errno=>51}]
[200, "Account created", {id=>9323}, {undo_data=>["delete_account"]}]As mentioned, an enveloped result can contain error code/message as well as the actual result. It can also be easily converted to HTTP response message. And it can also contain extra data, useful for things like the undo protocol (explained later).
Special arguments. Special arguments are some known arguments that start with dash (-) and serve special purposes. You need not specify them in the args metadata property. Examples of special arguments include -undo_action, -undo_data, -dry_run, and they will be explained in relevant sections below.
Functions vs methods. Since in many programming languages (like Perl 5, Python, Ruby, PHP) static functions are not that differentiated from methods, functions and methods share the same Rinci spec. But there are certain properties that can be used to declare if a function is (also) a method or not. See is_func, is_meth, is_class_meth properties below for details.
Multiple dispatch. This specification also does not (yet) have any recommendation on how to best handle functions in languages that support multiple dispatch, like Perl 6: whether we should create multiple metadata or just one. It is more up to the tool and what you want to do with the metadata.
In general, status codes map directly to HTTP response status codes. Below are the suggestion on which codes to use (or avoid).
1xx code
Currently not used.
2xx code
200 should be used to mean success.
206 can be used to signal partial content, for example: a
read_file()function which acceptsbyte_startandbyte_endarguments should return 206 when only partial file content is returned. But in general, use 200 as some callers will simply check for this exact code (instead of checking for range 200-299).3xx code
301 (moved) can be used to redirect callers to alternate location, although this is very rare.
304 (not modified, nothing done). Used for example by setup functions to indicate that nothing is being modified or no modifying action has been performed (see Setup::* modules in CPAN).
4xx code
400 (bad request, bad arguments) should be returned when the function encounters invalid input. A function wrapper can return this code when the function arguments fail the argument schema validation (specified in the
argsproperty).401 (authentication required).
403 (forbidden, access denied, authorization failed).
404 (not found). Can be used for example by an object-retrieval functions (like
get_user()) and the object is not found.For object-listing functions (like
list_users()), when there are no users found matching the requested criteria, 200 code should still be returned with an empty result (like an empty array or hash).Also in general, an object-deletion function (like
delete_user()) should also return 200 (or perhaps 304, but 200 is preferred) instead of 404 when the object specified to be deleted is not found, since the goal of the delete function is reached anyway.408 (request timeout).
409 (conflict). Can be used for example by a
create_user()function when receiving an already existing username.412 (precondition failed). Similar to 409, but can be used to indicate lack of resources, like disk space or bandwidth. For lacking authentication and authorization, use 401 and 403 respectively.
430 (transaction is not supported for this request).
431 (transaction is aborted).
5xx code
500 is the general code to use when a failure occurs during the execution of a function. for example when a
delete_file()function fails to delete specified file (though in this case it can also choose to return 403 instead, which is more specific).501 (not implemented)
503 (service unavailable). You can use this when service is temporarily unavailable, e.g. when system load is too high, a required service is down, etc. Users should try again at a later time.
507 (insufficient storage)
521 (maximum retries reach)
53x (bad metadata) is used when there is something wrong with the metadata.
Try not to use code greater than 555, as some tools use (CODE-300) for error codes that must fit in one unsigned byte (like Perinci::CmdLine).
Specify that the function can be called as a static function (i.e. procedural, not as a method). Default is true if unspecified, but becomes false if is_meth or is_class_meth is set to true.
Example:
# specify that function can be called a method *as well as* a static function
is_meth => 1,
is_func => 1, # if not specified, will default to false after is_meth set to 1Specify that the function can be called as an instance (object) method. Default is false.
Example:
# specify that function is a method
is_meth => 1,Specify that the function can be called as a class method. Examples of class methods include the constructor, but there are others. Default is false.
Example:
# specify that function is a class method
is_class_meth => 1,Specify arguments. Property value is hash of argument names and argument specification. Argument name must only contain letters, numbers, and underscores (and do not start with a number).
Argument specification is a hash containing these keys:
schema => SCHEMA
Data::Sah schema for argument value.
summary => STR
A one-line plaintext summary, much like the
summaryproperty in variable metadata.req => BOOL
Specify that argument is required (although its value can be undef/null). Default is false.
description => STR
A longer description of marked up text, much like the
descriptionproperty. It is suggested to format the text to 74 columns.tags => ARRAY OF (STR|HASH)
An array of tags, can be used by tools to categorize arguments. Not unlike the
tagsproperty.pos => INT
Argument position when specified in an ordered fashion, e.g. in an array. Starts from zero.
greedy => BOOL
Only relevant if pos is specified, specify whether argument should gobble up all remaining values in an ordered argument list into an array.
cmdline_aliases => HASH
Specify aliases for use in command-line options (or other possibly suitable situation where arguments are parsed from command-line-like options). Keys are alias names, values are itself hashes (alias specification). Valid alias specification keys:
summary(a string, optional),schema(optional, defaults to argument's schema),code(a code to set argument value, optional, will be given reference to arguments hash; if not set, the default behavior is simply to set the argument value).completion => CODE
A code to supply argument value completion. Will be explained in the examples.
Example function metadata and its implementation in Perl:
$SPEC{multiply2} = {
v => 1.1,
summary => 'Multiple two numbers',
args => {
a => {
summary => 'The first operand',
description => '... a longer description ...',
schema=>'float*',
pos => 0,
tags => ['category:operand'],
},
b => {
summary => 'The second operand',
description => '... a longer description ...',
schema => 'float*',
pos => 1,
tags => ['category:operand'],
},
round => {
summary => 'Whether to round result',
description => '... a longer description ...',
schema => [bool => {default=>0}],
pos => 2,
tags => ['category:options'],
cmdline_aliases => {
r=>{},
R=>{summary=>'Equivalent to --round=0',
code=>sub {$_[0]{round}=0}},
},
},
}
};
sub multiply2 {
my %args = @_;
my $res = $args{a} * $args{b};
$res = int($res) if $round;
[200, "OK", $res];
}By default, without any wrapper, the function is called with a named hash style:
multiply2(a=>4, b=>3); # 12But with the information from the metadata, a wrapper tool like Sub::Spec::Wrapper is able to change the calling style to positional:
multiply2(4, 3.1, 1); # 12A command-line tool will also enable the function to be called named options as well as positional arguments:
% multiply2 --a 2 --b 3
% multiply2 2 --b 3
% multiply2 2 3As mentioned earlier, cmdline_alises is parsed by command-line option parser:
% multiply2 2 3.5 -r ; # equivalent to multiply2 2 3 --round
% multiply2 2 3.5 -R ; # equivalent to multiply2 2 3 --noround (--round=0)Aliases in cmdline_aliases are not recognized as real arguments:
multiply2(a=>4, b=>3, r=>0); # unknown argument rAnother example (demonstrates cmdline_aliases):
$SPEC{smtpd} = {
v => 1.1,
summary => 'Control SMTP daemon',
args => {
action => {
schema => ['str*' => {in=>[qw/status start stop restart/]}],
pos => 0,
req => 1,
cmdline_aliases => {
status => {
schema => [bool=>{is=>1}],
summary => 'Alias for setting action=status',
code => sub { $_[0]{action} = 'status' },
},
start => {
schema => [bool=>{is=>1}],
summary => 'Alias for setting action=start',
code => sub { $_[0]{action} = 'start' },
},
stop => {
schema => [bool=>{is=>1}],
summary => 'Alias for setting action=stop',
code => sub { $_[0]{action} = 'stop' },
},
restart => {
schema => [bool=>{is=>1}],
summary => 'Alias for setting action=restart',
code => sub { $_[0]{action} = 'restart' },
},
},
},
force => {
schema => 'bool',
},
},
};Another example (demonstrates greedy):
$SPEC{multiply_many} = {
v => 1.1,
summary => 'Multiple numbers',
args => {
nums => {
schema => ['array*' => {of=>'num*', min_len=>1}],
pos => 0,
greedy => 1
},
},
};
sub multiply_many {
my %args = @_;
my $nums = $args{nums};
my $ans = 1;
$ans *= $_ for @$nums;
[200, "OK", $ans];
}After wrapping, in positional mode it can then be called:
multiply_many(2, 3, 4); # 24which is the same as (in normal named-argument style):
multiply_many(nums => [2, 3, 4]); # 24In command-line:
% multiply-many 2 3 4in addition to the normal:
% multiply-many --nums '[2, 3, 4]'completion. This argument specification key specifies how to complete argument value (e.g. in shell or Riap::HTTP) and is supplied an anonymous function as value. The function will be called with arguments: word=>... (which is the formed word so far), args=>... (which is the args property value). The function should return an array containing a list of possible candidates. For an example of implementation for this, see Perinci::BashComplete in Perl which provides tab completion for argument values. Example:
$SPEC{delete_user} = {
v => 1.1,
args => {
username => {
schema => 'str*',
pos => 0,
completion => sub {
my %args = @_;
my $word = $args{word} // "";
# find users beginning with $word
local $CWD = "/home";
return [grep {-d && $_ ~~ /^\Q$word/} <*>];
},
},
force => {schema=>[bool => {default=>0}]},
},
};When delete_user is executed over the command line and the Tab key is pressed:
$ delete-user --force --username fo<tab>
$ delete-user fo<tab>then bash will try to complete with usernames starting with fo.
Specify in what form the function expects the arguments. The value is actually implementation-specific since it describes the function implementation. For example in Perinci for Perl, these values are recognized: array, hash, arrayref, hashref. This property is useful for wrapper to be able to convert one form to another.
The default value is also left to the implementation.
For interimplementation communication (e.g. via Riap::HTTP or Riap::TCP), named arguments are always used so this property is irrelevant.
Specify function return value. It is a hash containing keys:
schema
A Sah schema to validate the result
summary
Like the
summaryproperty in variable metadata.description
Like the
descriptionproperty. Suggested to be formatted to 78 columns.
Note that since functions normally return enveloped result, instead of returning:
RESULTyour functions normally have to return an enveloped result:
[STATUS, MESSAGE, RESULT, METADATA]Examples:
# result is an integer
result => {schema => 'int*'}
# result is an integer starting from zero
result => {schema => ['int*' => {ge=>0}]}
# result is an array of records
result => {
summary => 'Matching addressbook entries',
schema => ['array*' => {
summary => 'blah blah blah ...',
of => ['hash*' => {allowed_keys=>[qw/name age address/]} ]
}]
}If set to true, specify that function does not envelope its results. The default is false, to encourage functions to create envelopes. However, wrapper should be able to create or strip envelope if needed. For example, if you have "traditional" functions which does not do envelopes, you can set this property to true, and the wrapper can generate the envelope for the functions.
This property allowed you to put examples in a detailed and structured way, as an alternative to putting everything in description.
Each example shows what arguments are used, what the results are, and some description. It can be used when generating API/usage documentation, as well as for testing data. It is expressed a hash containing these keys:
args => HASH
Arguments used to produce result.
argv => ARRAY
An alternative to
args, for example when function is run from the command line.status => INT
Status from envelope. If unspecified, assumed to be 200.
result => DATA
Result data.
summary => STR
A one-line summary of the example, much like
summaryproperty. You should describe, in one phrase or sentence, what the example tries to demonstrate. You can skip the summary if the example is pretty basic or things are already clear from theargsalone.description => STR.
Longer marked up text about the example (e.g. discussion or things to note), suggested to be formatted to 72 columns.
Example:
# part of metadata for Math::is_prime function
examples => [
{
args => {num=>10},
result => 0,
# summary no needed here, already clear.
},
{
argv => [-5],
result => 1,
summary => 'Also works for negative integers',
},
{
args => {},
result => 400,
summary => 'Num argument is required',
},
],The features property allows functions to express their features. Each hash key contains feature name, which must only contain letters/numbers/underscores.
Below is the list of defined features. New feature names may be defined by extension.
feature: reverse => BOOL
Default is false. If set to true, specifies that function supports reverse operation. To reverse, caller can add special argument
-reverse. For example:$SPEC{triple} = { v => 1.1, args => {num=>{schema=>'num*'}}, features => {reverse=>1} }; sub triple { my %args = @_; my $num = $args{num}; [200, "OK", $args{-reverse} ? $num/3 : $num*3]; } triple(num=>12); # => 36 triple(num=>12, -reverse=>1); # => 4NOTE: Abilitity to express conditional reversibility is considered.
feature: undo => BOOL
Default is false. If set to true, specifies that function supports undo operation. Undo is similar to
reversebut needs some state to be saved and restored for do/undo operation, while reverse can work solely from the arguments. Please see "The undo protocol" for details on undo.feature: dry_run => BOOL
Default is false. If set to true, specifies that function supports dry-run (simulation) mode. Example:
use Log::Any '$log'; $SPEC{rmre} = { summary => 'Delete files in curdir matching a regex', args => {re=>{schema=>'str*'}}, features => {dry_run=>1} }; sub rmre { my %args = @_; my $re = qr/$args{re}/; my $dry_run = $args{-dry_run}; opendir my($dir), "."; while (my $f = readdir($dir)) { next unless $f =~ $re; $log->info("Deleting $f ..."); next if $dry_run; unlink $f; } [200, "OK"]; }The above Perl function delete files, but if passed argument
-dry_run=> 1 (simulation mode), will not actually delete files, only display what files match the criteria and would have be deleted.feature: pure => BOOL
Default is false. If set to true, specifies that function is "pure" and has no "side effects" (these are terms from functional programming / computer science). Having a side effect means changing something, somewhere (e.g. setting the value of a global variable, modifies its arguments, writing some data to disk, changing system date/time, etc.) Specifying a function as pure means, among others:
the function needs not be involved in undo operation;
you can safely include it during dry run;
feature: immutable => BOOL
Default is false. If set to true, specifies that function always returns the same result when given the same argument values. This enables optimization like memoization. An example of an immutable function is
sub { $_[0]+$_[1] }where its results only depend on the arguments. Example of an mutable function would berand().
This property specifies function's dependencies to various things. It is a hash of dep types and values. Some dep types are special: all, any, and none.
deps => {
DEPTYPE => DEPVALUE,
...,
all => [
{DEPTYPE=>DEPVALUE, ...},
...,
},
any => [
{DEPTYPE => DEPVALUE, ...},
...,
],
none => [
{DEPTYPE => DEPVALUE, ...},
....,
],
}A dependency can be of any type: another function, environment variables, programs, OS software packages, etc. It is up to the dependency checker library to make use of this information.
For the dependencies to be declared as satisfied, all of the clauses must be satisfied.
Below is the list of defined dependency types. New dependency type may be defined by an extension.
dep: env => STR
Require that an environment variable exists and is true, where true is in the Perl sense (not an empty string or "0"; " " and "0.0" are both true). Example:
env => 'HTTPS'dep: prog => STR
Require that a program exists. If STR doesn't contain path separator character '/' it will be searched in PATH. Windows filesystem should also use Unix-style path, e.g. "C:/Program Files/Foo/Bar.exe".
prog => 'rsync' # any rsync found on PATH prog => '/bin/su' # won't accept any other sudep: code => CODE
Require that anonymous function returns a true value after called, where the notion of true depends on the host language. Example in Perl:
code => sub {$>} # i am not being run as rootExample in Ruby:
"code" => Proc.new { Process.euid > 0 } # i am not being run as rootdep: undo_storage => STORAGE_SPEC
Specify that function requires some external undo storage. STORAGE_SPEC is a hash. Currently the only known keys are:
tmp_dirwith the value of 1,trash_dirwith the value of 1 (trash_dir is basically the same as tmp_dir, but provides clue to the undo/transaction manager that the directory is used for trash, so it can arrange directory to be in the same partition, etc.)Example:
undo_storage => {trash_dir=>1}dep: all => [DEPHASH, ...]
A "meta" type that allows several dependencies to be joined together in a logical-AND fashion. All dependency hashes must be satisfied. For example, to declare a dependency to several programs and an environment variable:
all => [ {prog => 'rsync'}, {prog => 'tar'}, {env => 'FORCE'}, ],dep: any => [DEPHASH, ...]
Like
all, but specify a logical-OR relationship. Any one of the dependencies will suffice. For example, to specify requirement to alternative Perl modules:or => [ {perl_module => 'HTTP::Daemon'}, {perl_module => 'HTTP::Daemon::SSL'}, ],dep: none => [DEPHASH, ...]
Specify that none of the dependencies must be satisfied for this type to be satisfied. Example, to specify that the function not run under SUDO or by root:
none => [ {env => 'SUDO_USER' }, {code => sub {$> != 0} }, ],Note that the above is not equivalent to below:
none => [ {env => 'SUDO_USER', code => sub {$> != 0} }, ],which means that if none or only one of 'env'/'code' is satisfied, the whole dependency becomes a success (since it is negated by 'none'). Probably not what you want.
If you add a new language-specific dependency type, please prefix it with the language code, e.g. perl_module, perl_func, ruby_gem, python_egg. These dependency types have also been defined by some existing tools: deb (dependency to a Debian package), rpm (dependency to an RPM package), js_url (loading a remote JavaScript script URL), file (existence of a), perl_run_func (running a Perl subroutine and getting a successful enveloped result). Some of these might be declared as part of the core dependency types in the future.
This section describes an undo protocol, which is basically the non-OO version of the command pattern. The benefits of this protocol include: applicability to normal functions (no need to declare classes); ease of implementation of undo/transaction manager and remote operation (due to explicit transfer of undo data as normal data structure).
This undo protocol is centered around functions. The responsibility of implementing the undo functionality rests on each function. Not all functions needs to support undo. Those that do must advertise this fact by setting its undo feature to true.
Performing do. To indicate that we need undo, we call function by passing special argument -undo_action with the value of do. Function should perform its operation and save undo data along the way. If -undo_action is not passed or false/undef, function should assume that caller do not need undo later, so function need not save any undo data. After completing operation successfully, function should return status 200, the result, and undo data. Undo data is returned in the result metadata (the fourth element of result envelope), example:
[200, "OK", $result, {undo_data=>$undo_data}]Undo data should be serializable so it is easy to be made persistent if necessary (e.g. by some undo/transaction manager).
Performing undo. To perform an undo, caller must call the function again with the same previous arguments, except -undo_action should be set to undo and -undo_data set to undo data previously given by the function. Function should perform the undo operation using the undo data. Upon success, it must return status 200, the result, and an undo data (i.e., redo data, since it can be used to undo the undo operation).
Performing redo. To perform redo, caller can call the function again with <-undo_action> set to undo and -undo_data set to the redo data given in the undo step. Or, alternatively, caller can just perform a normal do (see above).
An example:
$SPEC{setenv} = {
v => 1.1,
summary => 'Set environment variable',
args => {
name => {req=>1, schema=>'str*'},
value => {req=>1, schema=>'str*'},
},
features => {undo=>1},
};
sub setenv {
my %args = @_;
my $name = $args{name};
my $value = $args{value};
my $undo_action = $args{-undo_action} // '';
my $undo_data = $args{-undo_data};
my $old;
if ($undo_action) {
# save original value and existence state
$old = [exists($ENV{$name}), $ENV{$name}];
}
if ($undo_action eq 'undo') {
if ($undo_data->[0]) {
$ENV{$name} = $undo_data->[1];
} else {
delete $ENV{$name};
}
} else {
$ENV{$name} = $value;
}
[200, "OK", undef, $undo_action ? {undo_data=>$old} : {}];
}The above example declares an undoable command setenv to set an environment variable (%ENV).
To perform command:
my $res = setenv(name=>"DEBUG", value=>1, -undo_action=>"do");
die "Failed: $res->[0] - $res->[1]" unless $res->[0] == 200;
my $undo_data = $res->[3]{undo_data};To perform undo:
$res = setenv(name=>"DEBUG", value=>1,
-undo_action="undo", -undo_data=>$undo_data);
die "Can't undo: $res->[0] - $res->[1]" unless $res->[0] == 200;After this undo, DEBUG environment variable will be set to original value. If it did not exist previously, it will be deleted.
To perform redo:
my $redo_data = $res->[3]{undo_data};
$res = setenv(name=>"DEBUG", value=>1,
-undo_action="undo", -undo_data=>$redo_data);or you can just do:
$res = setenv(name=>"DEBUG", value=>1, -undo_action="do");See also: Setup, an attempt for convention of undo data and writing undoable function.
Although the complete undo data can be returned by the function in the undo_data result metadata key, sometimes it is more efficient to just return a pointer to said undo data, while saving the actual undo data in some external storage.
For example, if a function deletes a big file and wants to save undo data, it is more efficient to move the file to trash directory and return its path as the undo data, instead of reading the whole file content and its metadata to memory and return it in undo_data result metadata.
It is sometimes desirable to let an external manager (an undo/transaction manager) manage this external storage, for the following functionalities: storage limits (e.g. maximum number of allowed undo steps, maximum size of trash), permissions, etc.
Functions which require an external storage should specify this in its metadata, through the undo_storage dependency clause. For example:
deps => {
...
undo_storage => {trash_dir=>1},
}When calling function, undo/transaction manager should provide appropriate references to required undo storage in the special argument -undo_storage. Example:
-undo_storage => {
trash_dir=>"/home/.trash/2fe2f4ad-a494-0044-b2e0-94b2b338056e",
}TBD: After using an external storage, to assist undo/transaction manager, function should also indicate this fact via the undo_storage (or undo_storage_usage?). For example: undo_storage_usage=>{trash_dir=>1}.
Application undo and transaction functionality can be built upon the undo protocol. Transaction and rollback can be implemented by grouping function calls and storing undo data of each call. There should be an undo/transaction manager with the responsibility of maintaining the list of transactions and each transaction's list of function calls and undo data.
See also: Riap::Transaction which specifies undo/transaction over Riap.
Example:
{
args => {
src => {
summary => "Source path",
description => "...",
schema => ["str*", {
summary => "...",
description => "...",
...
}],
...
},
dest => {
summary => "Target path",
description => "...",
schema => ["str*", {
summary => "...",
description => "...",
...
}],
...
},
...
},
}As you can see, each argument has a summary and description, but the schema for each argument also has a summary and description schema clauses. What are the difference and which should be put into which?
The argument specification's summary (and description) describe the argument itself, in this example it says that src means "The source path" and dest means "The target path". The argument schema's summary (and description) describe the data type and valid values. In this example it could say, e.g., "a Unix-path string with a maximum length of 255 characters". In fact, src and dest are probably of the same type ("Unix path") and can share schema.
{
...
args => {
src => {
...
schema => "unix_path",
},
dest => {
...
schema => "unix_path",
},
...
},
}Example:
# Note: remember that in Sah, str* is equivalent to [str => {req=>1}]
args => {
a => { schema=>"str" },
b => { schema=>"str*" },
c => { req=>1, schema=>"str" },
d => { req=>1, schema=>"str*" },
}In particular look at b and c. b is not a required argument (no req=>1 in the argument spec) but if it is specified, than it cannot be undef/null (since the schema says [str=>{req=>1}], a.k.a "str*"). On the other hand, c is a required argument (req=>1 in the argument spec) but you can specify undef/null as the value. The following are valid:
func(c=>undef, d=>1);But the following are not:
func(b=>1, d=>1); # c is not specified
func(b=>undef, c=>1, d=>1); # b has undef value
func(b=>1, c=>1, d=>undef); # d has undef valueShould I add a new metadata property, or add a new feature name to the features property, or add a new dependency type to the deps property?
If your property describes a dependency to something, it should definitely be a new dependency type. If your property only describes what the function can do and does not include any wrapper code, then it probably goes into features. Otherwise, it should probably become a new metadata property.
For example, if you want to declare that your function can only be run under a certain moon phase (e.g. full moon), it should definitely go as a new dependency type, so it becomes: deps => { moon_phase => 'full' }.
Another example, reverse is a feature name, because it just states that if we pass -reverse => 1 special argument to a reversible function, it can do a reverse operation. It doesn't include any wrapper code, all functionality is realized by the function itself. On the other hand, timeout is a metadata property because it involves adding adding some wrapping code (a timeout mechanism, e.g. an eval() block and alarm() in Perl).
Data::Sah
Rinci