Skip to content
All you need to generate the documentation in a specific format.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.circleci adds new clean option Jul 16, 2019
.github/ISSUE_TEMPLATE
bin setup option and refactor of CLI Jul 13, 2019
lib/Perl6 adds new clean option Jul 16, 2019
resources move everything to resources dir, close #42 Jul 13, 2019
t fix test and updates CI, #46 Jul 14, 2019
.gitignore updates .gitignore Jul 13, 2019
.travis.yml use jj docker container for testing Jun 8, 2019
Changes init using App::Mi6 May 22, 2019
LICENSE init using App::Mi6 May 22, 2019
META6.json
Makefile adds makefile to init highlighting #31 Jul 7, 2019
README.md adds new clean option Jul 16, 2019
app-start move server files to make it work #19 Jul 3, 2019
app.pl move server files to make it work #19 Jul 3, 2019

README.md

Perl6 Doc Tools CI Status Build Status artistic

In this repository you can find all logic responsible of generate the official documentation of Perl6.

Table of contents

Installation

$ zef install Perl6::Documentable

Usage

Before generate any documents you should execute:

documentable setup

in order to download the necessary files to generate the site (CSS, svg, ...). Alternatively, you can add your own.

# setup the working directory
documentable setup

# generate the documentation
documentable start [--topdir=<Str>] [-v|--verbose] [-c|--cache] [-p|--pods] [-s|--search-index]
                  [-i|--indexes] [-t|--type-images] [-f|--force] [-a|--all] [--highlight]
                  [-k|--kind] [--manage]

# update the documentation when a pod file has been modified
documentable update [--topdir=<Str>]

# clean the files generated by documentable setup
documentable clean

--topdir

Directory where the pod files are stored. Set to doc by default.

-v, --verbose

Be more verbose/talkative during the operation. Useful for debugging and seeing what's going on "under the hood".

-c, --cache

Usage of a cache of pod (cached used). Set to True by default.

-p, --pods

Generate one HTML page for every pod file.

-s, --search-index

Generate the search file search.js.

-i, --indexes

Generate all index files.

-t, --type-images

Generate and write (if necessary) all typegraph svg files.

-f, --force

If specified, typegraph svg files will be forcibly generated and written.

--highlight

If specified, code blocks will be highlighted using atom-language-perl6.

-k, --kind

If specified, kind files will be written for syntax and routine.

--manage

Flag to sort the Language page in sections, as described in language-order-control.json.

-a, --all

Equivalent to -p -s -i -t.

Perl6::Documentable

SYNOPSIS

use Perl6::Documentable;
use Perl6::Documentable::Processing;
use Pod::Load;

# get a pod source
my $pod = load("some-amaizing-pod-file.pod6")[0];

# create the first Documentable object from a pod
my $doc = process-pod-source(
    kind => "type",
    pod  => $pod,
    filename => "some-amazing-pod-file"
);

# and use it!
$doc.defs;

DESCRIPTION

Perl6::Documentable represents a piece of Perl 6 that is documented. It contains meta data about what is documented (for example (kind => 'type', subkinds => ['class'], name => 'Code') and in $.pod a reference to the current documentation.

Perl6::Documentable

    has Str  $.kind;
    has Str  @.subkinds;
    has Str  @.categories;

    has Str  $.name;
    has Str  $.url;
    has      $.pod;
    has Bool $.pod-is-complete;
    has Str  $.summary = '';

    has $.origin;

    has @.defs;
    has @.refs;

Str $.kind

This value is highly used in the documentation process, it can take six different values: [language programs type syntax reference routine].

The first three ones are only set when the source pod files are processed (see process-pod-dir and process-pod-source). In this case, the value of kind can tell you where this pod source comes from.

syntax and routine values are only set by find-definitions, when a new definition is found.

reference is only set by register-reference, whenever a new reference element is found.

Str @.subkinds

Can take one of the following values: <infix prefix postfix circumfix postcircumfix listop sub method term routine trait submethod constant variable twigil declarator quote>.

In addition, it can take the value of the meta part in a X<> definition (see find-definitions).

Str @.categories

It is assigned with what parse-definition-headerreturns. Its value will be one of this list.

In addition, if the Perl6::Documentable object comes from a type pod source, this category will be replaced by the information given by Perl6::TypeGraph (see process-pod-source).

Str $.name

Name of the Pod. It is usually set to the filename without the extension .pod6.

See parse-definition-header returns).

Str $.url

Work in progress :D.

$.pod

Perl6 pod structure.

Bool $.pod-is-complete

Indicates if the Pod represented by the Perl6::Documentable object is completed. A Pod is completed if itrepresents the whole pod source, that's to say, the $.pod attribute contains an entire pod file.

It will be considered incomplete if the Perl6::Documentable object represents a definition or a reference. In that case the $.pod attribute will only contain the part corresponding to the definition or the reference.

Str $.summary

Subtitle of the pod.

=begin pod :tag<tutorial>

=TITLE  Perl 6 by example

=SUBTITLE A basic introductory example of a Perl 6 program

...

=end pod

In this case $summary="A basic introductory example of a Perl 6 program".

$.origin

Documentable object that this one was extracted from, if any. This is used for nested definitions. Let's see an example:

=begin pod

Every one of this valid definitions is represented by a Perl6::Documentable object after being processed.

=head1 method a

=head2 method b

    In both cases $origin points to the Documentable object representing
    the method a.

=head2 method c

=head1 method d

    $origin in this case points to the Perl6::Documentable object
    containing the pod source.

=end pod

Two or more definitions are nested if they appears one after another and if the second one has a greater heading level than the second one.

Array @.defs

It contains all definitions found in $.pod. All of them have pod-is-complete set to false.

See find-definitions for more information.

Array @.refs

It contains all references found in $.pod. All of them have pod-is-complete set to false.

See find-references for more information.

method human-kind

method human-kind(
) return Str

Returns the transformation of $.kind to a "more understable" version. That means:

  • If $.kind is equal to language then it returns language documentation.
  • Otherwise, if @.categories is equal to operator then is set to @.subkinds operator. If not, it's set to the result of calling english-list with @.subkinds or $.kind if the previous one is not defined.

Examples:

declarator
do (statement prefix)

Note: english-list is a helper function which join a list using commas and add a final "and" word:

my @a = ["a","b","c"];
english-list(@a) # OUTPUT: a, b and c

method url

method url(
) return Str

Work in progress :D.

method categories

method categories(
) return Array

Returns @.categories. If @.categories it's not defined, sets @.categories to the same value as @.subkinds and returns it.

method get-documentables

method get-documentables(
) returns Array

Returns all Perl6::Documentable objects (@.defs+@.refs).

Perl6::Documentable::Registry

    has @.documentables;
    has Bool $.composed = False;
    has %!cache;
    has %!grouped-by;
    has @!kinds;
    has %!routines-by-type;
    has Bool $.verbose;

@.documentables

If it's not composed, it will contain only Perl6::Documentable objects with pod-is-complete set to True, that means, coming from entire pod files. After being composed, it will contain all Perl6::Documentable objects obtained from processing the previous ones (@.defs and @.refs).

See processing-pod-source.

Bool $.composed

Boolean indicating if the registry is composed. See compose.

%!cache

This Hash object works as a cache for lookup method.

See lookup for more information.

%!grouped-by

This Hash object works as a cache for grouped-by method.

See grouped-by for more information.

@!kinds

Array containing all different values of $kinds found in @!documentables. Set when compose is called.

$.tg

Instance of a Perl6::TypeGraph object (from this Perl6::TypeGraph module).

This object is responsible of giving us the correct categories and subkinds of a type. For instance, it sets the category of a type to domain-specific, exception, etc., which is used in the index generation.

%!routines-by-type

Hash object with the following structure:

  • key => name of the type (filename without the .pod6 extension)
  • value => all Perl6::Documentable objects with kind equal to routine and the name as the key.

See compose to see how it's initialized.

Bool $.verbose

Useful information will be printed if set to True (False by default).

submethod BUILD

submethod BUILD(
    Bool :$verbose?,
) returns Perl6::Documentable::Registry

In this method the attribute $.tg is configured as specified in the module documentation.

If $.verbose, then useful information will be printed.

Processing methods

method add-new

method add-new(
    Perl6::Documentable $d
) return Perl6::Documentable;

Adds a new Perl6::Documentable object to the registry. Returns the same object added.

Example:

use Perl6::Documentble::Processing;
use Pod::Load;

my $doc = processing-pod-source(
    pod      => load("pod.pod6"),
    origin   => Perl6::Documentable.new,
    filename => "pod"
);

my $registry = Perl6::Documentable::Registry.new;
$registry.add-new: $doc;

method compose

method compose(
) return Boolean;

This methods does several things:

  1. Initialize @!kinds and join all Perl6::Documentable objects found in every element of @!documentables (that means all @.defs and @.refs attributes).
  2. Compose every Perl6::Documentable object in @.documentables with kind set to type, calling compose-type.
  3. %!routines-by-type is set, grouping the Perl6::Documentable collection by kind="routine" using lookup. Then the result is classified by name. You may think that after this process, %!routines-by-type contains more things, apart from the "routines by type". You are right! But we do not care because we will consult this Hash using the names of the types, so we will get what we want anyway.
  4. Finally, $!composed is set to True and returns it.
use Perl6::Documentable::Processing;

my $registry = process-pod-collection(
    :cache,
    :verbose,
    :topdir("doc"),
    :dirs(["Language", "Type", "Programs", "Nativ" ])
);

# once we have processed all the pods we can compose it
$registry.compose;

# now all definitions and references are also available through
# @.documentables (and the original ones too)
say $registry.documentables;

method compose-type

method compose-type(
    Perl6::Documentable $doc
) return Mu;

This method is responsible of completing the pod of those Perl6::Documentable objects with kind set to type.

This method use the Perl6::TypeGraph object stored in $!tg, so if a type with the same name as the Perl6::Documentable object is not found in $!tg.types, it will simply return.

Otherwise, three things will be added in this order:

  1. A typegraph fragment generated by typegraph-fragment.
  2. All routines from the roles made by the type (using %!routuines-bytype, see compose).
  3. All routines from the class which the type inherits from and the routines of the roles made by those classes (using %!routuines-bytype, see compose).

method typegraph-fragment

method typegraph-fragment(
    Str $podname
) return Array[Pod::Block];

Given a $podname (filename with .pod6 extension), it returns a pod fragment containing a heading ("Type Graph") and the file template/tg-fragment.html correctly initialized.

Consulting methods

method get-kinds

method get-kinds(
) return Array

Returns an array containing all kind values after processing the pod collection. Currently there are only 6 different values: [language programs type syntax reference routine].

method grouped-by

method grouped-by(
    Str $what
) returns Array[Perl6::Documentable];

The first time it is called it initializes a key $what, with the result of classifying @!documentables by $what. $what needs to be the name of an attribute of a Perl6::Documentable object: kind, for instance.

This result is stored in %!grouped-by so the next time it's called it will be faster.

method lookup

method lookup(
    Str $what,
    Str $by
) returns Array[Documentable];

This method uses %!cache, which is a two-layer Hash object. That means you first consult it with one key, $by, and that returns another Hash which is consulted with the key $what.

So, $by has to be the name of an attribute of Perl6::Documentable class. Elements in @!documentables will be classified following that attribute. Then, $what must be one of the possible values that the attribute $by can take.

In this setting, lookup will return the Perl6::Documentable objects in @!documentables whose attribute $by is equal to $what.

This result is stored in %!cache so the next time it's called it will be faster.

my $registry = (... initialized previously ...);

# all Perl6::Documentable objects with kind set to type
$registry.lookup("type", :by<kind>)

# all Perl6::Documentable objects with name set to close
$registry.lookup("close", :by<name>)

Indexing methods

All *-index methods are used to generate the main index in the doc site (Language, Types, ...).

method programs-index

method programs-index(
) return Array[Hash];

It takes all Perl6::Documentable objects in the Perl6::Documentable::Registry with kind set to programs. After that it makes a map and creates the following Hash for each one:

%(
    name    => ...
    url     => ...
    summary => ...
)

Note: ... means that is the attribute of the Perl6::Documentable.

method language-index

method language-index(
) return Array[Hash];

It takes all Perl6::Documentable objects in the Perl6::Documentable::Registry with kind set to language. After that it makes a map and creates the following Hash for each one:

%(
    name    => ...
    url     => ...
    summary => ...
)

Note: ... it means that is the attribute of the Documentable.

method type-index

method type-index(
) return Array[Hash];

It takes all Perl6::Documentable objects in the Perl6::Documentable::Registry with kind set to type. After that it makes a map and creates the fololwing Hash for each one:

%(
    name     => ...
    url      => ...
    subkinds => ...
    summary  => ...
    subkind  => first subkind
)

Note: ... means that is the attribute of the Documentable.

method type-subindex

method type-subindex(
  Str :$category
) return Array[Hash];

Same as type-index but you can filter by $category. You can pass one of the following categories: <basic composite domain-specific exceptions>.

method routine-index

method routine-index(
) return Array;

It takes all Perl6::Documentable objects in the Perl6::Documentable::Registry with kind set to routine. After that it makes a map and creates the following Hash for each one:

%(
    name     => ...
    url      => ...
    subkinds => ...
    origins  => $from
)

Where $from is an array of [$name, $url] containing the names and urls of the Perl6::Documentable objects where the routine was found.

Note: ... means that is the attribute of the Documentable.

method routine-subindex

method routine-subindex(
  Str :$category
) return Array[Hash];

Same as routine-index but you can filter by $category. You can pass one of the following categories: <sub method term operator trait submethod>.

method generate-search-index

method generate-search-index(
) return Array[Str];

Returns an array whose items are in the following format: {category: "", value: "", url: ""}.

This array is initialized calling lookup with every possible value of kind (see get-kinds). category is set to kind or to subkind if lookup returns more than one Documentable object. value is set to the name of the Documentable and url too.

You can see the current search index here.

Perl6::Documentable::Processing

This is the module where all processing happens. It takes a pod collection and returns a Perl6::Documentable::Registry object completely initialized with all the necessary information.

sub process-pod-collection

sub process-pod-collection(
    Bool       :$cache,
    Bool       :$verbose,
    Str        :$topdir,
    Array[Str] :@dirs
) return Perl6::Documentable::Registry

Main function of this module. Process every directory in @dirs (with process-pod-dir) and add the results to the new Perl6::Documentable::Registry object.

If :$cache is True, pods will be extracted from a cache, if possible.

Example:

my $registry = process-pod-collection(
    cache => True,
    verbose => True,
    topdir => "doc",
    dirs => ["Language", "Programs", "Type", "Native"]
); # done! now you only need to compose it!

sub process-pod-dir

sub process-pod-dir(
    Str  :$topdir,
    Str  :$dir,
    Sub  :&load = configure-load-function(:!cache),
    Bool :$verbose
) return Array[Perl6::Documentable]

Process all pod files in $topdir/$dir. Load them with &load without using a cache (by default).

It returns an array containing all [Perl6::Documentable] objects (one per file) resulting process-pod-source.

Example:

my @documentables = process-pod-dir(
    topdir  => "doc",
    dir     => "Type",
    verbose => False # no output
); # without cache

my @documentables = process-pod-dir(
    topdir  => "doc",
    load    => configure-load-function(:cache),
    dir     => "Type",
    verbose => True   # informative output
); # using cache

sub process-pod-source

sub process-pod-source(
    Str        :$kind,
    Pod::Named :$pod,
    Str        :$filename
) return Perl6::Documentable

Given a pod, returns the associated Perl6::Documentable object, correctly initialized.

How it is initialized?

  • $name is set to $filename by default. If a =TITLE element is found, then it is set to its contents. In addition, if $kind is type, $name will be set to the last word of that content.
  • $summary is set to the content of the first =SUBTITLE element.
  • $pod-is-complete is set to True (becuase it's a complete pod).
  • $url is set to /$kind/$link, where $link is set to $filename if a link value is not set in the pod configuration.
  • $kind and $subkinds are set to $kind.
  • @.defs and @.refs are set by find-definitions and find-references respectively.

sub parse-definition-header

sub parse-definition-header(
    Pod::Heading :$heading
) return Hash

This method takes a Pod::Heading object and return a non-empty hash if it's a definition. This hash is something like this:

%(
    name       => ...
    kind       => ...
    subkinds   => ...
    categories => ...
)

The value of these parameters depends on the definition found, these are the four different types you will find:

=head2 The <name> <subkind>
Example:
=head2 The C<my> declarator
=head2 <subkind> <name>
Example:
=head2 sub USAGE
=head X<<name>|<subkind>>

First two types are parsed by Perl6::Documentable::Processing::Grammar and the last one does not need to be parsed because it will be considered a valid definition no matter what you type inside.

sub determine-subkinds

sub determine-subkinds(
    Str $name,
    Str $origin-name,
    Str $code
) return Array[Str]

This function returns the proper subkinds of a function based on its definition code (the one you find after the Defined as).

This is necessary because some functions are declared with several signatures.

sub find-definitions

sub find-definitions(
    Array                      :$pod,
    Perl6::Documentable        :$origin,
    Int                        :$min-level = -1,
    Array[Perl6::Documentable] :@defs
) return Int

This function takes a pod and returns an array of Perl6::Documentable objects containing all definitions found in the pod (to find valid definitions it uses parse-definition-header). It runs through the pod content and looks for valid headings @defs is populated recursively.

When we find a new definition, a new Perl6::Documentable object is created and initialized to:

  • $origin: to $origin.
  • $pod: It will be populated with the pod section corresponding to the definition and its subdefinitions (all valid headers with a greater level until one with the same or lower level is found).
  • $pod-is-complete: to false beacuse it's a definition.
  • name, kind, subkinds and categories to the output of parse-definition-header.

Example:

use Pod::Load;

my $pod = load("type/Any.pod6").first;
my $origin = Perl6::Documentable.new(:$pod);

my @documentables;
find-definitions(
    pod => $pod,
    :$origin,
    :defs(@documentables)
)

# this array contains all definitions found
say @documentables;

sub find-references

sub find-references(
    Array                      :$pod,
    Str                        :$url,
    Perl6::Documentable        :$origin,
    Array[Perl6::Documentable] :@refs
) return Mu

Quite similar to find-definitions. It runs through the entire pod looking for all X<> elements.

It goes through all the pod tree recursively searching for X<> elements (Pod::FormattingCode). When one is found, register-reference is called with the pod fragment associated to that element, the same origin and the associated url:

$url ~ '#' ~ index-entry-$pod.meta-$index-text

sub create-references

sub create-references(
    Pod::FormatingCode  :$pod,
    Perl6::Documentable :$origin,
    Str                 :$url
) return Array[Perl6::Documentable]

Every time it's called it creates a Perl6::Documentable object, with $kind and $subkinds set to references. name is taken from the meta part: the last element becomes the first and the rest are written right after.

Example:

Given the meta ["some", "reference"], the name would be: "reference (some)"

That's done for every element in meta, that means, if meta has 2 elements, then 2 Perl6::Documentable objetcs will be added (you can specify several elements in a X<> using ;).

If there is no meta part, then the pod content is taken as name.

Perl6::Documentable::Update

This module is responsible of updating the HTML documents of those files that have changed.

sub update-pod-collection

sub update-pod-collection(
    Str        :$topdir,
    Array[Str] :$filenames
) return Mu

Updates all HTML documents in filenames using update-file.

sub update-indexes

sub update-indexes(
    Array[Str]                      @kinds,
    Perl6::Documentable::Registry   $registry
) return Mu

Regenerates those indexes related to a given kinds.

sub update-file

sub update-file(
    Str                           $filename,
    Perl6::Documentable::Registry $registry
) return Mu

Given the name of a modified file, regenerates and rewrite all HTML documents related/coming from this file.

sub update-per-kind-files

sub update-per-kind-files(
    Str                 $kind,
    Perl6::Documentable $doc,
    Hash                %documentables
) return Mu

Given a kind and a Perl6::Documentable object, regenerates and rewrites all files related to that kind, related to $doc.

sub update-registry

sub update-registry(
    Str :$topdir
) return Perl6::Documentable::Registry

Reprocess the pod collection and returns an updated Perl6::Documentable::Registry object.

Perl6::Documentable::To::HTML::Wrapper

This module takes a pod and wrap it in our HTML template, adding a header and a footer. Each and every one of the files generated comes from p2h.

sub header-html

sub header-html(
    Str $current-selection,
    Str $pod-path
) return Str;

Returns the HTML header for every page. $current-selection has to be set to one element of the menu. If that element has a submenu, it will be created too.

$pod-path is the path relative to doc with the extension .pod6. Used in the edit buttom url.

sub footer-html

sub footer-html(
    Str $pod-path
) return Str;

Returns the HTML footer for every page. $pod-path is the path relative to doc with the extension .pod6. Used to the edit buttom url.

sub p2h

sub p2h(
    Pod::Block $pod,
    Str $selection,
    Str $pod-path
) return Str

Takes a pod a return its HTML page. $selection is an element from @menu, it indicates what field of the menu must appear be selected.

Perl6::Documentable::To::HTML

sub *-index-html

sub *-index-html(
    Array[Hash] @index
) return Str

Takes the index generated the Perl6::Documentable::Registry and return its HTML version.

Notes: in language-index-html you have and additional parameter:

sub *-index-html(
    Array[Hash] @index,
    Bool        $manage = False
) return Str

This parameter is used to sort the index elements in a certain way (following the configuration file language-order-control.json).

sub generate-kind-file

sub generate-kind-file(
    Str                        $name,
    Array[Perl6::Documentable] @docs,
    Str                        $kind
) return [$name, Str]

Given a name and a list containing all Perl6::Documentableobjects related to that name, it creates its HTML representation.

sub generate-kind

sub generate-kind(
    Perl6::Documentable::Registry $registry,
    Str                           $kind
) return Array[$name, Str]

Generates all per kind files of $kind. This is done grouping the Perl6::Documentable objects by kind and after that classify them by .name. When this is done, generate-kind-file, once per file.

Perl6::Utils

Some auxiliar functions to ease the job.

sub recursive-dir

sub recursive-dir(
    Str :$dir
) return Array;

This function returns a List of IO objects. Each IO object is one file in $dir.

sub get-pod-names

sub get-pod-names(
    Str :$topdir
    Str :$dir
) return Array;

What does the following array look like? An array of sorted pairs

  • the sort key defaults to the base filename stripped of '.pod6'.
  • any other sort order has to be processed separately as in 'Language'.

The sorted pairs (regardless of how they are sorted) must consist of:

  • key: base filename stripped of its ending .pod6
  • value: filename relative to the "$topdir/$dir" directory

sub pod-path-from-url

sub pod-path-from-url(
    Str $url
) return Str;

Determine the path to source POD from the POD object's url attribute.

sub svg-for-file

sub svg-for-file(
    Str $file
) return Str;

Return the SVG for the given file, without its XML header

For instance, given: t/html/basic.svg, it will return t/html/basic-without-xml.svg.

URL logic

This one is quite a mess, it will be a todo for now.

Resources

Templates

head.html

Some meta info and stylesheet for every page in the doc site.

header.html

Header of every HTML page in the doc site. MENU will be replaced by Perl6::Documentable::To::HTML with the generated menu.

footer.html

Footer for every page in the site. SOURCEURL, SOURCECOMMIT and DATETIME will be replaced by Perl6::Documentable::To::HTML.

tg-fragment.html

Used by typegrah-fragment.

search_template.js

Search funtion for the doc site. It uses Sift 4 algorithm for strings comparison.

Tests

Test files follow this convention:

  • From 0 to 99: basic tests, not related with the core functionality of the module.
  • From 100-199: Perl6::Utils related tests.
  • From 200 to 299: Perl6::Documentable related tests.
  • From 300 to 399: Perl6::Documentable::Registry related tests.
  • From 400 to 499: Perl6::Documentable::Registry::To::HTML related tests.

AUTHORS

Moritz Lenz <@moritz>

Jonathan Worthington <@jnthn>

Richard <@finanalyst>

Will Coleda <@coke>

Aleks-Daniel <@AlexDaniel>

Sam S <@smls>

Alexander Moquin <@Mouq>

Antonio antoniogamiz10@gmail.com

COPYRIGHT AND LICENSE

Copyright 2019 Perl6 Team

This library is free software; you can redistribute it and/or modify it under the Artistic License 2.0.


You can’t perform that action at this time.