Permalink
Switch branches/tags
2016.01-preparation RT123215 RT132710-traits-from-all-multies RT132710-traits-warn-on-non-proto Routine_consolidate_flag_fields actions_shared alpha angular_bracket_literal_semantics arr-con-not-item awesome-malformed-loops better-O better-loop better_line_coverage birdless-2 birdless braids cache_most_common_cds callable-default-value callsite_flags_sso car-grant-unreduce claim-prototype coercer-fixes compiled-hyper-dispatch cont container-compare-metaop cpp cur-candidates curfs-candidates-refactor curly definedness_for_hash_and_arr_sigil dist_resources eval-server-improvements export-constant failed_match2Nil faster-permutations fix-Callable-composition fix-Hash-Mu-keys--RT-1357201 fix-R1723 fix-equiv fix-thunks fix_eval_in_precomp fix_precompilationstore_abstraction froggs_multibyte from_nqp generate_buildallplan generate_buildallplan_2 generate_buildallplan_3 glrish gmr hllbool hotfix-2015.01 hyper_threaded hyper_tracks_sequence_numbers inherit-from-nqp-class interface-inconsistency iterator_mixin_re-use jit_nativecall js jsoff json_timing_stuff jvm-begin-eval jvm-sockets jvm_interop_dispatchers jvmbl language_versions lazy-subst leave less-wrapping lex2loc2 lexical_module_load lexical_require lines-vm-chomp main_named_params master missing-clones moar-gen2-frame-opts moar/reframe monkey-implies-no-precomp more-local-lowering multidim named_pod_vars native-str-ops nativecall-cint nativecall_specialized_sub_bodies new-nil newio ng nil-assign nil-noniterable no-strict no-vm-at-startup no_p5_warnings nom nqp-dogfood nqp-lib-windows nqp-mbc optimize_for_again optimizer_lexicalref_lowering p6for_op parameter-perl parrot-690 parrot-icu-required parrot-iocleanup1-take2 platform_library_name pod-slang-tbrowder pod-table postrelease-opts pr/229 precomp-singleprocess-resurrection precomp-store-redesign priv-role-attrs prune_matches query_repos_old query_repos raccoon ratlab-fattish-rat recursive-gist regex_optimizer release/2015.07.2 relocateable-precomp remove-migration repo_v1 repository_registry rescalar resources return-type-check-plugin return-without-lexotic richer-positional role_diamond rt-127977 rt128156_fix_precomp_deps_validation safely_stringify_core_exceptions scientific-notation-using-div_In segv-coercer-qast setdispatcherfor setops sha1bin sigsp sink-phasers sized-arrays smile spacey spesh-plugins speshplugin_guardstaticcode staged-settings standalone-jar supplier-preserving-refactor support_meta_classes_written_in_perl6 support_perl6_meta_classes there_is_no_return tmp_highfive try-does-use-fatal tune-hash udp_receive_hostname_port undefinitehow-default-defaults unfaster-words unifyunit use-nqp varopt vmarray-list vmarray whenever_last_redo wip-openpipe worry_broken_heredoc_stopper
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
238 lines (173 sloc) 8.48 KB

Notes and hints for working with Rakudo NQP and Pod

Traps for the Perl 6 programmer

  • DO NOT use '$0' in match results - The Perl 6 shorthand for a match variable '$0' doesn't work in NQP. Instead, use $/[0] for the zeroeth match. Note the parser will be very confused otherwise and it currently cannot point to the error.

  • DO NOT use 'nqp::say' - The routine 'say' is an NQP built-in and it does not need the 'nqp::' prefix. You can sometimes get away with using 'nqp::say' but, when you least expect it, the parser will fail without a helpful error message.

  • DO use 'nqp::die' - As opposed to 'say', 'die' does need to be qualified with 'nqp::'. If used without the 'nqp::' prefix, you sometimes may get a very unhelpful error message.

  • BE WARNED about '$<some-match-var>' inside a sub with a '$/' arg - Use the full syntax for a match variable ('/$<some-match-var') for more reliable (or at least self-documenting) results.

  • BE WARNED about '$<a-match-var>' versus '$<a-match-var>**1' - The first form will result in a scalar object while the '**' form will result in an array. Either form may be appropriate for the situation, but proper handling will vary for each.

  • BE WARNED about "return if (...)" statements - Sometimes they work and sometimes not. But the failure message is usually good enough to find the offending code.

For example, all these failed:

return if !nqp::elems(@arr);
return unless nqp::elems(@arr);

but this finally worked:

if !nqp::elems(@arr) {
    return;
}

Pod compilation overview

Pod is parsed as it is discovered during the parsing phase of each compilation unit. Each pod object (string, paragraph, block, configuration, term, heading, item, etc.) is serialized as it is completed, and that result is a QAST node. The appropriate assembly of QAST nodes (which have also been marked as a compile_time_constant) are grouped into instances of pod classes as defined in src/core/Pod.pm6.

Pod block text content handling

Text inside pod blocks that are contents rather than markup is comprised of intermixed text and formatting code characters. Newlines and contiguous whitespace may or may not be significant depending upon the general block type (abbreviated, paragraph, delimited, or declarator) or block identifier (e.g., code, input, output, defn, comment, or data).

The content as it is parsed in Grammar.nqp is first broken into individual characters which are then assigned to one of three token groups: regular text, text with formatting code, and text that is to be unchanged from its input form (code, input, and output).

The regular text and intermingled formatted text are then divided into two more categories: text that will form one or more paragraphs and text that is part of a table. Ultimately, each paragraph of text should be grouped into the @contents array of a single Pod::Block::Para, but not all pod handling per S26 has been fully implemented.

Some notable, not-yet-implemented (NYI) features (in order of one dev's TODO list)

  1. NYI: %config :numbered aliasing with '#' for paragraph or delimited blocks

  2. NYI: pod data blocks

  3. NYI: formatting code in defn block terms

  4. NYI: formatting code in table cells

  5. NYI: pod configuration aliasing

  6. NYI: formatting code in declarator blocks (not described in S26, but a user-requested feature)

  7. NYI: consistent use of the Pod::Block::Para as the leaf parent of all regular text

  8. NYI: pod configuration lines

  9. NYI: Pod::Ambient class

  10. NYI: nested delimited comment blocks

  11. NYI: configuration data on continuation lines are not always handled correctly

Anyone wanting to work on any of the NYI items please coordinate on IRC #perl6-dev to avoid duplicate efforts. Most of the items are being worked on in a generally logical order of need and knowledge gained during the process of implementing pod features.

Pod nesting

Complicating work with pod is that pod blocks can be nested, i.e., a pod block can have pod blocks as children, to any depth. Necessarily that applies, in general, to delimited blocks. (Other block types may have single blocks as children, usually as one or two Pod::Block::Paras.)

One consequence of this is that a pod block with children cannot be created until all its children have been created. Another consequence is that a pod block can have several parts, some of which cannot be created until child components are analyzed or created.

Pod block parts

A pod block can have several parts, all of which must be created before the block itself can be created. Those parts are:

  • Configuration - %.config [all blocks inheriting from class Pod::Block]

    • The configuration cannot be created until the block text data are analyzed.

    • Note that abbreviated blocks cannot have an explicit configuration section, but they may have limited implicit configuration data through use of :numbered aliasing (see below).

  • Contents - @.contents [all blocks inheriting from class Pod::Block]

    • The contents cannot be created until all child blocks are created.
  • Term - $.term [defn blocks]

    • The term cannot be created until the block text data are analyzed.
  • Caption - $.caption [table blocks]

    • The caption cannot be created until the configuration is analyzed.
  • Headers - @.headers [table blocks]

    • The headers cannot be created until the block text data are analyzed.

The <pod_textcontent> token

The token pod_textcontent is the match object for regular text and formatted code as described above. It is the source of the final contents object for regular text containers except for the table blocks which will be discussed separately. It has a corresponding action method.

Tracing the pod class building code is tedious and not well documented. Tokens in the grammar are often made early, along with other objects, and attached to that token's match object's .ast attribute which is then used later in another object. The developer who wants to change the called .ast code in that other object (which may be in the grammar, actions, or src/Perl6/Pod.nqp) has to refer back to the original make point to see its format before doing any changes--not fun! There is an ongoing effort to better document the process for later developers.

Following is the start of a table to show the grammar tokens that have action methods and their resulting .ast values.

Actions method made (ast) value
pod_textcontent
pod_block:sym<delimited> QAST node [instance of Pod::Heading]
pod_block:sym<delimited> QAST node [instance of Pod::Item]
pod_block:sym<delimited> QAST node [instance of Pod::Defn]
pod_block:sym<delimited> QAST node [instance of Pod::Block::Named]
pod_content:sym<block> $<pod_block>.ast;
pod_content_toplevel $<pod_block>.ast

:numbered aliasing

S26 allows for the '#' character (Unicode name NUMBER SIGN), as the first word in a block, to turn on the :numbered %config key; in that case the '#' will be removed from the data. The user can allow a '#' to be recognized as data by either (1) setting the %config numbered key to false, typically with the :!numbered form, or (2) using the V formatting code around the '#' in the data like this: V<#>.

Proper handling of this feature requires changing the block's %config hash after the block data have been parsed or possibly changing the parsing of the first block data word due to the presence of :!numbered in the %config hash. Another problem is how to handle duplicate or incompatible %config keys and values.

The easiest case to handle is the abbreviated block which cannot have explicit %config data and for which the :numbered alias is most useful. Examples of the abbreviated blocks most likely to use this option are the =item, =head, and =defn types.

The '#' turns on the :numbered configuration in all these cases:

=item # foo bar

=item #
foo bar

=item
#
foo bar

Following are examples of situations in other block types that are not good practice but have to be handled gracefully:

=for para :!numbered
# foo bar

The :!numbered is interpreted to mean accepting the '#' as part of block data.

=for para :numbered
# foo bar

The '#' means the same as the :numbered option: the renderer should number the paragraph and the two :numbered keys (one explict and one implicit) are redundant.