Skip to content

Commit

Permalink
Documentation in progress.
Browse files Browse the repository at this point in the history
  • Loading branch information
ColinH committed May 7, 2024
1 parent 0cdcabd commit e013ae1
Show file tree
Hide file tree
Showing 21 changed files with 1,881 additions and 1,449 deletions.
140 changes: 140 additions & 0 deletions doc/Action-Reference.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
# Action Reference

The reference documention for all action classes and class templates.

### Included

Only `tao/pegtl/nothing.hpp` which defines `nothing` and `maybe_nothing` is automatically included with `tao/pegtl.hpp`.
For all other actions the appropriate header in `tao/pegtl/action/` needs to be included manually.

### Functions



### Namespaces

All action classes and class templates reside in namespace `tao::pegtl`.
This default can be changed via the macro `TAO_PEGTL_NAMESPACE` in `tao/pegtl/config.hpp`.


## Contents

* [Nothing](#nothing)
* [Maybe Nothing](#maybe-nothing)
* [Add Guard](#add-guard)
* [Add State](#add-state)
* [Change Action](#change-action)
* [Change Action and State](#change-action-and-state)
* [Change Action and States](#change-action-and-states)
* [Change Control](#change-control)
* [Change State](#change-state)
* [Change States](#change-states)
* [Control Action](#control-action)
* [Disable Action](#disable-action)
* [Enable Action](#enable-action)
* [Require Apply](#require-apply)
* [Require Apply0](#require-apply0)

### Nothing

An action class template that does nothing, simply a *nop* action.

Serves as default `Action` template parameter to `parse()` and `parse_nested()`.

```c++
template< typename Rule >
struct nothing {};
```
Usually also serves as base class for the default case of custom action class templates to indicate that the default is "no action".
```c++
template< typename Rule >
struct my_action
: tao::pegtl::nothing< Rule > {};
// ...plus some (partial) specialisations of my_action
// that define static apply() or apply0() functions.
```

### Maybe Nothing

An action class alias defined as `nothing< void >`.
An action class for `Rule` that is not derived from `nothing< Rule >` but intentionally has no `apply()` or `apply0()` must/should (TODO) derive from `maybe_nohting` to signal that the absence of these functions is not an error.

### Add Guard
add_guard

### Add State
add_state

### Change Action

An action class template with a `match()` function that parses the rule to which it is attached with its template parameter as action class template.
In other words, an action version of the [`action`](Rule-Reference.md#action) rule.

Publicly derives from [`maybe_nothing`](#maybe-nothing).
Included via `tao/pegtl/action/change_action.hpp`.

### Change Action and State
change_action_and_state

### Change Action and States
change_action_and_states

### Change Control

An action class template with a `match()` function that parses the rule to which it is attached with its template parameter as control class template.
In other words, an action version of the [`control`](Rule-Reference.md#control) rule.

Publicly derives from [`maybe_nothing`](#maybe-nothing).
Included via `tao/pegtl/action/change_control.hpp`.

### Change State
change_state

### Change States
change_states

### Control Action
control_action

### Disable Action

An action class with a `match()` function that parses the rule to which it is attached with actions disabled.
In other words, an action version of the [`disable`](Rule-Reference.md#disable) rule.

Publicly derives from [`maybe_nothing`](#maybe-nothing).
Included via `tao/pegtl/action/disable_action.hpp`.

### Enable Action

An action class with a `match()` function that parses the rule to which it is attached with actions enabled.
In other words, an action version of the [`enable`](Rule-Reference.md#enable) rule.

Publicly derives from [`maybe_nothing`](#maybe-nothing).
Included via `tao/pegtl/action/enable_action.hpp`.

### Require Apply

The class `require_apply` is an empty class used as tag.
An action class that is publicly derived from `require_apply` must define a static `apply()` function that can be called as action function.

Wins against [`maybe_nothing`](#maybe-nothing) whan a class has both as public base classes.
Included via `tao/pegtl/action/require_apply.hpp`.

### Require Apply0

The class `require_apply0` is an empty class used as tag.
An action class that is publicly derived from `require_apply` must define a static `apply0()` function that can be called as action function.

Wins against [`maybe_nothing`](#maybe-nothing) whan a class has both as public base classes.
Included via `tao/pegtl/action/require_apply0.hpp`.

---

This document is part of the [PEGTL](https://github.com/taocpp/PEGTL).

Copyright (c) 2014-2024 Dr. Colin Hirsch and Daniel Frey<br>
Distributed under the Boost Software License, Version 1.0<br>
See accompanying file [LICENSE_1_0.txt](../LICENSE_1_0.txt) or copy at https://www.boost.org/LICENSE_1_0.txt
38 changes: 29 additions & 9 deletions doc/Changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

**Not yet released**

* *Moar templates*, more templates in more places!
* Use the [**migration guide**](Migration-Guide.md#version-400) when updating.
* Infrastructure
* Switched to Boost Software License, Version 1.0.
Expand All @@ -22,32 +23,49 @@
* Added functions to visit and flatten [nested exceptions](Contrib-and-Examples.md#taopegtlcontribnested_exceptionshpp).
* Inputs
* Standardised on line - column - count order.
* The input classes have been heavily refactored.
* Most input classes can use any data type instead of just `char`.
* The end-of-line handling has been heavily refactored.
* The input classes have been *heavily* refactored.
* Most input classes can use any data type instead of being hardwired to `char`.
* The end-of-line handling has been *heavily* refactored and extended.
* Choice of statically or dynamically allocated buffer inputs.
* Everything related to buffered inputs is now in `include/tao/buffer/`.
* Nothing related to buffered inputs is now included in `<tao/pegtl.hpp>`.
* Nothing related to buffered inputs is not included in `<tao/pegtl.hpp>`.
* Removed `action_t` type alias from all input classes in favour of using `internal::action_input`.
* Never use unaligned memory accesses (unless the compiler generates them).
* Rule Changes
* Added Unicode rules that adapt to the input's data size.
* Changed the ASCII rules to adapt to the input's data size.
* Changed the integer rules to adapt to the input's data size.
* Added special end-of-line rules in `tao::pegtl::eols`.
* Added special end-of-line rules in multiple places.
* Added new atomic rule [`function`](Rule-Reference.md#function-f-).
* Added new ASCII rule [`cntrl`](Rule-Reference.md#cntrl).
* Added new ASCII rule [`cr`](Rule-Reference.md#cr).
* Added new ASCII rule [`cr_lf`](Rule-Reference.md#cr_lf).
* Added new ASCII rule [`cr_crlf`](Rule-Reference.md#cr_crlf).
* Added new ASCII rule [`cr_lf_crlf`](Rule-Reference.md#cr_lf_crlf).
* Added new ASCII rule [`crlf`](Rule-Reference.md#crlf).
* Added new ASCII rule [`esc`](Rule-Reference.md#esc).
* Added new ASCII rule [`ff`](Rule-Reference.md#ff).
* Added new ASCII rule [`graph`](Rule-Reference.md#graph).
* Added new ASCII rule [`ht`](Rule-Reference.md#ht).
* Added new ASCII rule [`ione`](Rule-Reference.md#ht).
* Added new ASCII rule [`lf`](Rule-Reference.md#lf).
* Added new ASCII rule [`lfcr`](Rule-Reference.md#lfcr).
* Added new ASCII rule [`lf_crlf`](Rule-Reference.md#lf_crlf).
* Added new ASCII rule [`not_ione`](Rule-Reference.md#ht).
* Added new ASCII rule [`sp`](Rule-Reference.md#sp).
* Added new ASCII rule [`vt`](Rule-Reference.md#vt).
* Added new ASCII rules that only match in the range 0 to 127.
* Added new Unicode rule [`cr`](Rule-Reference.md#cr-1).
* Added new Unicode rule [`cr_lf`](Rule-Reference.md#cr_lf-1).
* Added new Unicode rule [`cr_crlf`](Rule-Reference.md#cr_crlf-1).
* Added new Unicode rule [`cr_lf_crlf`](Rule-Reference.md#cr_lf_crlf-1).
* Added new Unicode rule [`crlf`](Rule-Reference.md#crlf-1).
* Added new Unicode rule [`lf`](Rule-Reference.md#lf-1).
* Added new Unicode rule [`lf_crlf`](Rule-Reference.md#lf_crlf-1).
* Added new Unicode rule [`ls`](Rule-Reference.md#ls).
* Added new Unicode rule [`nel`](Rule-Reference.md#nel).
* Added new Unicode rule [`ps`](Rule-Reference.md#ps).
* Added new Unicode rule [`eol1`](Rule-Reference.md#eol1).
* Added new Unicode rule [`eolu`](Rule-Reference.md#eolu).
* Added dedicated end-of-line rules for end-of-line scanning.
* Added dedicated end-of-line rules for lazy end-of-line mode.
* Added new atomic rule [`consume`](Rule-Reference.md#consume-count-).
* Added new atomic rule [`everything`](Rule-Reference.md#everything).
* Added new generic rule [`combine`](Rule-Reference.md#combine-r-l-).
Expand All @@ -67,8 +85,10 @@
* Added rule [`try_catch_std_raise_nested`](Rule-Reference.md#try_catch_std_raise_nested-r-).
* Added rule [`try_catch_type_raise_nested`](Rule-Reference.md#try_catch_type_raise_nested-e-r-).
* Added rules for matching signed integers mirroring the existing ones for unsigned integers.
* Optimised `utf8::string` by expanding code points to UTF-8 sequences at compile time.
* Optimised `utf8::string` by expanding `char32_t` code points to UTF-8 sequences at compile time.
* Refactored the implementation of `one`, `range`, `ranges`, `not_one` and `not_range`.
* Added new rules for enum types to the [binary rules](Rule-Reference.md#binary)
* Added new rules that operate on [members](Rule-Reference.md#member) instead of directly on values.
* Added new customization point for error messages. -- TODO!
* Added optional source line output for the tracer. -- TODO?
* Other
Expand Down
130 changes: 130 additions & 0 deletions doc/Control-Reference.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
# Control Reference

The reference documention for all control class templates.

### Included

Only `tao/pegtl/normal.hpp` which defines `normal` is automatically included with `tao/pegtl.hpp`.
For all other controls the appropriate header in `tao/pegtl/control/` needs to be included manually.

### Templates

Template parameters to control adapters that are documented as `Base` need to be control classes.

Template parameters to control adapters that are documented as `Control` need to be control class templates.

TODO: Examples? Explanations?

### Forwarding

The control class adapters that modify the states before forwarding the control functions to a different control will modify the states for all control functions **except** `match()` -- otherwise the modification would be applied multiple times for nested rule invocations.

### Namespaces

All control class templates reside in namespace `tao::pegtl`.
This default can be changed via the macro `TAO_PEGTL_NAMESPACE` in `tao/pegtl/config.hpp`.


## Contents

* [Normal](#normal)
* [Input Control](#input-control)
* [Must-If](#must-if)
* [Remove First State](#remove-first-state)
* [Remove Last States](#remove-last-states)
* [Reverse States](#reverse-states)
* [Rewind Control](#rewind-control)
* [Rewind State Control](#rewind-state-control)
* [Rotate States Left](#rotate-states-left)
* [Rotate States Right](#rotate-states-right)
* [Shuffle States](#shuffle-states)
* [State Control](#state-control)

### Normal

The control class template `normal< Rule >` contains the default control functions.

Serves as default `Control` template parameter to `parse()` and `parse_nested()`.

### Input Control

TODO

### Must-If

TODO

### Remove First State

The control class adapter `remove_first_state< Base >` calls the control functions in `Base` with the first state removed.

### Remove Last States

The control class adapter `remove_last_states< Base, N >` calls the control functions in `Base` with the last `N` states removed.

### Reverse States

The control class adapter `reverse_states< Base >` calls the control functions in `Base` with the states in reverse order.

This control adapter class template is implemented via `shuffle_states` and defined in `tao/pegtl/control/shuffle_states.hpp`.

### Rewind Control

The control class adapter `rewind_control< Control >::type< Rule >` adds rewind control functions.

More precisely, it implements a `guard()` control function that will call additional control functions on `Control< Rule >`, namely

* `prep_rewind()` when a rewind guard is instantiated,
* `will_rewind()` just before a rewind guard will rewind,
* `wont_rewind()` when a rewind guard will not rewind.

These rewind control functions are only called when the `rewind_mode` is `required`.

These rewind control functions are called with the input and all states as arguments.

These rewind control functions are not part of the normal control functions because calling them incurs a run-time overhead.

When using `rewind_control< Control >::type` instead of `Control` then `Control< Rule >::guard()` is never called.

### Rewind State Control

TODO

### Rotate States Left

The control class adapter `rotate_states_left< Base, N = 1 >` calls the control functions in `Base` with the states rotated left by `N`.

This control adapter class template is implemented via `shuffle_states` and defined in `tao/pegtl/control/shuffle_states.hpp`.

### Rotate States Right

The control class adapter `rotate_states_right< Base, N = 1 >` calls the control functions in `Base` with the states rotated right by `N`.

This control adapter class template is implemented via `shuffle_states` and defined in `tao/pegtl/control/shuffle_states.hpp`.

### Shuffle States

The control class adapter `shuffle_states< Base, Shuffle >` calls the control functions in `Base` with the states shuffled according to `Shuffle`.

The `Shuffle` template parameter must be a type that defines a variable template `value`.
The `I`-th shuffled state with `S` total states will be `shuffle< I, S >::value`.

```c++
struct shuffle
{
template< std::size_t I, std::size_t S >
static constexpr std::size_t value = ...
};
```

### State Control

TODO

---

This document is part of the [PEGTL](https://github.com/taocpp/PEGTL).

Copyright (c) 2014-2024 Dr. Colin Hirsch and Daniel Frey<br>
Distributed under the Boost Software License, Version 1.0<br>
See accompanying file [LICENSE_1_0.txt](../LICENSE_1_0.txt) or copy at https://www.boost.org/LICENSE_1_0.txt
5 changes: 3 additions & 2 deletions doc/Development.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,9 @@ Version 5.x of the PEGTL will make the jump to C++20 or even C++23.

### Other Things

* Build a compile-time facility to convert Unicode code points to UTF8 sequences!
* Investigate whether we are crazy enough to attempt parsing linked lists or trees.
* ~~Build a compile-time facility to convert Unicode code points to UTF8 sequences!~~
* Done in version 4.0.
* Investigate whether we are crazy enough to attempt parsing linked lists or trees?

### Buffer Inputs

Expand Down
Loading

0 comments on commit e013ae1

Please sign in to comment.