Margaret - Janet’s peg/match in Janet

Table of Contents

• Background
• Status
• Why?
• Setup
• Usage
• Testing
• Specials Implementation Status
• Implementation Notes
• Acknowledgments

This aims to be a pure-Janet implementation of Janet’s peg/match.

Background

While reading bakpakin’s How Janet’s PEG module works, started typing the code in and evaluating it, next thing I knew…​

Status

All peg specials except split have implementations and tests.

Why?

• Aid in understanding Janet’s PEG system

• Generate example PEG specials usages

• Experiment with adding diagnostic info / tracing

• Experiment with additional constructs

Setup

• Clone this repository and cd to the relevant directory

• Ensure janet and jpm are on your PATH

• Install if you like: jpm install

Usage

meg/match is an attempt at implementing peg/match, so for example:

(import margaret/meg)

(meg/match ~(capture (some "smile")) "smile!")

should work.

There is some support for diagnostic output via the dynamic variable :meg-debug.

To try it, do:

(setdyn :meg-debug true)

or for more detailed info:

(setdyn :meg-debug {:captures true
                    :peg true
                    :text true})

before calling meg/match.

Output format is still evolving but likely will support at least:

1. capture stack

2. tags

3. specials execution order

Testing

Run all tests by:

• jpm test from the project directory of margaret

There are currently tests for:

• margaret’s meg module

• peg module tutorial

• modified code from bakpakin’s article

Some of the tests were adapted / copied from Janet’s tests.

Specials Implementation Status

Each of the Janet PEG specials has an initial implementation with tests. Specifically, that includes:

1. Primitive Patterns

   1. integer patterns

   2. range

   3. set

   4. string patterns

   5. boolean patterns

2. Combinators

   1. any

   2. at-least

   3. at-most

   4. backmatch

   5. between

   6. choice, +

   7. if

   8. if-not

   9. look, >

   10. opt, ?

   11. repeat, "n"

   12. sequence, *

   13. some

   14. sub

   15. thru

   16. to

   17. unref

3. Captures

   1. accumulate, %

   2. argument

   3. backref, ->

   4. capture, <-, quote

   5. cmt

   6. column

   7. constant

   8. drop

   9. error

   10. group

   11. int

   12. int-be

   13. lenprefix

   14. line

   15. number

   16. position, $

   17. replace, /

   18. uint

   19. uint-be

Implementation Notes

In many cases an attempt to follow the original peg.c implementation by bakpakin was made. Some motivations for doing so include:

1. If peg.c changes, tracking those changes may be easier.

2. If an experimentally added special in margaret proves useful, it might be easier to port it to peg.c.

3. Arriving at a correct implementation might be easier because comparing it with a similar one is more meaningful.

4. Debugging information obtained here might be more relevant when trying to understand a situation in the original peg.c context.

5. Reading margaret’s implementation might be an easier place to start if one wants to study peg.c.

Some differences include:

1. peg.c creates a bytecode representation before execution and some information (e.g. tag names) is not retained. In at least one case, some uses of range get compiled to the same type of bytecode instruction used by set.

2. peg.c has protections for too much recursion.

Acknowledgments

Thanks to (at least) the following folks:

• ahungry

• andrewchambers

• bakpakin

• CosmicToast

• crocket

• goto-engineering

• GrayJack

• ikarius

• jcmkk3

• LeafGarland

• leahneukirchen

• LeviSchuck

• MikeBeller

• nate

• pepe

• pyrmont

• Saikyun

• skuza

• subsetpark

• swlkr

• tami5

• uvtc

• yumaikas

…​and other Janet community members :)