Skip to content

Commit

Permalink
Language and structure improvements
Browse files Browse the repository at this point in the history
  • Loading branch information
afreeland committed Feb 20, 2024
1 parent 4af7244 commit f23af3f
Show file tree
Hide file tree
Showing 2 changed files with 21 additions and 10 deletions.
2 changes: 1 addition & 1 deletion book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
+ [Using `Extras`](./extras.md)
+ [Using callbacks](./callbacks.md)
+ [Common regular expressions](./common-regex.md)
+ [Debug Graph](./debug-graph.md)
+ [Debugging](./debugging.md)
+ [Examples](./examples.md)
+ [Brainfuck interpreter](./examples/brainfuck.md)
+ [JSON parser](./examples/json.md)
Expand Down
29 changes: 20 additions & 9 deletions book/src/debug-graph.md → book/src/debugging.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
# Debug Graph
# Debugging

Gain deeper insights into your code's behavior with this debugging section. Here you can discover techniques and tools to explore and understand your project's internals.

## Visualizing Logos Graph

Logos works by creating a graph that gets derived from the tokens that you create. This graph visualizes how the lexer moves through different states when processing input.

It may be beneficial during debugging to be able to visualize how Logos has generated its graph.

Expand Down Expand Up @@ -29,7 +35,7 @@ fn main() {
}
```

With our graph debugging enabled, we see the following output:
Logos actually constructs a graph that contains the logic for matching tokens:
```
graph = {
1: ::Fast,
Expand Down Expand Up @@ -58,21 +64,26 @@ graph = {
},
}
```
This graph can help us understand how our patterns are matched, and maybe understand why we have a bug at some point.

Let's get started by trying to understand how Logos is matching the `.` character, which we've tokenized as `Token::Period`.

We can begin our search by looking at number `@9` for the character `.`. We can see that if Logos matches a `.` it will jump `=>` to number `2`. We can then follow that by looking at `@2` which resolves to our `::Period` token.

Lets look at `@9` for the character `.` we see that it will jump `=>` to `2`. We can then follow that by looking at `@2` which resolves to our `::Period` token. It will then continue to look for any matches in the case there is potential continuation after the `.` character. In the _input_ we provided there is not, since it is the end of our input.
Logos will then continue to look for any matches past our `.` character. This is required in case there is potential continuation after the `.` character. Although, in the _input_ we provided there are not any additional characters, since it is the end of our input.

We also can try to identify how `fast` works by looking at `@9` that `f` jumps to `7`. This will then resolve the last letters of our word _fast_ by matching `ast` which jumps to `8`. Since our provided _input_ to the lexer does not include alphanumeric character after the word "fast" but rather whitespace, the token `::Fast` will be recognized. Then the graph will look for further potential continuation (ie `[g-z] => 4`)
We also can try to identify how the token `fast` works by looking at `@9`, first, and seeing that `f` will cause Logos to jump to `7`. This will then resolve the last letters of our word _fast_ by matching `ast` which jumps to `@8`. Since our provided _input_ to the lexer does not include alphabetic characters after the word "fast" but rather whitespace, the token `::Fast` will be recognized. Then the graph will look for further potential continuation (i.e., `[g-z] => 4`)

## Enabling
### Enabling

### Step 1
#### Step 1
Start by pulling the Logos repo locally
```
git clone git@github.com:maciejhirsz/logos.git
```


### Step 2
#### Step 2
In the [Generator::new](https://github.com/maciejhirsz/logos/blob/master/logos-codegen/src/generator/mod.rs#L47) function you can add `dbg!(graph);` like the following:

```rust,no_run,noplayground
Expand Down Expand Up @@ -103,11 +114,11 @@ impl<'a> Generator<'a> {
}
```

### Step 3
#### Step 3
Run `cargo build`


### Step 4
#### Step 4
Now in your code that uses Logos, point your Logos dependency in `Cargo.toml` to your local repo
```
[dependencies]
Expand Down

0 comments on commit f23af3f

Please sign in to comment.