Skip to content

Commit

Permalink
docs: chore
Browse files Browse the repository at this point in the history
  • Loading branch information
HK-SHAO committed May 11, 2024
1 parent 341ec8b commit 22d0dc3
Show file tree
Hide file tree
Showing 6 changed files with 109 additions and 98 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ Use a loop statement in the generator. For example:

```py
def repeat4(self, s):
l = []
l: list[str] = []
for _ in range(4):
l.append((yield s))
self.do_my_own_thing(l)
Expand Down
2 changes: 1 addition & 1 deletion README.zh-hans.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ def diagram(self):

```py
def repeat4(self, s):
l = []
l: list[str] = []
for _ in range(4):
l.append((yield s))
self.do_my_own_thing(l)
Expand Down
5 changes: 5 additions & 0 deletions docs/source/examples/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Examples
===

- [Tests](https://github.com/YieldLang/yieldlang/tree/main/tests) - See the tests for more examples.
- [Visualization of syntax tree](https://github.com/YieldLang/yieldlang/discussions/17)
8 changes: 2 additions & 6 deletions docs/source/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,11 +122,8 @@

## Introduction

YieldLang is a [meta-language](https://en.wikipedia.org/wiki/Metalanguage) for generating structured text (ST) that can provide corpora for large language models (LLMs) or guide LLMs to generate ST. Currently provided as a Python package.
YieldLang is a [meta-language](https://en.wikipedia.org/wiki/Metalanguage) for generating structured text (ST) that can provide corpora for large language models (LLMs) or guide LLMs to generate ST. Currently provided as a [Python package](https://pypi.org/project/yieldlang/).

- Based on a coroutine generator and sampler architecture
- Stream-sends characters and parses the context above into a syntax tree
- Build formal grammars with classes, methods, and combinators

**Work in progress now.**

Expand All @@ -135,11 +132,10 @@ YieldLang is a [meta-language](https://en.wikipedia.org/wiki/Metalanguage) for g


```{toctree}
:maxdepth: 2
:maxdepth: 3
:caption: Getting Started
overview
installation
publications
```

Expand Down
100 changes: 10 additions & 90 deletions docs/source/overview.md
Original file line number Diff line number Diff line change
@@ -1,97 +1,17 @@
Overview
========

YieldLang is a [meta-language](https://en.wikipedia.org/wiki/Metalanguage) for LLMs to process (produce and parse) structured info.
```{toctree}
:maxdepth: 2
:hidden:
- View our [publications](publications.md) for more information

## Simple Usage

```bash
pip install yieldlang
```

Import the `TextGenerator` class and define a generator. The `top` method always serves as the entry point for the generator. You can treat the generator as an iterator and use a `for` loop to iterate over the generated text. For example:

```py
from yieldlang import TextGenerator

class G(TextGenerator):
def top(self):
yield "Hello, World!"

for text in G():
print(text)
```

Set another sampler for the generator (default is random sampling). For example, set the large language model sampler:

```py
sampler = MyLLMSampler()
print(list(G(sampler)))
```

Use combinators (e.g., `select`, `repeat`, `join`, etc.) to define grammar rules in the `TextGenerator`. For example, for JSON values:


```py
def value(self):
yield select(
self.object,
self.array,
self.string,
self.number,
self.boolean,
self.null
)
```

This is equivalent to the EBNF form:

```ebnf
value = object
| array
| string
| number
| boolean
| null
```

Generate a sequence easily. For example:

```py
def array(self):
yield select(
('[', self.ws, ']'),
('[', self.elements, ']')
)
```

You can get the string just generated and add branches, loops, and other control structures to the generation rules. For example:

```py
def diagram(self):
match (yield self.diagram_type):
case "flowchart":
yield self.flowchart
case "gannt":
yield self.gannt
```

Use a loop statement in the generator. For example:

```py
def repeat4(self, s):
l = []
for _ in range(4):
l.append((yield s))
self.do_my_own_thing(l)
installation
usages/simple
examples/index
```

Print the generated context tree (convertible to an abstract syntax tree):
YieldLang is a [meta-language](https://en.wikipedia.org/wiki/Metalanguage) for LLMs to process (produce and parse) structured info. View our [publications](publications.md) for more information.

```py
def print_context_tree():
ctx = yield from G()
print(ctx)
```
- 🧠 Based on a coroutine generator and sampler architecture
- 🤖 Stream-sends characters and parses the context above into a syntax tree
- 🦾 Build formal grammars with classes, methods, and combinators
90 changes: 90 additions & 0 deletions docs/source/usages/simple.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Simple Usage

```bash
pip install yieldlang
```

Import the `TextGenerator` class and define a generator. The `top` method always serves as the entry point for the generator. You can treat the generator as an iterator and use a `for` loop to iterate over the generated text. For example:

```py
from yieldlang import TextGenerator

class G(TextGenerator):
def top(self):
yield "Hello, World!"

for text in G():
print(text)
```

Set another sampler for the generator (default is random sampling). For example, set the large language model sampler:

```py
sampler = MyLLMSampler()
print(list(G(sampler)))
```

Use combinators (e.g., `select`, `repeat`, `join`, etc.) to define grammar rules in the `TextGenerator`. For example, for JSON values:


```py
def value(self):
yield select(
self.object,
self.array,
self.string,
self.number,
self.boolean,
self.null
)
```

This is equivalent to the EBNF form:

```ebnf
value = object
| array
| string
| number
| boolean
| null
```

Generate a sequence easily. For example:

```py
def array(self):
yield select(
('[', self.ws, ']'),
('[', self.elements, ']')
)
```

You can get the string just generated and add branches, loops, and other control structures to the generation rules. For example:

```py
def diagram(self):
match (yield self.diagram_type):
case "flowchart":
yield self.flowchart
case "gannt":
yield self.gannt
```

Use a loop statement in the generator. For example:

```py
def repeat4(self, s):
l: list[str] = []
for _ in range(4):
l.append((yield s))
self.do_my_own_thing(l)
```

Print the generated context tree (convertible to an abstract syntax tree):

```py
def print_context_tree():
ctx = yield from G()
print(ctx)
```

0 comments on commit 22d0dc3

Please sign in to comment.