Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 49 additions & 52 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,56 +18,55 @@ $ pip install dipdup

## Creating indexer

If you want to see dipdup in action before diving into details you can run a demo project and use it as reference:
If you want to see dipdup in action before diving into details you can run a demo project and use it as reference. Clone this repo and run the following command in it's root directory:

```shell
$ dipdup -c src/demo_hic_et_nunc/dipdup.yml run
```

Examples in this guide are simplified Hic Et Nunc demo.

### Write configuration file

Create a new YAML file and adapt the following example to your needs:

```yaml
spec_version: 0.0.1
package: dipdup_hic_et_nunc
package: demo_hic_et_nunc

database:
kind: sqlite
path: db.sqlite3

contracts:
HEN_objkts:
address: ${HEN_OBJKTS:-KT1Hkg5qeNhfwpKW4fXvq7HGZB9z2EnmCCA9}
HEN_minter:
address: ${HEN_MINTER:-KT1RJ6PbjHpwc3M5rw5s2Nbmefwbuwbdxton}
HEN_objkts:
address: ${HEN_OBJKTS:-KT1RJ6PbjHpwc3M5rw5s2Nbmefwbuwbdxton}
typename: hen_objkts
HEN_minter:
address: ${HEN_MINTER:-KT1Hkg5qeNhfwpKW4fXvq7HGZB9z2EnmCCA9}
typename: hen_minter

datasources:
tzkt_mainnet:
kind: tzkt
url: ${TZKT_URL:-https://staging.api.tzkt.io}

indexes:
operations_mainnet:
hen_mainnet:
kind: operation
datasource: tzkt_mainnet
contract: HEN_objkts
first_block: 0
contract: HEN_minter
handlers:

- callback: on_mint
pattern:
- destination: HEN_objkts
entrypoint: mint_OBJKT
- destination: HEN_minter
entrypoint: mint_OBJKT
- destination: HEN_objkts
entrypoint: mint

- callback: on_transfer
pattern:
- destination: HEN_minter
entrypoint: transfer
```

Each handler in index config matches an operation group based on operations' entrypoints and destination addresses in pattern. Matched operation groups will be passed to handlers you define.

### Initialize project structure

Run the following command replacing `config.yml` with path to YAML file you just created:
Expand All @@ -79,30 +78,24 @@ $ dipdup -c config.yml init
This command will create a new package with the following structure (some lines were omitted for readability):

```
dipdup_hic_et_nunc/
demo_hic_et_nunc/
├── handlers
│ ├── on_mint.py
│ ├── on_rollback.py
│ └── on_transfer.py
│ └── on_rollback.py
├── hasura-metadata.json
├── models.py
├── schemas
│ ├── KT1Hkg5qeNhfwpKW4fXvq7HGZB9z2EnmCCA9
│ │ └── parameter
│ │ └── mint_OBJKT.json
│ └── KT1RJ6PbjHpwc3M5rw5s2Nbmefwbuwbdxton
│ └── parameter
│ └── mint.json
└── types
├── KT1Hkg5qeNhfwpKW4fXvq7HGZB9z2EnmCCA9
├── hen_minter
│ ├── storage.py
│ └── parameter
│ └── mint_OBJKT.py
└── KT1RJ6PbjHpwc3M5rw5s2Nbmefwbuwbdxton
└── hen_objkts
├── storage.py
└── parameter
└── mint.py
```

`schemas` directory is JSON schemas describing parameters of corresponding contract entrypoints. `types` are Pydantic dataclasses of these schemas. These two directories are autogenerated, you don't need to modify them. `models` and `handlers` modules will be discussed later.
`types` directory is Pydantic dataclasses of contract storage and parameter. This directory is autogenerated, you shouldn't modify any files in it. `models` and `handlers` modules are placeholders for your future code and will be discussed later.

You could invoke `init` command on existing project (must be in your `PYTHONPATH`. Do it each time you update contract addresses or models. Code you've wrote won't be overwritten.

Expand All @@ -115,43 +108,43 @@ Now open `models.py` file in your project and define some models:
from tortoise import Model, fields


class Address(Model):
class Holder(Model):
address = fields.CharField(58, pk=True)


class Token(Model):
id = fields.IntField(pk=True)
token_id = fields.IntField()
token_info = fields.CharField(255)
holder = fields.ForeignKeyField('models.Address', 'tokens')
id = fields.BigIntField(pk=True)
creator = fields.ForeignKeyField('models.Holder', 'tokens')
supply = fields.IntField()
level = fields.BigIntField()
timestamp = fields.DatetimeField()
```

### Write event handlers

Now take a look at `handlers` module generated by `init` command. When operation group matching `pattern` block of corresponding handler at config will arrive callback will be fired. This example will simply save minted Hic Et Nunc tokens and their owners to the database:

```python
from demo_hic_et_nunc.models import Holder, Token
from demo_hic_et_nunc.types.hen_minter.parameter.mint_objkt import MintOBJKT
from demo_hic_et_nunc.types.hen_objkts.parameter.mint import Mint
from dipdup.models import HandlerContext, OperationContext
from tests.test_dipdup.dipdup_hic_et_nunc.models import *
from tests.test_dipdup.dipdup_hic_et_nunc.types.KT1Hkg5qeNhfwpKW4fXvq7HGZB9z2EnmCCA9 import MintObjkt
from tests.test_dipdup.dipdup_hic_et_nunc.types.KT1RJ6PbjHpwc3M5rw5s2Nbmefwbuwbdxton.parameter.mint import Mint


async def on_mint(
ctx: HandlerContext,
mint_OBJKT: OperationContext[MintObjkt],
mint: OperationContext[Mint],
ctx: HandlerContext,
mint_objkt: OperationContext[MintOBJKT],
mint: OperationContext[Mint],
) -> None:
address, _ = await Address.get_or_create(address=mint.parameter.address)

for _ in range(int(mint.parameter.amount)):
token = Token(
token_id=int(mint.parameter.token_id),
token_info=mint.parameter.token_info[''],
holder=address,
transaction=mint.transaction,
)
await token.save()
holder, _ = await Holder.get_or_create(address=mint.parameter.address)
token = Token(
id=mint.parameter.token_id,
creator=holder,
supply=mint.parameter.amount,
level=mint.data.level,
timestamp=mint.data.timestamp,
)
await token.save()
```

Handler name `on_rollback` is reserved by dipdup, this special handler will be discussed later.
Expand Down Expand Up @@ -190,6 +183,8 @@ $ # edit `secrets.env` file, change credentials
$ docker-compose up dipdup
```

For debugging purposes you can index specific block range only and skip realtime indexing. To do this set `first_block` and `last_block` fields in index config.

### Index templates

Sometimes you need to run multiple indexes with similar configs whose only difference is contract addresses. In this case you can use index templates like this:
Expand Down Expand Up @@ -228,6 +223,7 @@ indexes:
token: FA20_token
```

Template values mapping could be accessed from within handlers at `ctx.template_values`.

### Optional: configure Hasura GraphQL Engine

Expand Down Expand Up @@ -276,7 +272,8 @@ You may want to tune logging to get notifications on errors or enable debug mess

## Contribution

To set up development environment you need to install [poetry](https://python-poetry.org/docs/#installation) package manager. Then run one of the following commands at project's root:
To set up development environment you need to install [poetry](https://python-poetry.org/docs/#installation) package manager and GNU Make. Then run one of the following commands at project's root:

```shell
$ # install project dependencies
$ make install
Expand Down