diff --git a/README.md b/README.md index 4a592126a..74d992924 100644 --- a/README.md +++ b/README.md @@ -18,29 +18,33 @@ $ pip install dipdup ## Creating indexer -If you want to see dipdup in action before diving into details you can run a demo project and use it as reference: +If you want to see dipdup in action before diving into details you can run a demo project and use it as reference. Clone this repo and run the following command in it's root directory: ```shell $ dipdup -c src/demo_hic_et_nunc/dipdup.yml run ``` +Examples in this guide are simplified Hic Et Nunc demo. + ### Write configuration file Create a new YAML file and adapt the following example to your needs: ```yaml spec_version: 0.0.1 -package: dipdup_hic_et_nunc +package: demo_hic_et_nunc database: kind: sqlite path: db.sqlite3 contracts: - HEN_objkts: - address: ${HEN_OBJKTS:-KT1Hkg5qeNhfwpKW4fXvq7HGZB9z2EnmCCA9} - HEN_minter: - address: ${HEN_MINTER:-KT1RJ6PbjHpwc3M5rw5s2Nbmefwbuwbdxton} + HEN_objkts: + address: ${HEN_OBJKTS:-KT1RJ6PbjHpwc3M5rw5s2Nbmefwbuwbdxton} + typename: hen_objkts + HEN_minter: + address: ${HEN_MINTER:-KT1Hkg5qeNhfwpKW4fXvq7HGZB9z2EnmCCA9} + typename: hen_minter datasources: tzkt_mainnet: @@ -48,26 +52,21 @@ datasources: url: ${TZKT_URL:-https://staging.api.tzkt.io} indexes: - operations_mainnet: + hen_mainnet: kind: operation datasource: tzkt_mainnet - contract: HEN_objkts - first_block: 0 + contract: HEN_minter handlers: - - callback: on_mint pattern: - - destination: HEN_objkts - entrypoint: mint_OBJKT - destination: HEN_minter + entrypoint: mint_OBJKT + - destination: HEN_objkts entrypoint: mint - - - callback: on_transfer - pattern: - - destination: HEN_minter - entrypoint: transfer ``` +Each handler in index config matches an operation group based on operations' entrypoints and destination addresses in pattern. Matched operation groups will be passed to handlers you define. + ### Initialize project structure Run the following command replacing `config.yml` with path to YAML file you just created: @@ -79,30 +78,24 @@ $ dipdup -c config.yml init This command will create a new package with the following structure (some lines were omitted for readability): ``` -dipdup_hic_et_nunc/ +demo_hic_et_nunc/ ├── handlers │ ├── on_mint.py -│ ├── on_rollback.py -│ └── on_transfer.py +│ └── on_rollback.py ├── hasura-metadata.json ├── models.py -├── schemas -│ ├── KT1Hkg5qeNhfwpKW4fXvq7HGZB9z2EnmCCA9 -│ │ └── parameter -│ │ └── mint_OBJKT.json -│ └── KT1RJ6PbjHpwc3M5rw5s2Nbmefwbuwbdxton -│ └── parameter -│ └── mint.json └── types - ├── KT1Hkg5qeNhfwpKW4fXvq7HGZB9z2EnmCCA9 + ├── hen_minter + │ ├── storage.py │ └── parameter │ └── mint_OBJKT.py - └── KT1RJ6PbjHpwc3M5rw5s2Nbmefwbuwbdxton + └── hen_objkts + ├── storage.py └── parameter └── mint.py ``` -`schemas` directory is JSON schemas describing parameters of corresponding contract entrypoints. `types` are Pydantic dataclasses of these schemas. These two directories are autogenerated, you don't need to modify them. `models` and `handlers` modules will be discussed later. +`types` directory is Pydantic dataclasses of contract storage and parameter. This directory is autogenerated, you shouldn't modify any files in it. `models` and `handlers` modules are placeholders for your future code and will be discussed later. You could invoke `init` command on existing project (must be in your `PYTHONPATH`. Do it each time you update contract addresses or models. Code you've wrote won't be overwritten. @@ -115,15 +108,16 @@ Now open `models.py` file in your project and define some models: from tortoise import Model, fields -class Address(Model): +class Holder(Model): address = fields.CharField(58, pk=True) class Token(Model): - id = fields.IntField(pk=True) - token_id = fields.IntField() - token_info = fields.CharField(255) - holder = fields.ForeignKeyField('models.Address', 'tokens') + id = fields.BigIntField(pk=True) + creator = fields.ForeignKeyField('models.Holder', 'tokens') + supply = fields.IntField() + level = fields.BigIntField() + timestamp = fields.DatetimeField() ``` ### Write event handlers @@ -131,27 +125,26 @@ class Token(Model): Now take a look at `handlers` module generated by `init` command. When operation group matching `pattern` block of corresponding handler at config will arrive callback will be fired. This example will simply save minted Hic Et Nunc tokens and their owners to the database: ```python +from demo_hic_et_nunc.models import Holder, Token +from demo_hic_et_nunc.types.hen_minter.parameter.mint_objkt import MintOBJKT +from demo_hic_et_nunc.types.hen_objkts.parameter.mint import Mint from dipdup.models import HandlerContext, OperationContext -from tests.test_dipdup.dipdup_hic_et_nunc.models import * -from tests.test_dipdup.dipdup_hic_et_nunc.types.KT1Hkg5qeNhfwpKW4fXvq7HGZB9z2EnmCCA9 import MintObjkt -from tests.test_dipdup.dipdup_hic_et_nunc.types.KT1RJ6PbjHpwc3M5rw5s2Nbmefwbuwbdxton.parameter.mint import Mint async def on_mint( - ctx: HandlerContext, - mint_OBJKT: OperationContext[MintObjkt], - mint: OperationContext[Mint], + ctx: HandlerContext, + mint_objkt: OperationContext[MintOBJKT], + mint: OperationContext[Mint], ) -> None: - address, _ = await Address.get_or_create(address=mint.parameter.address) - - for _ in range(int(mint.parameter.amount)): - token = Token( - token_id=int(mint.parameter.token_id), - token_info=mint.parameter.token_info[''], - holder=address, - transaction=mint.transaction, - ) - await token.save() + holder, _ = await Holder.get_or_create(address=mint.parameter.address) + token = Token( + id=mint.parameter.token_id, + creator=holder, + supply=mint.parameter.amount, + level=mint.data.level, + timestamp=mint.data.timestamp, + ) + await token.save() ``` Handler name `on_rollback` is reserved by dipdup, this special handler will be discussed later. @@ -190,6 +183,8 @@ $ # edit `secrets.env` file, change credentials $ docker-compose up dipdup ``` +For debugging purposes you can index specific block range only and skip realtime indexing. To do this set `first_block` and `last_block` fields in index config. + ### Index templates Sometimes you need to run multiple indexes with similar configs whose only difference is contract addresses. In this case you can use index templates like this: @@ -228,6 +223,7 @@ indexes: token: FA20_token ``` +Template values mapping could be accessed from within handlers at `ctx.template_values`. ### Optional: configure Hasura GraphQL Engine @@ -276,7 +272,8 @@ You may want to tune logging to get notifications on errors or enable debug mess ## Contribution -To set up development environment you need to install [poetry](https://python-poetry.org/docs/#installation) package manager. Then run one of the following commands at project's root: +To set up development environment you need to install [poetry](https://python-poetry.org/docs/#installation) package manager and GNU Make. Then run one of the following commands at project's root: + ```shell $ # install project dependencies $ make install