Skip to content

Commit

Permalink
feat: 🎸 update readme with project description
Browse files Browse the repository at this point in the history
Signed-off-by: Marcin Pastecki <marcin.pastecki@inndei.com>
  • Loading branch information
polymath-eric authored and mpastecki committed Apr 18, 2024
1 parent 31bedcf commit 9027478
Showing 1 changed file with 15 additions and 11 deletions.
26 changes: 15 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,24 +5,14 @@

# Polymesh Subquery

This project has been initially created to replace [the harvester](https://github.com/PolymathNetwork/polkascan-pre-harvester) meaning the output of both should match as much as possible, however since the harvester serializes chain objects in slightly different ways than polkadot-js, this project contains a list of special cases handled both at the initial deserialization level (in [project.yaml](project.yaml).network.types) and at the serialization level in [serializeLikeHarvester.ts](src/mappings/serializeLikeHarvester.ts).
Polymesh Subquery allows for querying historical events on the Polymesh chain through a GraphQL interface. This project is based on [SubQuery](https://github.com/subquery/subql), and contains the mappings specific to Polymesh.

## Running

1. In the [docker-compose.yml](docker-compose.yml) file, set the appropriate [environment variables](#env-settings) for `subquery-node` container
2. Install subql cli: `npm i -g @subql/cli`
3. `./rerun.sh` (requires docker compose). To persist data between runs, remove the `-v` flag, which causes the docker volume to be removed

## Using event_arg_x indexes.

The event_arg_x columns are now text, in order for them to fit in BTree indexes they are truncated to 100 characters like in the harvester.
This means that if you want to take advantage of the index in your query, you must use an expression like: `WHERE left(event_arg_x, 100) = 'foobar'`.
Otherwise using `WHERE event_arg_x = 'foobar'` will result in a full table scan.

## Debugging using the found_types table

The found_types table generated by subquery contains all types that have been serialized like the harvester, the actual type is in column ID which is the primary key and therefore unique and the raw type is in the raw_type column, if things break this is the first place to look for potential mismatches between types.

## Version

This SubQuery version works with chain versions 6.2.x
Expand All @@ -40,6 +30,20 @@ More advanced options are:
- `NO_NATIVE_GRAPHQL_DATA` — this will set the event handler to only record what is necessary for tooling gql. This allows a stable indexer to be ran as native GraphQL handlers get developed
- `MAX_OLD_SPACE_SIZE` — this will be passed onto the node process as `--max-old-space-size` flag. The recommendation is for this to be ~75% of available RAM. Defaults to 1536, a setting appropriate for 2GB.

## Historical note

This project has been initially created to replace [the harvester](https://github.com/PolymathNetwork/polkascan-pre-harvester) meaning the output of both should match as much as possible, however since the harvester serializes chain objects in slightly different ways than polkadot-js, this project contains a list of special cases handled both at the initial deserialization level (in [project.yaml](project.yaml).network.types) and at the serialization level in [serializeLikeHarvester.ts](src/mappings/serializeLikeHarvester.ts).

### Using event_arg_x indexes.

The event_arg_x columns are now text, in order for them to fit in BTree indexes they are truncated to 100 characters like in the harvester.
This means that if you want to take advantage of the index in your query, you must use an expression like: `WHERE left(event_arg_x, 100) = 'foobar'`.
Otherwise using `WHERE event_arg_x = 'foobar'` will result in a full table scan.

### Debugging using the found_types table

The found_types table generated by subquery contains all types that have been serialized like the harvester, the actual type is in column ID which is the primary key and therefore unique and the raw type is in the raw_type column, if things break this is the first place to look for potential mismatches between types.

## License

This project uses [SubQuery](https://github.com/subquery/subql), which is [Apache 2.0 licensed](./LICENSE).
Expand Down

0 comments on commit 9027478

Please sign in to comment.