Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: using the indexer with PostgreSQL #1495

Merged
merged 7 commits into from
Sep 20, 2023
Merged
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
121 changes: 115 additions & 6 deletions docs/pages/indexer.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -284,12 +284,14 @@ To understand the result, it is easiest to look at it inside Node.

To index a different blockchain, you specify these environment variables:

| Variable | Value |
| --------------- | -------------------------------------------------------------------------------------------------------- |
| RPC_HTTP_URL | The URL to access the blockchain using HTTP |
| RPC_WS_URL | The URL to access the blockchain using WebSocket |
| START_BLOCK | The block to start indexing from. The block in which the `World` contract was deployed is a good choice. |
| SQLITE_FILENAME | `<blockchain name>.db` |
| Variable | Value |
| ----------------------- | -------------------------------------------------------------------------------------------------------- |
| RPC_HTTP_URL | The URL to access the blockchain using HTTP |
| RPC_WS_URL | The URL to access the blockchain using WebSocket |
| START_BLOCK<sup>1</sup> | The block to start indexing from. The block in which the `World` contract was deployed is a good choice. |
| SQLITE_FILENAME | `<blockchain name>.db` |

(1) This value is optional, but highly recommended in chains where the world was deployed after a large number of blocks had already passed.

After you do that, start the indexer with this command:

Expand All @@ -298,3 +300,110 @@ pnpm start:sqlite
```

Note that in your queries you need to specify the correct chainId for the chain you are indexing.

## Using the indexer with PostgreSQL

To run the indexer with [PostgreSQL](https://www.postgresql.org/), follow these steps:

1. Download and install [PostgreSQL](https://www.postgresql.org/download/).

1. To use the default setting (local PostgreSQL, using the `postgres` database, indexing the local `anvil` chain), run:

```sh copy
pnpm start:postgres:local
```

Otherwise, set these environment variables:

| Variable | Value |
| ------------------------ | -------------------------------------------------------------------------------------------------------- |
| CHAIN_ID | ChainID for the chain you index |
| RPC_HTTP_URL<sup>1</sup> | The URL to access the blockchain using HTTP |
| RPC_WS_URL<sup>1</sup> | The URL to access the blockchain using WebSocket |
| START_BLOCK<sup>2</sup> | The block to start indexing from. The block in which the `World` contract was deployed is a good choice. |
| DATABASE_URL | URL for the database, of the form `postgres://<host>/<database>` |

(1) Not necessary in the case of a Lattice testnet.

(2) This value is optional, but highly recommended in chains where the world was deployed after a large number of blocks had already passed for performance reasons.

And then run:

```sh copy
pnpm start:postgres
```

### Reading information from PostgreSQL

You can now use [`psql`](https://www.postgresql.org/docs/current/app-psql.html) to view the indexer information.

Note: This example uses [the template](http://localhost:3000/tutorials/minimal).
If you are indexing a different `World`, change the world address and use the name of a table you have rather than `Counter`.

1. Get the list of schemas to verify `__mud_internal` is there.

```sql copy
\dn
```

1. Get the list of relations in the schema.

```sql copy
\dt __mud_internal.*
```

You should get two tables, `chain` and `tables`.

1. Get the list of chains being indexed.

```sql copy
SELECT * FROM __mud_internal.chain;
```

1. The `tables` table is a lot bigger, so first get the list of columns within the table.

```sql copy
\d+ __mud_internal.tables
```

1. Get the list of MUD table names.

```sql copy
SELECT name FROM __mud_internal.tables;
```

1. Get the key and value schemas for `Counter`.

```sql copy
SELECT key_schema, value_schema FROM __mud_internal.tables WHERE name='Counter';
```

1. The schema for a world is `<address>__`.
Get the list of tables for a specific world.

```sql copy
\dt "0x5FbDB2315678afecb367f032d93F642f64180aa3__".*
```

1. Specify the world schema is the default.

```sql copy
SET search_path TO "0x5FbDB2315678afecb367f032d93F642f64180aa3__";
```

1. Get the value of the counter.

```sql copy
SELECT * FROM "Counter";
```

### Deleting the PostgreSQL data

PostgreSQL is a persistent data storage.
This means that if you restart the blockchain without manually deleting the data, your indexer will use old data and provide incorrect results.

To avoid this, enter `psql` and run this command:

```sql copy
DROP OWNED BY <your user name>;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what happens when people execute within docker? docker has different usernames I think.
should we drop the DB instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docker directions cover this.

```