Skip to content

Commit

Permalink
Merge branch 'main' of github.com:blockcoders/ink-substrate-explorer-…
Browse files Browse the repository at this point in the history
…api into docs/fixes
  • Loading branch information
0xslipk committed Sep 28, 2022
2 parents bc08919 + 73baa3a commit 4e264a0
Show file tree
Hide file tree
Showing 12 changed files with 161 additions and 80 deletions.
Binary file added .images/graphql_example.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .images/pg_admin_connection.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .images/pg_admin_select_server.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .images/pg_admin_server_name.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .images/pg_admin_up.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM node:16.15-alpine AS builder
FROM node:16.16.0-alpine AS builder

# Create app directory
WORKDIR /usr/src/app
Expand All @@ -23,7 +23,7 @@ RUN pnpm prune --prod --no-optional

# By using the FROM statement again, we are telling Docker that it should create a new,
# fresh image without any connection to the previous one.
FROM node:16.15-alpine AS build
FROM node:16.16.0-alpine AS build

# Create app directory
WORKDIR /usr/src/app
Expand Down
94 changes: 66 additions & 28 deletions README-es.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,8 @@ cp .env.sample .env

- **BLOCK_CONCURRENCY**=1000 - <span style="color:#2a98db"> Número de bloques a procesar simultáneamente. Esto puede acelerar o retrasar el proceso de sincronización.</span>

## **Levantando el servicio (DEV)**

### Instanciar una BD Postgres utilizando docker (opcional)

Para levantar el servicio es necesario contar con una **BD PostgreSQL**. Para esto, el archivo **dev-docker-compose.yaml** ya tiene una imagen configurada lista para usar.
Expand All @@ -115,6 +117,29 @@ Ejecutando el siguiente comando tambien instanciara un contenedor para pgAdmin:
docker-compose -f dev-docker-compose.yaml up -d
```

Una vez que el servicio se está ejecutando, se puede acceder a pgAdmin siguiendo el enlace que se muestra en la terminal (en este caso localhost:80).

![pgAdmin](/.images/pg_admin_up.png)

Las credenciales para acceder a pgAdmin son (establecidas en el archivo dev-docker-compose):

- PGADMIN_DEFAULT_EMAIL: "admin@admin.com"
- PGADMIN_DEFAULT_PASSWORD: "admin"

Registre un nuevo servidor en pgAdmin y establezca las credenciales para la base de datos PostgreSQL:

Haga clic derecho en 'Servidores' y seleccione "Registrarse" -> "Servidor"

![pgAdmin](/.images/pg_admin_select_server.png)

Establezca un nombre para el servidor (en este ejemplo, "Docker")

![pgAdmin](/.images/pg_admin_server_name.png)

Establezca las credenciales para la base de datos PostgreSQL (esto se puede encontrar en el archivo dev-docker-compose):

![pgAdmin](/.images/pg_admin_connection.png)

### Instanciar un nodo local de Substrate (opcional)

El servicio necesita conectarse a una blockchain basada en Substrate. Para esto, el archivo **dev-docker-compose.yaml** ya tiene una imagen configurada lista para usar.
Expand All @@ -128,7 +153,7 @@ Otra forma de ejecutar un nodo local es con [esta guía de paritytech](https://g

**Nota**: Cambie la variable WS_PROVIDER en el archivo **.env** para que sea `ws://127.0.0.1:9944`

### Levantando el servicio (DEV)
### Levantando el servicio

- ### `pnpm start:dev`

Expand All @@ -137,21 +162,7 @@ El servicio se recargará si realiza ediciones.

**Nota**: Se requiere una base de datos postgres en funcionamiento y una conexión válida a un nodo de Substrate.

### Levantando el servicio (PROD)

Para iniciar tanto el contenedor del servicio de back-end como el contenedor de la base de datos, ejecute:

- ### `docker-compose up -d`

### Test

Ejecución de las pruebas unitarias.

- ### `pnpm test`

Ejecución de la cobertura de pruebas.

- ### `pnpm test:cov`
## **Levantando el servicio (PROD)**

## Ejecutar la imagen de Docker del servicio back-end

Expand Down Expand Up @@ -180,7 +191,31 @@ f31a7d0fd6c8 blockcoders/ink-substrate-explorer-api "docker-entrypoint.s…"

Después de que se inició el servidor, los bloques deberían guardarse en la base de datos.

## **API**
Para iniciar tanto el contenedor del servicio de back-end como el contenedor de la base de datos, ejecute:

- ### `docker-compose up -d`

El servicio se conectará al contenedor DB y comenzará a procesar bloques.

## **Testing**

Ejecución de las pruebas unitarias.

- ### `pnpm test`

Ejecución de la cobertura de pruebas.

- ### `pnpm test:cov`

Probando las consultas de GraphQL.

- ### `{"level":30,"time":1664298430389,"pid":1388770,"hostname":"username","name":"ink-substrate-explorer-api","msg":"App listening on http://0.0.0.0:5000"}`

Una vez que el servicio back-end se está ejecutando, se puede acceder a GraphQL Playground en http://localhost:5000/graphql

![backend](/.images/graphql_example.png)

## **Definicion de la API**

Una vez que el servicio esta levantado y corriendo correctamente se provee una API que puede utilizarse enviado consultas de GraphQL.

Expand Down Expand Up @@ -402,7 +437,7 @@ query {

```graphql
query {
decodeEvents(contractAddress: "5G24svh2w4QXNhsHU5XBxf8N3Sw2MPu7sAemofv1bCuyxAzc")
decodeEvents(contractAddress: "5G24svh2w4QXNhsHU5XBxf8N3Sw2MPu7sAemofv1bCuyxAzc")
}
```

Expand Down Expand Up @@ -446,7 +481,10 @@ query {

```graphql
mutation Upload {
uploadMetadata(contractAddress: "5G24svh2w4QXNhsHU5XBxf8N3Sw2MPu7sAemofv1bCuyxAzc", metadata: "ewogICJzb3VyY2UiOiB7CiAgICAiaGFzaCI6I...(base64)")
uploadMetadata(
contractAddress: "5G24svh2w4QXNhsHU5XBxf8N3Sw2MPu7sAemofv1bCuyxAzc"
metadata: "ewogICJzb3VyY2UiOiB7CiAgICAiaGFzaCI6I...(base64)"
)
}
```

Expand Down Expand Up @@ -609,15 +647,15 @@ En caso de un tiempo de inactividad del nodo, las suscripciones se reconectarán
### Algunos puntos de referencia

#### Utilizando BLOCK_CONCURRENCY = 100
- 100 bloques en ~ 6 segundos
- 1000 bloques en ~ 30.5 segundos
- 10000 bloques en ~ 4:24 minutos
- 100000 bloques en ~ 39.57 minutos

#### Utilizando BLOCK_CONCURRENCY = 1000
- 100 bloques en ~ 0.5 segundos
- 1000 bloques en ~ 5 segundos
- 10000 bloques en ~ 3 minutos
- 100000 bloques en ~ 24 minutos
- 100 bloques en ~ 6 segundos
- 1000 bloques en ~ 30.5 segundos
- 10000 bloques en ~ 4:24 minutos
- 100000 bloques en ~ 39.57 minutos

#### Utilizando BLOCK_CONCURRENCY = 1000

- 100 bloques en ~ 0.5 segundos
- 1000 bloques en ~ 5 segundos
- 10000 bloques en ~ 3 minutos
- 100000 bloques en ~ 24 minutos
110 changes: 76 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,8 @@ FIRST_BLOCK_TO_LOAD=0
BLOCK_CONCURRENCY=1000
```

## **Starting the project (DEV)**

### Start a Postgres DB using docker (optional)

To start the project a **PostgreSQL DB** is needed. For that, the **dev-docker-compose.yaml** file already has an image set up ready to use.
Expand All @@ -88,6 +90,29 @@ Running this command it will also start a container for pgAdmin:
docker-compose -f dev-docker-compose.yaml up -d
```

Once the service is running, pgAdmin can be accessed following the link that is shown in the terminal (In this case localhost:80).

![pgAdmin](/.images/pg_admin_up.png)

The credentials to access pgAdmin are (set in the dev-docker-compose file):

- PGADMIN_DEFAULT_EMAIL: "admin@admin.com"
- PGADMIN_DEFAULT_PASSWORD: "admin"

Register a new server in pgAdmin and set the credentials for the PostgreSQL DB:

Right click on 'Servers' and select "Register" -> "Server"

![pgAdmin](/.images/pg_admin_select_server.png)

Set a name for the server (In this example "Docker")

![pgAdmin](/.images/pg_admin_server_name.png)

Set the credentials for the PostgreSQL DB (this can be found in the dev-docker-compose file):

![pgAdmin](/.images/pg_admin_connection.png)

### Start a local Substrate Node (optional)

The service needs to connect to a Substrate Blockchain. For that, the **dev-docker-compose.yaml** file already has an image set up ready to use.
Expand All @@ -101,7 +126,7 @@ Another way to run a local node is with [this paritytech guide](https://github.c

**Note**: Change the WS_PROVIDER var in the **.env** file to be `ws://127.0.0.1:9944`

### Starting the project (DEV)
### Start the service

```sh
pnpm start:dev
Expand All @@ -112,27 +137,7 @@ The service will reload if you make edits.

**Note**: A postgresDB up and running and a valid connection to a substrate node are required.

### Starting the project (PROD)

To start both the Back-end service container and the DB container run:

```sh
docker-compose up -d
```

### Test

Running the unit tests.

```sh
pnpm test
```

Running the test coverage.

```sh
pnpm test:cov
```
## **Starting the project (PROD)**

## Running the Back-end service Docker image

Expand Down Expand Up @@ -167,7 +172,39 @@ f31a7d0fd6c8 blockcoders/ink-substrate-explorer-api "docker-entrypoint.s…"

After the server started, the blocks should be being saved on the DB.

## **API**
To start both the Back-end service container and the DB container run:

```sh
docker-compose up -d
```

The service will connect to the DB container and start processing blocks.

## **Testing**

Running the unit tests.

```sh
pnpm test
```

Running the test coverage.

```sh
pnpm test:cov
```

Testing the GraphQL queries.

```sh
{"level":30,"time":1664298430389,"pid":1388770,"hostname":"username","name":"ink-substrate-explorer-api","msg":"App listening on http://0.0.0.0:5000"}
```

Once the back-end service is running, the GraphQL Playground can be accessed at http://localhost:5000/graphql

![backend](/.images/graphql_example.png)

## **API definition**

With the service up and running an API is provided by using GraphQL queries.

Expand Down Expand Up @@ -389,7 +426,7 @@ query {

```graphql
query {
decodeEvents(contractAddress: "5G24svh2w4QXNhsHU5XBxf8N3Sw2MPu7sAemofv1bCuyxAzc")
decodeEvents(contractAddress: "5G24svh2w4QXNhsHU5XBxf8N3Sw2MPu7sAemofv1bCuyxAzc")
}
```

Expand Down Expand Up @@ -433,7 +470,10 @@ query {

```graphql
mutation Upload {
uploadMetadata(contractAddress: "5G24svh2w4QXNhsHU5XBxf8N3Sw2MPu7sAemofv1bCuyxAzc", metadata: "ewogICJzb3VyY2UiOiB7CiAgICAiaGFzaCI6I...(base64)")
uploadMetadata(
contractAddress: "5G24svh2w4QXNhsHU5XBxf8N3Sw2MPu7sAemofv1bCuyxAzc"
metadata: "ewogICJzb3VyY2UiOiB7CiAgICAiaGFzaCI6I...(base64)"
)
}
```

Expand Down Expand Up @@ -591,21 +631,23 @@ The first time the node is started, it may need to start from the block 0 and lo

In case of a downtime of the node, the subscriptions will be reconnected automatically recovering all new blocks from the last block that was processed.

**Note**: Load all blocks may take a long time depending on the number of blocks that need to be loaded. It is recommended to use a node with a fast internet connection. The node will be able to process all blocks in a few hours.
**Note**: Load all blocks may take a long time depending on the number of blocks that need to be loaded. It is recommended to use a node with a fast internet connection. The node will be able to process all blocks in a few hours.

### Some benchmarks

#### Using BLOCK_CONCURRENCY = 100
- 100 blocks ~ 6 seconds
- 1000 blocks ~ 30.5 seconds
- 10000 blocks ~ 4:24 minutes
- 100000 blocks ~ 39.57 minutes

- 100 blocks ~ 6 seconds
- 1000 blocks ~ 30.5 seconds
- 10000 blocks ~ 4:24 minutes
- 100000 blocks ~ 39.57 minutes

#### Using BLOCK_CONCURRENCY = 1000
- 100 blocks ~ 0.5 seconds
- 1000 blocks ~ 5 seconds
- 10000 blocks ~ 3 minutes
- 100000 blocks ~ 24 minutes

- 100 blocks ~ 0.5 seconds
- 1000 blocks ~ 5 seconds
- 10000 blocks ~ 3 minutes
- 100000 blocks ~ 24 minutes

## Change Log

Expand Down
2 changes: 1 addition & 1 deletion dev.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM node:16.15-alpine
FROM node:16.16.0-alpine

# Create app directory
WORKDIR /usr/src/app
Expand Down
11 changes: 7 additions & 4 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
version: '3.7'
networks:
ink-explorer-network:
name: ink-explorer-network
default:
driver: ink-explorer-network

services:
backend:
Expand All @@ -18,7 +21,7 @@ services:
- /usr/src/app/node_modules
- /usr/src/app/dist
networks:
ink-explorer-network:
ink-explorer-network:
aliases:
- "backend"
env_file:
Expand All @@ -33,7 +36,7 @@ services:
- POSTGRES_PASSWORD=password
- POSTGRES_DB=ink
networks:
ink-explorer-network:
ink-explorer-network:
aliases:
- "postgres"
pgadmin:
Expand All @@ -46,6 +49,6 @@ services:
depends_on:
- postgres
networks:
ink-explorer-network:
ink-explorer-network:
aliases:
- "pgadmin"
- "pgadmin"
Loading

0 comments on commit 4e264a0

Please sign in to comment.