test: integrate devmode and SDK for custom test txs #1306
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jyeshe
reviewed
May 9, 2023
| }; | ||
|
|
||
| const devModeRequest = async (path) => { | ||
| const response = await fetch(`http://mdw.aeternity.localhost:3313${path}`, { |
There was a problem hiding this comment.
This port is the port exposed exclusively by dev mode, which allows you to create key/micro blocks on demand
There was a problem hiding this comment.
Got ECONNREFUSED:
$ docker-compose -f docker-compose-dev.yml run node_sdk node index.js
Creating ae_mdw_node_sdk_run ... done
/app/node_modules/node-fetch/lib/index.js:1461
reject(new FetchError(`request to ${request.url} failed, reason: ${err.message}`, 'system', err));
^
FetchError: request to http://mdw.aeternity.localhost:3313/emit_kb failed, reason: connect ECONNREFUSED 127.0.0.1:3313
at ClientRequest.<anonymous> (/app/node_modules/node-fetch/lib/index.js:1461:11)
at ClientRequest.emit (node:events:511:28)
at Socket.socketErrorListener (node:_http_client:495:9)
at Socket.emit (node:events:511:28)
at emitErrorNT (node:internal/streams/destroy:151:8)
at emitErrorCloseNT (node:internal/streams/destroy:116:3)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
type: 'system',
errno: 'ECONNREFUSED',
code: 'ECONNREFUSED'
}
Node.js v20.1.0
ERROR: 1
There was a problem hiding this comment.
Tried running the sequence of run_sdk_tests.sh that creates the dev image with DEV_MODE=true and exports the AETERNITY_CONFIG=/home/aeternity/aeternity-dev.yaml
jyeshe
reviewed
May 10, 2023
| These tests allow you to create your own transactions using the devmode (plus the JS SDK). To add newer tests you need to: | ||
|
|
||
| 1. Add the transactions creation on `node_sdk/index.js`. | ||
| 2. Run the JavaScript file using `docker-compose -f docker-compose-dev.yml run node_sdk node index.js`. |
There was a problem hiding this comment.
Needed npm i @aeternity/aepp-sdk under node_sdk because the global is not added to the container
jyeshe
reviewed
May 10, 2023
| @@ -39,6 +39,8 @@ RUN BUILD_REV="$(git log -1 --format=%h)" && echo $BUILD_REV > /home/aeternity/n | |||
| WORKDIR /home/aeternity/node | |||
|
|
|||
| # Download, and unzip latest aeternity release archive | |||
| ARG DEV_MODE="false" | |||
There was a problem hiding this comment.
Shouldn't be on args in docker-compose-dev.yml?
There was a problem hiding this comment.
It's ok, found in the run_sdk_tests.sh the docker-compose -f docker-compose-dev.yml build --build-arg DEV_MODE=true ae_mdw
The docker-compose -f docker-compose-dev.yml run node_sdk node index.js has not appeared in the output though, what would be missing (:?
$ scripts/run_sdk_tests.sh
+ docker-compose -f docker-compose-dev.yml build --build-arg DEV_MODE=true ae_mdw
Building ae_mdw
[+] Building 0.3s (51/51) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 5.02kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 435B 0.0s
=> [internal] load metadata for docker.io/hexpm/elixir:1.13.4-erlang-23.3.4.18-debian-bullseye-20220801-slim 0.0s
=> [internal] load metadata for docker.io/hexpm/elixir:1.13.4-erlang-23.3.4.18-debian-bullseye-20220801 0.0s
=> [internal] load build context 0.2s
=> => transferring context: 435.93kB 0.2s
=> [stage-1 1/12] FROM docker.io/hexpm/elixir:1.13.4-erlang-23.3.4.18-debian-bullseye-20220801 0.0s
=> [builder 1/33] FROM docker.io/hexpm/elixir:1.13.4-erlang-23.3.4.18-debian-bullseye-20220801-slim 0.0s
=> CACHED [stage-1 2/12] RUN apt-get update -y && apt-get install -y git curl libstdc++6 openssl libncurses5 loc 0.0s
=> CACHED [stage-1 3/12] RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen 0.0s
=> CACHED [stage-1 4/12] WORKDIR /home/aeternity/node 0.0s
=> CACHED [builder 2/33] RUN apt-get update -y && apt-get install -y build-essential git sed curl libncurses5 li 0.0s
=> CACHED [builder 3/33] RUN mkdir -p /home/aeternity/node 0.0s
=> CACHED [builder 4/33] COPY ./docker/aeternity.yaml /home/aeternity/aeternity.yaml 0.0s
=> CACHED [builder 5/33] RUN mkdir /home/aeternity/node/ae_mdw 0.0s
=> CACHED [builder 6/33] COPY .git .git 0.0s
=> CACHED [builder 7/33] RUN BUILD_REV="$(git log -1 --format=%h)" && echo $BUILD_REV > /home/aeternity/node/ae_ 0.0s
=> CACHED [builder 8/33] WORKDIR /home/aeternity/node 0.0s
=> CACHED [builder 9/33] RUN mkdir -p ./local/rel/aeternity/data/mnesia 0.0s
=> CACHED [builder 10/33] RUN curl -L --output aeternity.tar.gz https://github.com/aeternity/aeternity/releases/d 0.0s
=> CACHED [builder 11/33] RUN chmod +x /home/aeternity/node/local/rel/aeternity/bin/aeternity 0.0s
=> CACHED [builder 12/33] RUN cp -r ./local/rel/aeternity/lib local/ 0.0s
=> CACHED [builder 13/33] RUN sed -i 's/{max_skip_body_length, [0-9]\+}/{max_skip_body_length, 10240}/g' /home/ae 0.0s
=> CACHED [builder 14/33] RUN /home/aeternity/node/local/rel/aeternity/bin/aeternity check_config /home/aeternity 0.0s
=> CACHED [builder 15/33] WORKDIR /home/aeternity/node/ae_mdw 0.0s
=> CACHED [builder 16/33] RUN mix local.hex --force && mix local.rebar --force 0.0s
=> CACHED [builder 17/33] COPY mix.exs mix.lock ./ 0.0s
=> CACHED [builder 18/33] RUN mix deps.get --only dev 0.0s
=> CACHED [builder 19/33] RUN mkdir config 0.0s
=> CACHED [builder 20/33] COPY config/config.exs config/dev.exs config/ 0.0s
=> CACHED [builder 21/33] RUN mix deps.compile 0.0s
=> CACHED [builder 22/33] COPY priv priv 0.0s
=> CACHED [builder 23/33] COPY lib lib 0.0s
=> CACHED [builder 24/33] COPY scripts scripts 0.0s
=> CACHED [builder 25/33] COPY docs docs 0.0s
=> CACHED [builder 26/33] RUN mix compile 0.0s
=> CACHED [builder 27/33] COPY config/runtime.exs config/ 0.0s
=> CACHED [builder 28/33] RUN mix run --no-start -e 'IO.puts(Mix.Project.config[:version])' >AEMDW_VERSION 0.0s
=> CACHED [builder 29/33] RUN scripts/swagger-docs.py >priv/static/swagger/swagger_v2.yaml 0.0s
=> CACHED [builder 30/33] COPY docker/aeplugin_dev_mode aeplugin_dev_mode 0.0s
=> CACHED [builder 31/33] RUN ./scripts/install-devmode.sh 0.0s
=> CACHED [builder 32/33] COPY rel rel 0.0s
=> CACHED [builder 33/33] RUN mix release 0.0s
=> CACHED [stage-1 5/12] COPY --from=builder /home/aeternity/node/ae_mdw/_build/dev/rel/ae_mdw ./ 0.0s
=> CACHED [stage-1 6/12] COPY --from=builder /home/aeternity/node/local ./local 0.0s
=> CACHED [stage-1 7/12] COPY ./docker/aeternity.yaml /home/aeternity/aeternity.yaml 0.0s
=> CACHED [stage-1 8/12] COPY ./docker/aeternity-dev.yaml /home/aeternity/aeternity-dev.yaml 0.0s
=> CACHED [stage-1 9/12] COPY ./docker/healthcheck.sh /home/aeternity/healthcheck.sh 0.0s
=> CACHED [stage-1 10/12] RUN chmod +x /home/aeternity/healthcheck.sh 0.0s
=> CACHED [stage-1 11/12] RUN useradd --uid 1000 --shell /bin/bash aeternity && chown -R aeternity:aeternity 0.0s
=> CACHED [stage-1 12/12] RUN mkdir -p ./local/rel/aeternity/data/mnesia && mkdir -p ./local/rel/aeternity/da 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:dec068d4ec18501b52eb739a5a5d0ec33811eaaac9047383d7dafb0f7960cc23 0.0s
=> => naming to docker.io/aeternity/ae_mdw_dev:latest 0.0s
+ export AETERNITY_CONFIG=/home/aeternity/aeternity-dev.yaml
+ AETERNITY_CONFIG=/home/aeternity/aeternity-dev.yaml
+ docker-compose -f docker-compose-dev.yml up --detach ae_mdw
Creating ae_mdw_ae_mdw_1 ... done
+ sleep 10
+ docker-compose -f docker-compose-dev.yml exec ae_mdw ./bin/ae_mdw rpc ':aeplugin_dev_mode_app.start_unlink()'
Running the AETERNITY_CONFIG=/home/aeternity/aeternity-dev.yaml docker-compose -f docker-compose-dev.yml run node_sdk node index.js after the run_sdk_tests.sh gives the ECONNREFUSED.
There was a problem hiding this comment.
This is probably due to the server going down after running it with --detach, try running
docker-compose -f docker-compose-dev.yml up --detach ae_mdw without the --detach flag to see if the server continues to run or returns an error.
Two things I forgot to include as part of the script are:
- Using a different database (you will need to clear the data/ directory before running run_sdk_test)
- Like you mentioned above running
npm installoryarn installto install node missing deps
There was a problem hiding this comment.
It fails on:
ae_mdw_1 | =INFO REPORT==== 10-May-2023::15:39:41.883304 ===
ae_mdw_1 | Reading config file /home/aeternity/aeternity-dev.yaml
ae_mdw_1 |
ae_mdw_1 | =INFO REPORT==== 10-May-2023::15:39:41.932548 ===
ae_mdw_1 | validation: /home/aeternity/aeternity-dev.yaml
ae_mdw_1 | result: [{error,[{data_invalid,#{<<"$schema">> =>
ae_mdw_1 | <<"http://json-schema.org/draft-04/schema#">>,
...
ae_mdw_1 | [{setup,run_setup,0,
ae_mdw_1 | [{file,
ae_mdw_1 | "/home/aeternity/node/ae_mdw/deps/setup/src/setup.erl"},
ae_mdw_1 | {line,946}]},
ae_mdw_1 | {ae_plugin_utils,start_aecore,1,
ae_mdw_1 | [{file,
ae_mdw_1 | "/home/aeternity/node/ae_mdw/deps/ae_plugin/src/ae_plugin_utils.erl"},
ae_mdw_1 | {line,36}]},
ae_mdw_1 | {ae_plugin_app,start,2,
ae_mdw_1 | [{file,
ae_mdw_1 | "/home/aeternity/node/ae_mdw/deps/ae_plugin/src/ae_plugin_app.erl"},
ae_mdw_1 | {line,18}]},
ae_mdw_1 | {application_master,start_it_old,4,
ae_mdw_1 | [{file,"application_master.erl"},{line,277}]}]}}}}
ae_mdw_1 | type: permanent
ae_mdw_1 |
ae_mdw_1 | {"Kernel pid terminated",application_controller,"{application_start_failure,ae_plugin,{bad_return,{{ae_plugin_app,start,[normal,[]]},{'EXIT',{{validation_failed,[{error,[{data_invalid,#{<<\"$schema\">> => <<\"http://json-schema.org/draft-04/schema#\">>,<<\"additionalProperties\">> => false,<<\"definitions\">> => #{<<\"key_value_pattern\">> => #{<<\"pattern\">> => <<\"^[a-zA-Z0-9\\-_\\.]+\\h*:\\h*[0-9]+(\\h*,\\h*[a-zA-Z_]+\\h*:\\h*[0-9]+)*\">>}},<<\"properties\">> => #{<<\"blocked_peers\">>
...
jyeshe
reviewed
May 10, 2023
| if [ "$DEV_MODE" = "true" ]; then | ||
| echo "Installing dev mode.." | ||
| mkdir src | ||
| cp -r aeplugin_dev_mode/src src/aeplugin_dev_mode |
There was a problem hiding this comment.
$ DEV_MODE=true scripts/install-devmode.sh
+ '[' true = true ']'
+ echo 'Installing dev mode..'
Installing dev mode..
+ mkdir src
+ cp -r aeplugin_dev_mode/src src/aeplugin_dev_mode
cp: cannot stat 'aeplugin_dev_mode/src': No such file or directory
There was a problem hiding this comment.
May we move this to docker dir? It's not supposed to be called directly like I did, right?
jyeshe
reviewed
May 10, 2023
Great idea, done! |
jyeshe
reviewed
May 11, 2023
There was a problem hiding this comment.
Deleting data subdirs and running ./scripts/run_sdk_tests.sh gives without --detach:
ae_mdw_1 | =INFO REPORT==== 11-May-2023::09:36:32.029093 ===
ae_mdw_1 | validation: /home/aeternity/aeternity-dev.yaml
ae_mdw_1 | result: [{error,[{data_invalid,#{<<"$schema">> =>
ae_mdw_1 | <<"http://json-schema.org/draft-04/schema#">>,
This is a problem with the new structure of the config file with 6.8.1. Try updating the NODE_VERSION to it like this |
sborrazas
force-pushed
the
devmode-sdk-tests
branch
from
4cf2fbe
to
c94e16c
Compare
jyeshe
reviewed
May 15, 2023
There was a problem hiding this comment.
Could run ./scripts/run_sdk_tests.sh after 6.8.1 rebase and it created the transaction.
On final step got this:
Excluding tags: [:test, :integration]
Including tags: [:devmode]
1) test tx it gets transactions in forwards order (Devmode.AeMdwWeb.TxControllerTest)
test/devmode/ae_mdw_web/tx_controller_test.exs:13
match (=) failed
code: assert %{"data" => [tx]} = conn |> get("/v2/txs", direction: "forward", limit: 1) |> json_response(200)
left: %{"data" => [tx]}
right: %{"data" => [], "next" => nil, "prev" => nil}
stacktrace:
test/devmode/ae_mdw_web/tx_controller_test.exs:16: (test)
Finished in 11.9 seconds (11.9s async, 0.04s sync)
861 tests, 1 failure, 860 excluded
Randomized with seed 211743
ERROR: 2
Full log: devmode.log
sborrazas
force-pushed
the
devmode-sdk-tests
branch
from
c94e16c
to
644882b
Compare
sborrazas
force-pushed
the
devmode-sdk-tests
branch
from
644882b
to
5e8963a
Compare
jyeshe
approved these changes
Jun 16, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I've added a sample
tx_controller_test.exsfile, it would be reasonable to add additional test cases with more complex use cases in the future (contract calls, NFTs transfers, etc).