Skip to content

Commit

Permalink
[refactor] #4161: rewrite config, _minimally_ (#4239)
Browse files Browse the repository at this point in the history
* [refactor]: wip

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: update structure

- exclude genesis block loading from config
- construct `KeyPair` on `iroha` completion
- use full field names in `complete()`s

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: apply lints

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [feat]: include more ENV vars, refactor

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [feat]: impl merging

- `UserField` wrap instead of `Option`
- move trusted peers uniqueness check

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: update Kagami

- Remove `config` subcommand
- Update default `genesis` building

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: restructure code

- move generic tools to `iroha_config_base`
- remove `iroha_client_config` crate
- define client config in `iroha_client::config`
- add client config sample TOML

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: update `iroha_logger`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: update `iroha_telemetry`

- update usage of `iroha_config`
- use `Duration` in `RetryPeriod`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [fix]: fix util macro

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: re-struct config, update `iroha_core`

- split "user-layer" and "actual" config modules
- update logger, telemetry, and kagami (genesis)

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: update more crates

- `iroha_client`
- `iroha_torii`
- move Torii `uri` to `iroha_torii_const`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [feat]: compile `iroha`!

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [feat]: compile `iroha_client`!

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [feat]: compile `iroha_client_cli`!

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [feat]: compile everything *_*

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [fix]: chores

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: pass tests

- update signature of `PeerId::new` (avoid extra clone)
- fix `UserField::set`
- fix deps of `iroha_config_base`
- resolve runtime todos in `iroha_config`
- update `iroha_swarm` generation output
- some other refactoring

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [feat]: implement `extends`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [chore]: update default snapshot storage

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [fix]: update after rebase

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: update just everything

- `test_env.py`: use toml configs
- remove other channel configs
- put example configs into `config_samples` dir
- put docker setup into `config_samples/swarm` dir
- pytests: split settings for CLI and config paths
- make test env runnable

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: refactored config boilerplate into dedicated modules

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: apply some lints

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: update config naming in `[queue]`

also run tests with all features enabled

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [feat]: implement `Config::load` shorthand

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [docs]: add comment

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [ci]: fix typo

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [ci]: install `tomli_w` via pacman

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [chore]: fix whitespace

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [revert]: update the snapshot store path

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: fix pytests

- do not mutate client config from tests,
  override via env instead
- add `TORII_URL` env var to client config

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: apply suggestions from code review

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: curl up `SumeragiStartArgs`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: refine telemetry

- simplify `regular_telemetry` to just `telemetry`
- clearer config passing

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: use `Infallible`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: simplify client config

- `nonce` instead of `add_nonce`
- move `[api]` to root

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: update `--config` arg

- remove default value
- remove `IROHA_CONFIG` env

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [docs]: update `read_config_and_genesis` docs

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: chore

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: just `idle_time`, without `query_`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [revert]: use `ident_length_limits` in _actual_ config

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: use `Config`, not vague `Root`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [chore]: remove comment

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: use `capacity` term in Queue

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: `set_creation_time_ms`, use `*` instead of `mul`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [fix]: remove dead code

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: update `--config` args

- use `PathBuf` again
- remove default value at Client CLI

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [chore]: cleaning

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: use `strum` in place of `parse_display`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: no unsafe code any more

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: docs & refinements

- document code in:
  - `iroha_config_base`
  - `iroha_config`
  - `iroha_client::config`
- refactor `iroha_config_base` APIs
- move `ExtendsPaths` into `iroha_config_base`
- remove `[iroha]` user config section
  - move `chain_id` and key pair to the root
  - move `p2p_address` to `network.address`
- rename `user_layer` modules to `user`
- add `_bytes` suffix for relevant fields,
  put a TODO to add a newtype for it

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [docs]: fill `peer.example.toml`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [chore]: re-export `ConfigurationDTO` from client

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [test]: fix them

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: apply lints

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [test]: fix pytests

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [misc]: re-arrange sample configurations

- use `configs` dir again
- name example configs as _templates_

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [docs]: update README

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: remove `parse-display` from deps

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: remove extra `iroha_config_base` exposure from the client

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [fix]: import `Deserialize` in the macro

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: use `PrivateKey::into_raw` instead

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [ci]: use `--break-system-packages` `pip` flag

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [fix]: regenerate swarms

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [ci]: fix pytests workflow

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [fix]: remove `PrivateKey::payload()` access

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [docs]: fix "unable to validate" doc

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: accept `Duration` for `set_creation_time`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: `PrivateKey::to_raw`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [fix]: do not extend trusted peers in config

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: hide user view from `iroha_client::config`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: rename `*configuration` to `config` everywhere﹡

﹡except crypto

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [fix]: update default wasm fuel limit

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: change docs and methods of `WebLogin`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [fix]: trusted peers and config tests

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: rename `telemetry.dev.out_file`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [docs]: update `GenesisNetwork::new` errors

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: use `serde_with`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [misc]: move `nonzero_ext` to workspace level

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [fix]: make `--config` optional

It is still possible to set full config via env
(e.g. `iroha_swarm` case)

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [chore]: remove "regular" from telemetry re-export

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [chore]: rename imports in `wasm.rs`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: lints

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: use `serde_with` in swarm

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: rename parameters, cover full config in tests, fix bugs

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [test]: cover absolute paths

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: update client configs, cover full in tests

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [refactor]: lints

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [fix]: also rename ENVs

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [build]: add notes to Dockerfile

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [chore]: update style in `iroha_test_config.toml`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [test]: fix old params in `test_env.py`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

* [test]: add a note to `panic_on_invalid_genesis.sh`

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>

---------

Signed-off-by: Dmitry Balashov <43530070+0x009922@users.noreply.github.com>
  • Loading branch information
0x009922 committed Feb 16, 2024
1 parent 0f83d26 commit 9644767
Show file tree
Hide file tree
Showing 175 changed files with 5,189 additions and 5,785 deletions.
15 changes: 6 additions & 9 deletions .github/workflows/iroha2-dev-pr.yml
Expand Up @@ -28,12 +28,6 @@ jobs:
- name: Check genesis.json
if: always()
run: ./scripts/tests/consistency.sh genesis
- name: Check client/config.json
if: always()
run: ./scripts/tests/consistency.sh client
- name: Check peer/config.json
if: always()
run: ./scripts/tests/consistency.sh peer
- name: Check schema.json
if: always()
run: ./scripts/tests/consistency.sh schema
Expand Down Expand Up @@ -144,11 +138,10 @@ jobs:
- uses: Swatinem/rust-cache@v2
- name: Build binaries
run: |
cargo build --bin iroha_client_cli
cargo build --bin kagami
cargo build --bin iroha
cargo build -p iroha_client_cli -p kagami -p iroha
- name: Setup test Iroha 2 environment on the bare metal
run: |
pip3 install -r scripts/requirements.txt --no-input --break-system-packages
./scripts/test_env.py setup
- name: Mark binaries as executable
run: |
Expand All @@ -159,6 +152,10 @@ jobs:
poetry install
- name: Run client cli tests
working-directory: client_cli/pytests
env:
# prepared by `test_env.py`
CLIENT_CLI_BINARY: ../../test/iroha_client_cli
CLIENT_CLI_CONFIG: ../../test/client.toml
run: |
poetry run pytest
- name: Cleanup test environment
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/iroha2-release-pr.yml
Expand Up @@ -36,6 +36,7 @@ jobs:
cargo build --bin iroha
- name: Setup test Iroha 2 environment on bare metal
run: |
pip3 install -r scripts/requirements.txt --no-input --break-system-packages
./scripts/test_env.py setup
- name: Mark binaries as executable
run: |
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Expand Up @@ -222,7 +222,7 @@ Follow these commit guidelines:
- To run the source-code based tests, execute [`cargo test`](https://doc.rust-lang.org/cargo/commands/cargo-test.html) in the Iroha root. Note that this is a long process.
- To run benchmarks, execute [`cargo bench`](https://doc.rust-lang.org/cargo/commands/cargo-bench.html) from the Iroha root. To help debug benchmark outputs, set the `debug_assertions` environment variable like so: `RUSTFLAGS="--cfg debug_assertions" cargo bench`.
- If you are working on a particular component, be mindful that when you run `cargo test` in a [workspace](https://doc.rust-lang.org/cargo/reference/workspaces.html), it will only run the tests for that workspace, which usually doesn't include any [integration tests](https://www.testingxperts.com/blog/what-is-integration-testing).
- If you want to test your changes on a minimal network, the provided [`docker-compose.yml`](docker-compose.yml) creates a network of 4 Iroha peers in docker containers that can be used to test consensus and asset propagation-related logic. We recommend interacting with that network using either [`iroha-python`](https://github.com/hyperledger/iroha-python), or the included `iroha_client_cli`.
- If you want to test your changes on a minimal network, the provided [`docker-compose.yml`](configs/swarm/docker-compose.yml) creates a network of 4 Iroha peers in docker containers that can be used to test consensus and asset propagation-related logic. We recommend interacting with that network using either [`iroha-python`](https://github.com/hyperledger/iroha-python), or the included `iroha_client_cli`.
- Do not remove failing tests. Even tests that are ignored will be run in our pipeline eventually.
- If possible, please benchmark your code both before and after making your changes, as a significant performance regression can break existing users' installations.

Expand Down
148 changes: 108 additions & 40 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 9644767

Please sign in to comment.