Skip to content

Commit

Permalink
feat(external-node): external node distributed operation mode (#1457)
Browse files Browse the repository at this point in the history
## What ❔

Adds a way to run External Node in a "distributed setup": that if there
is a need to launch different components of the EN on different machines
for performance reasons. The currently supported component split is
* `core`
* `tree`
* `tree_api` (depends on `tree`)
* `api` (may be split into two components: `http_api` and `ws_api`)

Example setup:

```
zk external-node -- --components=core
```

```
 env EN_HEALTHCHECK_PORT=3082 EN_PROMETHEUS_PORT=3323 zk external-node -- --components="tree,tree_api"
```
## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.
  • Loading branch information
montekki committed Apr 2, 2024
1 parent afa1cf1 commit 777ffca
Show file tree
Hide file tree
Showing 4 changed files with 380 additions and 97 deletions.
25 changes: 25 additions & 0 deletions core/bin/external_node/src/config/mod.rs
Expand Up @@ -261,6 +261,19 @@ pub struct OptionalENConfig {
pub l1_batch_commit_data_generator_mode: L1BatchCommitDataGeneratorMode,
}

#[derive(Debug, Clone, PartialEq, Deserialize)]
pub struct ApiComponentConfig {
/// Address of the tree API used by this EN in case it does not have a
/// local tree component running and in this case needs to send requests
/// to some external tree API.
pub tree_api_url: Option<String>,
}

#[derive(Debug, Clone, PartialEq, Deserialize)]
pub struct TreeComponentConfig {
pub api_port: Option<u16>,
}

impl OptionalENConfig {
const fn default_filters_limit() -> usize {
10_000
Expand Down Expand Up @@ -554,6 +567,8 @@ pub struct ExternalNodeConfig {
pub optional: OptionalENConfig,
pub remote: RemoteENConfig,
pub consensus: Option<consensus::Config>,
pub api_component: ApiComponentConfig,
pub tree_component: TreeComponentConfig,
}

impl ExternalNodeConfig {
Expand All @@ -568,6 +583,14 @@ impl ExternalNodeConfig {
.from_env::<OptionalENConfig>()
.context("could not load external node config")?;

let api_component_config = envy::prefixed("EN_API")
.from_env::<ApiComponentConfig>()
.context("could not load external node config")?;

let tree_component_config = envy::prefixed("EN_TREE")
.from_env::<TreeComponentConfig>()
.context("could not load external node config")?;

let client = HttpClientBuilder::default()
.build(required.main_node_url()?)
.expect("Unable to build HTTP client for main node");
Expand Down Expand Up @@ -619,6 +642,8 @@ impl ExternalNodeConfig {
required,
optional,
consensus: read_consensus_config().context("read_consensus_config()")?,
tree_component: tree_component_config,
api_component: api_component_config,
})
}
}
Expand Down

0 comments on commit 777ffca

Please sign in to comment.