Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(vlog): Remove env getters from vlog #1077

Merged
merged 7 commits into from
Feb 20, 2024
Merged

feat(vlog): Remove env getters from vlog #1077

merged 7 commits into from
Feb 20, 2024

Conversation

popzxc
Copy link
Member

@popzxc popzxc commented Feb 15, 2024

What ❔

  • Removes *_from_env functions from vlog crate.
  • Introduces ObservabilityConfig for core and EN with env variable based constructors that match the old behavior.
  • Changes the binaries to use new approach.

Why ❔

  • vlog is a generic library, so it should have no assumptions on the configuration system. It should be for users of vlog to decide where they get the configuration from.

Deniallugo
Deniallugo previously approved these changes Feb 20, 2024
View reviewed changes
perekopskiy
perekopskiy reviewed Feb 20, 2024
View reviewed changes
Copy link
Contributor

@perekopskiy perekopskiy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should add ObservabilityConfig to protobug config crate as well

perekopskiy
perekopskiy previously approved these changes Feb 20, 2024
View reviewed changes
@popzxc popzxc added this pull request to the merge queue Feb 20, 2024
Merged via the queue into main with commit 00d3429 Feb 20, 2024
37 checks passed
@popzxc popzxc deleted the popzxc-observability-config branch February 20, 2024 11:51
This was referenced Feb 20, 2024
Merged
Merged
RomanBrodetski pushed a commit that referenced this pull request Feb 26, 2024
@zksync-era-bot
chore(main): release core 20.8.0 (#1104)
11c8ff7 
🤖 I have created a release *beep* *boop*
---


##
[20.8.0](core-v20.7.0...core-v20.8.0)
(2024-02-26)


### Features

* Add more buckets to call tracer
([#1137](#1137))
([dacd8c9](dacd8c9))
* **api:** add a config flag for disabling filter api
([#1078](#1078))
([b486d7e](b486d7e))
* **api:** Create RPC method to return all tokens
([#1103](#1103))
([b538d1a](b538d1a))
* **api:** Implement TxSink abstraction
([#1204](#1204))
([11a34d4](11a34d4))
* **en:** Add health checks for EN components
([#1088](#1088))
([4ea1520](4ea1520))
* **en:** Start health checks early into EN lifecycle
([#1146](#1146))
([f983e80](f983e80))
* **en:** switch to tree light mode
([#1152](#1152))
([ce6c120](ce6c120))
* **en:** Take into account nonce from tx proxy
([#995](#995))
([22099cb](22099cb))
* **healthcheck:** Various healthcheck improvements
([#1166](#1166))
([1e34148](1e34148))
* Integration tests enhancement for L1
([#1209](#1209))
([a1c866c](a1c866c))
* **node_framework:** Support Eth Watch in the framework
([#1145](#1145))
([4f41b68](4f41b68))
* **shared bridge:** preparation for shared bridge migration (server)
([#1012](#1012))
([2a766a7](2a766a7))
* **vlog:** Remove env getters from vlog
([#1077](#1077))
([00d3429](00d3429))
* **vm:** Add new VM folder
([#1208](#1208))
([66cdefc](66cdefc))
* **vm:** integrate new vm version
([#1215](#1215))
([63d1f52](63d1f52))


### Bug Fixes

* **contract-verifier:** Add force_evmla flag
([#1179](#1179))
([e75aa11](e75aa11))
* **contract-verifier:** allow other zksolc settings
([#1174](#1174))
([72c60bd](72c60bd))
* **state-keeper:** Add GasForBatchTip criterion
([#1096](#1096))
([de4d729](de4d729))


### Performance Improvements

* **db:** Improve `get_logs_by_tx_hashes` query
([#1171](#1171))
([0dda7cc](0dda7cc))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
github-merge-queue bot pushed a commit that referenced this pull request Mar 5, 2024
@zksync-era-bot
chore(main): release prover 12.0.0 (#970)
7b54689 
🤖 I have created a release *beep* *boop*
---


##
[12.0.0](prover-v11.0.0...prover-v12.0.0)
(2024-03-04)


### ⚠ BREAKING CHANGES

* **prover:** Add EIP4844 support for provers subsystem
([#1200](#1200))
* Set 21 as latest protocol version
([#1262](#1262))

### Features

* Adding ability to generate 4844 setup key and refactor
([#1143](#1143))
([975f54b](975f54b))
* **api:** Remove unused and obsolete token info
([#1071](#1071))
([e920897](e920897))
* **dal:** `zksync_types::Transaction` to use protobuf for wire encoding
(BFT-407)
([#1047](#1047))
([ee94bee](ee94bee))
* **db:** Soft-remove `storage` table
([#982](#982))
([601f893](601f893))
* **en:** Integrate snapshots recovery into EN
([#1032](#1032))
([c7cfaf9](c7cfaf9))
* **healthcheck:** Various healthcheck improvements
([#1166](#1166))
([1e34148](1e34148))
* improving verification key generation
([#1050](#1050))
([6f715c8](6f715c8))
* Prover interface and L1 interface crates
([#959](#959))
([4f7e107](4f7e107))
* **prover:** Add EIP4844 support for provers subsystem
([#1200](#1200))
([6953e89](6953e89))
* **prover:** Added --recompute-if-missing option to key generator
([#1151](#1151))
([cad7278](cad7278))
* **prover:** Added 4844 circuit to verification keys
([#1141](#1141))
([8b0cc4a](8b0cc4a))
* **prover:** Adding first support for 4844 circuit
([#1155](#1155))
([6f63c53](6f63c53))
* **prover:** adding keystore object to handle reading and writing of
prover keys
([#1132](#1132))
([1471615](1471615))
* **prover:** merging key generation into a single binary
([#1101](#1101))
([6de8b84](6de8b84))
* **prover:** Moved setup key generation logic to test harness
([#1113](#1113))
([469ab06](469ab06))
* **prover:** Use new shivini function for 4844 circuits
([#1205](#1205))
([376c09e](376c09e))
* Set 21 as latest protocol version
([#1262](#1262))
([30579ef](30579ef))
* **vlog:** Remove env getters from vlog
([#1077](#1077))
([00d3429](00d3429))


### Bug Fixes

* fix link
([#1007](#1007))
([f1424ce](f1424ce))
* make `zk status prover` use the new prover table
([#1044](#1044))
([9b21d7f](9b21d7f))
* **prover:** Decouple core/ prover database management
([#1029](#1029))
([37674fd](37674fd))
* **prover:** Fix initial prover migration
([#1083](#1083))
([6d54010](6d54010))
* **prover:** QoL socket utilization
([#1020](#1020))
([13a6816](13a6816))
* update harness to include fix to new boojum OOM
([#1053](#1053))
([4976941](4976941))


### Performance Improvements

* bump harness version
([#1003](#1003))
([1cbb4c9](1cbb4c9))
* reduce memory consumption of witness generation
([#696](#696))
([dea6768](dea6768))
* upgrade harness version to improve witness generation memory spike
([#1034](#1034))
([09bbb84](09bbb84))
* use jemalloc in witness generator
([#1014](#1014))
([917b2dc](917b2dc))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
mationorato pushed a commit to lambdaclass/zksync-era that referenced this pull request Mar 20, 2024
@toni-calvin @Artemka374
@popzxc @slowli @Deniallugo @ly0va @benceharomi @mm-zk @perekopskiy @EmilLuta @AnastasiiaVashchuk @lferrigno @itegulov @jorbush @ilitteri @zk-Lumi @ColoCarletti @Oppen
Update validium exampl (#207)
bd90d92 
* feat: Add more buckets to call tracer (matter-labs#1137)

## What ❔

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

* feat(vlog): Remove env getters from vlog (matter-labs#1077)

## What ❔

- Removes `*_from_env` functions from `vlog` crate.
- Introduces `ObservabilityConfig` for core and EN with env variable
based constructors that match the old behavior.
- Changes the binaries to use new approach.

## Why ❔

- `vlog` is a generic library, so it should have no assumptions on the
configuration system. It should be for users of `vlog` to decide where
they get the configuration from.

* feat(api): Create RPC method to return all tokens (matter-labs#1103)

## What ❔

- Creates `en_syncTokens` RPC method that returns all tokens optionally
filtered by the miniblock at which the token was deployed.
- Uses this method during snapshot recovery to recover tokens.

## Why ❔

It's necessary to recover tokens in order for the VM sandbox to work
properly.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `zk spellcheck`.
- [x] Linkcheck has been run via `zk linkcheck`.

* fix(zk_stack): Use deployer private key and remove create2 address (matter-labs#1098)

## What ❔

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

---------

Signed-off-by: Danil <deniallugo@gmail.com>

* feat(shared bridge): preparation for shared bridge migration (server) (matter-labs#1012)

## What ❔

This PR aims to reduce the diff between main and matter-labs#298 to simplify the
future review.

The changes do not affect current server behavior.
Only changes related to the zkSync server went into this PR.

What didn't go into this PR:

- various variable renamings
- integration tests
- changes that depend on the future version of era-contracts

Where applicable, stubs and dummy (for now) config variables were
introduced, which can be replaced/populated later.

## Why ❔

Breaking up a PR into smaller chunks should be easier to review.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

---------

Co-authored-by: Bence Haromi <56651250+benceharomi@users.noreply.github.com>
Co-authored-by: Bence Haromi <bence.haromi@gmail.com>

* feat(en): Take into account nonce from tx proxy (matter-labs#995)

## What ❔

Take into account nonce from TxProxy, now if tx was submitted but
miniblock was not synced yet. en will return correct nonce

## Why ❔

[EN doesn't consider
](https://github.com/matter-labs/zksync-era/blob/main/core/lib/zksync_core/src/api_server/web3/namespaces/eth.rs#L303)transactions
currently residing in the TxProxy when calculating pending nonce, which
results in returned nonces being incorrect t times
## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

---------

Signed-off-by: Danil <deniallugo@gmail.com>
Co-authored-by: Alex Ostrovski <aov@matterlabs.dev>

* docs(db): Document DB invariants after snapshot recovery (matter-labs#1133)

## What ❔

Documents the invariants after snapshot recovery, e.g. in the DAL crate
readme.

## Why ❔

Better DevEx (primarily internal for now).

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `zk spellcheck`.
- [x] Linkcheck has been run via `zk linkcheck`.

* feat(prover): Added 4844 circuit to verification keys (matter-labs#1141)

## What ❔

* Added support for 4844 circuit to verification key generator

## Why ❔

* 4844 is the new circuit that will be used from the upcoming release.

* feat(en): switch to tree light mode (matter-labs#1152)

## What ❔

EN runs tree in light mode.

## Why ❔

Full tree mode is no longer required for EN as commitment generation is
moved to a new component.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

* feat: Adding ability to generate 4844 setup key and refactor (matter-labs#1143)

## What ❔

* Refactored the code, to keep passing ProverServiceDataKey as a circuit
identifier
* Added support for 4844 setup key generation

## Why ❔

* 4844 circuit was recently added.

* feat(en): Start health checks early into EN lifecycle (matter-labs#1146)

## What ❔

- Implements shared thread-safe container for health checks,
`AppHealthCheck`.
- Refactors EN initialization to start the healthcheck server early into
the node lifecycle.
- Adds a healthcheck for snapshot recovery.
- Uses healthchecks in the snapshot recovery integration test.

## Why ❔

This increases EN observability.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `zk spellcheck`.
- [x] Linkcheck has been run via `zk linkcheck`.

* feat(prover): Adding first support for 4844 circuit (matter-labs#1155)

## What ❔

* Adding first part of support for 4844 circuit in prover.
* Added it to prover enums (like proof wrapper etc).
* But it is not connected to witness generators yet.

## Why ❔

* This is a part of the larger effort to handle 4844 blob support for
era.

---------

Co-authored-by: EmilLuta <EmilLuta@users.noreply.github.com>
Co-authored-by: Emil <evl@matterlabs.dev>

* fix: Return back sepolia.json (matter-labs#1165)

## What ❔

Return back token config for sepolia

## Why ❔

It was accidentally removed

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

* perf(db): Improve `get_logs_by_tx_hashes` query (matter-labs#1171)

## What ❔

Removes ordering by `tx_index_in_block`

## Why ❔

Ordering by `tx_index_in_block` is not needed, (`miniblock_number `,
`event_index_in_block`) is enough. We currently have an index on
(`miniblock_number `, `tx_index_in_block`, `event_index_in_block`) and
it can be removed after this PR is applied.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

* fix(zk_stack): fix docker compose and prover setup  (matter-labs#1164)

## What ❔

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

---------

Signed-off-by: Danil <deniallugo@gmail.com>

* fix(contract-verifier): allow other zksolc settings (matter-labs#1174)

## What ❔

Contract verifier allows any fields in standard json input settings and
passes them to zksolc.

## Why ❔

New fields are added to settings from time to time, contract verifier
should just accept them and pass to zksolc.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

* fix(zk_stack): Use correct owner and db url (matter-labs#1178)

## What ❔

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

Signed-off-by: Danil <deniallugo@gmail.com>

* feat(prover): Added --recompute-if-missing option to key generator (matter-labs#1151)

## What ❔

* Added recompute-if-missing option to key generator
* Updated setup.sh to use this option.

## Why ❔

* This allows to quickly recompute the setup keys that were missing,
making it easier to run local provers.

* chore(prover): Remove obsolete files (matter-labs#1148)

## What ❔

* Removing data files that are no longer needed

## Why ❔

* verification_leaf_1 - is now called 'verification_scheduler'
* verification_leaf_2 - is now called "verification_node'
* witness_arttifacts.json was used to generate a set of basic circuits,
but now we depend on the test_harness to get this list
* scheduler.bin was used in the past to generate snark wrapper
verification key - but since then we moved to verification key
generation that doesn't need a real proof anymore.
*

* fix(contract-verifier): Add force_evmla flag (matter-labs#1179)

## What ❔

- adds force_evmla flag

## Why ❔

- it should be processed explicitly by verifier to pass `--force-evmla`
to zksolc

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

* feat(node_framework): Support Eth Watch in the framework (matter-labs#1145)

## What ❔
- Small change in the `EthWatch` component. Now `Eth Watch` 's `pool`
field is used instead of `ConnectionPool` passed in the Eth Watch's
`run()` method.
- Adds `Eth Watch` implementation, i. e. `EthWatchLayer` and
`EthWatchTask`, in the same way it was done for `StateKeeper `
[here](https://github.com/matter-labs/zksync-era/pull/1043/files#diff-8ff8babf7b83c79dbf96f4998cf71d888beaeb265c7ce33192b0bb0c808f662b).

Current "external" point of view for the framework(the only thing
changed is that we add ` .add_eth_watch_layer()?` after
`.add_state_keeper_layer()?`):

```rust
fn main() -> anyhow::Result<()> {
    #[allow(deprecated)] // TODO (QIT-21): Use centralized configuration approach.
    let log_format = vlog::log_format_from_env();
    let _guard = vlog::ObservabilityBuilder::new()
        .with_log_format(log_format)
        .build();

    MainNodeBuilder::new()
        .add_pools_layer()?
        .add_fee_input_layer()?
        .add_object_store_layer()?
        .add_metadata_calculator_layer()?
        .add_state_keeper_layer()?
        .add_eth_watch_layer()?
        .add_healthcheck_layer()?
        .build()
        .run()?;

    Ok(())
}
```

* Improve config files for validium

* zk fmt on improving validium config files

* modify comments

* chore: fix MIT license link (matter-labs#1195)

## What ❔

https://opensource.org/license/mit/ ->
https://opensource.org/blog/license/mit/

## Why ❔

CI is broken, seems like https://opensource.org changed the base URL for
the license pages

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [x] Linkcheck has been run via `zk linkcheck`.

* feat(api): add a config flag for disabling filter api (matter-labs#1078)

## What ❔

Added a config flag to disable all filter-related methods. If turned on,
they report that the method is not implemented.

## Why ❔

If a node is behind a load balancer then there is no way for client to
reliably hit the same node where he/she created the filter. Hence
supporting this is a very awkward UX. Making this a flag since external
nodes might want to turn this on if they only have a single node mapped
to a static address.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `zk spellcheck`.
- [x] Linkcheck has been run via `zk linkcheck`.

* Implement serde solution

* chore(prover): small features for 4844 proofs (matter-labs#1181)

## What ❔

* Added verification key and finalization hint file for 4844 circuit
* Actually start saving finalization hint when generating verification
keys
* Fixed witness vector generator for 4844 circuit - used by GPU prover

## Why ❔

* Necessary steps to start creating 4844 proofs

* refactor(state-keeper): Propagate I/O errors in state keeper (matter-labs#1080)

## What ❔

- Propagates I/O errors in the state keeper using `anyhow::Result`,
adding context to calls where appropriate.
- Propagates DB errors in `FactoryDepsDal`.

## Why ❔

Propagating errors makes it easier to attach additional context to them;
thus, it improves DevEx and code maintainability.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `zk spellcheck`.
- [x] Linkcheck has been run via `zk linkcheck`.

* feat(prover): Use new shivini function for 4844 circuits (matter-labs#1205)

## What ❔

* Start using the new shivini function for  4844 circuit

## Why ❔

* Needed for GPU provers

* fix(witness_generator): Add trusted setup to wit gens (matter-labs#1207)

## What ❔

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

* Update contracts submodule commit

* Use serve(default) instead

* Update contracts submodule commit

* feat(vm): Add new VM folder (matter-labs#1208)

## What ❔

Copy `vm_latest`, pasted as `vm_1_4_1`. Replaced all occurrence of
`vm_latest` to `vm_1_4_1` inside `vm_1_4_1` folder

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

* Add unit test

* Refactor name

* feat(api): Implement TxSink abstraction (matter-labs#1204)

## What ❔

Implements a new abstraction: `TxSink` to be used in `TxSender`.
Right now we have to use either `TxProxy` or master pool (e.g. only one
component), but the code allows invalid configurations. Moreover, mixing
two approaches on the `TxSender` level is unreasonably clunky and
complex.
Having a single abstraction seems to be a better approach.

Additionally, it's a prerequisite for implementing an API layer for the
framework.

⚠️ There are quite a few possibilities for improvements here, but I
suggest not going down the rabbit hole. As long as it's perceived as an
incremental improvement, let's not focus on the adjacent code. We can
always revisit this place next time it becomes problematic.

* Update contracts submodule commit

* Update contracts submodule commit

* Update contracts submodule commit

* set BatchCommitDataGenerator in external node init

* add panic if commitment mode differs

* use assert_eq, panic if eth response is invalid

* use a function for the function selector

* chore(docker): Exclude trusted_setup from .dockerignore (matter-labs#1224)

## What ❔

Excludes `trusted_setup.json` from .dockerignore so witness generator
docker image can be build

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

* feat: Integration tests enhancement for L1 (matter-labs#1209)

## What ❔

Make tests send transactions subsequently, use Promise.all() only on
waiting receipt.
Add more logging
Increase gas price by 30% to prevent test timeout

## Why ❔

Integrationg tests for L1 are not working properly.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

* feat(vm): integrate new vm version (matter-labs#1215)

## What ❔

Integrates new VM version

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

* Apdapt protobuf_config

* Apdapt testonly

* chore: slightly change instructions in node version to match yarn (matter-labs#1221)

## What ❔

* nit: there was a mismatch in instructions of setting node version vs
yarn version

## Why ❔

* make more clear

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

* add support for validium in external mode

* update tests

* fix(zk): Adapting zk status prover to new Database layout (matter-labs#1229)

## What ❔

* Database was fully split - with prover data now living in separate one
* Updating `zk status prover` to work

## Why ❔

* zk status prover is helpful for local debugging of prover progress.

* remove unused parameter

* Update contracts submodule commit to matter-labs

* Update contracts submodule commit

* Update contracts submodule commit

* Remove obscure use of VALIDIUM_MODE env

* Revert "use a function for the function selector"

This reverts commit 96035d8.

* Remove call implementation

* impl Detokenize for L1BatchCommitDataGeneratorMode

* Add executor_contract getter

* Call getPubdataPriceMode using Executor's contract ABI

* Fix unhandled error

* Update contracts submodule commit

* Fix L1BatchCommitDataGeneratorMode::from_tokens

* Add comment

* refactor configLine into struct

* Update contracts submodule commit

* update ext node config

* rename var

* Update contracts submodule commit

* Update contracts submodule commit

* Stash

* Update contracts submodule commit

* add validium mode assert for smallPubdata test

* remove console.log

* add documentation

* Check mode

* fix effective gas price calculation to be backward compatible with legacy txs

* Remove unused function

* Remove unnecessary test and TODOs

* add ext-node-validium.toml file

* Update how the config is set now

* Update .gitignore

* Revert "rename var"

This reverts commit d5fdb5c.

* Revert "update ext node config"

This reverts commit 272e35f.

* Revert "refactor configLine into struct"

This reverts commit 629c325.

* Delete infrastructure/zk/src/config_constants.ts

* Update config.ts

* Init system using DeploymentMode enum

* Fix lightweight-init and reinit cmds

* Fix hyperchain_wizard

* Fix conditional

* Fix merge

* Fix compilation error in tests (because of a merge)

* Update contracts submodule commit

* Remove unused imports

* Update contracts submodule commit

* Revert "Add executor_contract getter"

This reverts commit 3a9e133.

* Fix method name

* Improve assert comment

* small refactor

* Fix call to getPubdataPricingMode method

* Refactor

* Remove unused import

* Handle special case where these changes are merged before the contracts are deployed on testnet/mainnet

* Update era.dic

* Fix comment

* Address nits

* Add zksync_eth_client as dep in EN

* ConsistencyChecker now receives impl EthInterface instead of eth_client_url

* Cross-check batch commit data generation mdoe with L1

* Refactor cross-check function

* Merge main

* Rename `CommitBatches` and `CommitBatchInfo` to `CommitBatchesRollup` and `CommitBatchInfoRollup`

* Big L1CommitBatchesDataGenerator refactor

* Fix publish criterion

* Fixes?

* Remove FIXME

* Restore contracts

* Restore consistency checker

- First splits the functionality of
  `L1CommitBatchesDataGenerator::l1_commit_data` into two separate
  functions, one for a single batch and one for the whole array that
  includes the previous block.
- Then uses that for publish criteria and consistency checking, as they
  work on a per-batch basis.

* Fix double hardcoding to CommitBatchesRollup

* Fix double 'into_tokens'

* Lints

* Lints

* Remove generic in ConsistencyChecker::new

* Slightly more specific comments to new structures

* Minor refactor for encode_l1_commit

* Doc comments

* Copy-paste

* add PubdataPricing trait

* fix tests

* fix protobuf

* Revert "Merge remote-tracking branch 'origin/validium_mode_example_crate' into add-commitment-mode-cross-check"

This reverts commit e392d18, reversing
changes made to 08c969d.

* zk fmt

* move PubdataPricing

* fmt

* fmt

* spellcheck

* refactor state_keeper tests

* Update submodule commit

* remove validium check

* fix typo and add await

* add threshold variable

* remove bob account

* Ensure we use the result of ensure_l1_batch_commit_data_generation_mode

* Add unit tests for ensure_l1_batch_commit_data_generation_mode

* add test_casing for unittests

* remove debugging assert

* Update submodule commit

* refactor eth_sender unittests

* refactor state_keeper unittests

* rename DeploymentMode

* refactor gas_adjuster unittests

* Revert "fix effective gas price calculation to be backward compatible with legacy txs"

This reverts commit 8525ce9.

* move DeploymentMode to testonly

* Update the legacy test so that the gas price from the legacy transaction is less than or equal to the expected gas price

* Restore mistakenly removed comment

* Fix merge conflict

* Revert "Ensure we use the result of ensure_l1_batch_commit_data_generation_mode"

This reverts commit dd73f50.
Both `async` functions and functions returning a `Result<_>` are already
`must_use` by default, so the annotation is redundant.

* Rephrase comment

* Refactor mock errors by taking the value

* Use assert_matches for Ok cases

* zk fmt

* Update era.dic

* refactor tests

* fix build commit tx test

---------

Signed-off-by: Danil <deniallugo@gmail.com>
Co-authored-by: Lech <88630083+Artemka374@users.noreply.github.com>
Co-authored-by: Igor Aleksanov <popzxc@yandex.ru>
Co-authored-by: Alex Ostrovski <slowli@users.noreply.github.com>
Co-authored-by: Danil <deniallugo@gmail.com>
Co-authored-by: Lyova Potyomkin <lyova.potyomkin@gmail.com>
Co-authored-by: Bence Haromi <56651250+benceharomi@users.noreply.github.com>
Co-authored-by: Bence Haromi <bence.haromi@gmail.com>
Co-authored-by: Alex Ostrovski <aov@matterlabs.dev>
Co-authored-by: Marcin M <128217157+mm-zk@users.noreply.github.com>
Co-authored-by: perekopskiy <53865202+perekopskiy@users.noreply.github.com>
Co-authored-by: EmilLuta <EmilLuta@users.noreply.github.com>
Co-authored-by: Emil <evl@matterlabs.dev>
Co-authored-by: AnastasiiaVashchuk <72273339+AnastasiiaVashchuk@users.noreply.github.com>
Co-authored-by: Leandro Ferrigno <leanrafa@gmail.com>
Co-authored-by: Daniyar Itegulov <di@matterlabs.dev>
Co-authored-by: Jordi <jordi.bonet.valiente@lambdaclass.com>
Co-authored-by: ilitteri <ilitteri@fi.uba.ar>
Co-authored-by: Santiago Pittella <santiagopittella@Santiagos-MacBook-Air-6.local>
Co-authored-by: Daniel Lumi <149794418+zk-Lumi@users.noreply.github.com>
Co-authored-by: Ivan Litteri <67517699+ilitteri@users.noreply.github.com>
Co-authored-by: Joaquin Carletti <joaquin.carletti@lambdaclass.com>
Co-authored-by: Mario Rugiero <mrugiero@gmail.com>
Co-authored-by: Joaquin Carletti <56092489+ColoCarletti@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Deniallugo Deniallugo Deniallugo approved these changes

@perekopskiy perekopskiy perekopskiy approved these changes

@slowli slowli Awaiting requested review from slowli

@montekki montekki Awaiting requested review from montekki

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@popzxc @Deniallugo @perekopskiy