Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update validium exampl #207

Merged
merged 207 commits into from
Mar 20, 2024
Merged

Conversation

toni-calvin
Copy link

What ❔

Why ❔

Checklist

  • PR title corresponds to the body of PR (we generate changelog entries from PRs).
  • Tests for the changes have been added / updated.
  • Documentation comments have been added / updated.
  • Code has been formatted via zk fmt and zk lint.
  • Spellcheck has been run via zk spellcheck.
  • Linkcheck has been run via zk linkcheck.

Artemka374 and others added 30 commits February 19, 2024 15:03
## What ❔

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.
## What ❔

- Removes `*_from_env` functions from `vlog` crate.
- Introduces `ObservabilityConfig` for core and EN with env variable
based constructors that match the old behavior.
- Changes the binaries to use new approach.

## Why ❔

- `vlog` is a generic library, so it should have no assumptions on the
configuration system. It should be for users of `vlog` to decide where
they get the configuration from.
## What ❔

- Creates `en_syncTokens` RPC method that returns all tokens optionally
filtered by the miniblock at which the token was deployed.
- Uses this method during snapshot recovery to recover tokens.

## Why ❔

It's necessary to recover tokens in order for the VM sandbox to work
properly.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `zk spellcheck`.
- [x] Linkcheck has been run via `zk linkcheck`.
…atter-labs#1098)

## What ❔

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

---------

Signed-off-by: Danil <deniallugo@gmail.com>
…matter-labs#1012)

## What ❔

This PR aims to reduce the diff between main and #298 to simplify the
future review.

The changes do not affect current server behavior.
Only changes related to the zkSync server went into this PR.

What didn't go into this PR:

- various variable renamings
- integration tests
- changes that depend on the future version of era-contracts

Where applicable, stubs and dummy (for now) config variables were
introduced, which can be replaced/populated later.

## Why ❔

Breaking up a PR into smaller chunks should be easier to review.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

---------

Co-authored-by: Bence Haromi <56651250+benceharomi@users.noreply.github.com>
Co-authored-by: Bence Haromi <bence.haromi@gmail.com>
## What ❔

Take into account nonce from TxProxy, now if tx was submitted but
miniblock was not synced yet. en will return correct nonce

## Why ❔

[EN doesn't consider
](https://github.com/matter-labs/zksync-era/blob/main/core/lib/zksync_core/src/api_server/web3/namespaces/eth.rs#L303)transactions
currently residing in the TxProxy when calculating pending nonce, which
results in returned nonces being incorrect t times
## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

---------

Signed-off-by: Danil <deniallugo@gmail.com>
Co-authored-by: Alex Ostrovski <aov@matterlabs.dev>
…#1133)

## What ❔

Documents the invariants after snapshot recovery, e.g. in the DAL crate
readme.

## Why ❔

Better DevEx (primarily internal for now).

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `zk spellcheck`.
- [x] Linkcheck has been run via `zk linkcheck`.
## What ❔

* Added support for 4844 circuit to verification key generator

## Why ❔

* 4844 is the new circuit that will be used from the upcoming release.
## What ❔

EN runs tree in light mode.

## Why ❔

Full tree mode is no longer required for EN as commitment generation is
moved to a new component.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.
…labs#1143)

## What ❔

* Refactored the code, to keep passing ProverServiceDataKey as a circuit
identifier
* Added support for 4844 setup key generation

## Why ❔

* 4844 circuit was recently added.
## What ❔

- Implements shared thread-safe container for health checks,
`AppHealthCheck`.
- Refactors EN initialization to start the healthcheck server early into
the node lifecycle.
- Adds a healthcheck for snapshot recovery.
- Uses healthchecks in the snapshot recovery integration test.

## Why ❔

This increases EN observability.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `zk spellcheck`.
- [x] Linkcheck has been run via `zk linkcheck`.
## What ❔

* Adding first part of support for 4844 circuit in prover.
* Added it to prover enums (like proof wrapper etc).
* But it is not connected to witness generators yet.

## Why ❔

* This is a part of the larger effort to handle 4844 blob support for
era.

---------

Co-authored-by: EmilLuta <EmilLuta@users.noreply.github.com>
Co-authored-by: Emil <evl@matterlabs.dev>
## What ❔

Return back token config for sepolia

## Why ❔

It was accidentally removed

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.
## What ❔

Removes ordering by `tx_index_in_block`

## Why ❔

Ordering by `tx_index_in_block` is not needed, (`miniblock_number `,
`event_index_in_block`) is enough. We currently have an index on
(`miniblock_number `, `tx_index_in_block`, `event_index_in_block`) and
it can be removed after this PR is applied.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.
## What ❔

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

---------

Signed-off-by: Danil <deniallugo@gmail.com>
## What ❔

Contract verifier allows any fields in standard json input settings and
passes them to zksolc.

## Why ❔

New fields are added to settings from time to time, contract verifier
should just accept them and pass to zksolc.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.
## What ❔

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.

Signed-off-by: Danil <deniallugo@gmail.com>
…atter-labs#1151)

## What ❔

* Added recompute-if-missing option to key generator
* Updated setup.sh to use this option.

## Why ❔

* This allows to quickly recompute the setup keys that were missing,
making it easier to run local provers.
## What ❔

* Removing data files that are no longer needed

## Why ❔

* verification_leaf_1 - is now called 'verification_scheduler'
* verification_leaf_2 - is now called "verification_node'
* witness_arttifacts.json was used to generate a set of basic circuits,
but now we depend on the test_harness to get this list
* scheduler.bin was used in the past to generate snark wrapper
verification key - but since then we moved to verification key
generation that doesn't need a real proof anymore.
*
## What ❔

- adds force_evmla flag

## Why ❔

- it should be processed explicitly by verifier to pass `--force-evmla`
to zksolc

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [ ] Linkcheck has been run via `zk linkcheck`.
…#1145)

## What ❔
- Small change in the `EthWatch` component. Now `Eth Watch` 's `pool`
field is used instead of `ConnectionPool` passed in the Eth Watch's
`run()` method.
- Adds `Eth Watch` implementation, i. e. `EthWatchLayer` and
`EthWatchTask`, in the same way it was done for `StateKeeper `
[here](https://github.com/matter-labs/zksync-era/pull/1043/files#diff-8ff8babf7b83c79dbf96f4998cf71d888beaeb265c7ce33192b0bb0c808f662b).

Current "external" point of view for the framework(the only thing
changed is that we add ` .add_eth_watch_layer()?` after
`.add_state_keeper_layer()?`):

```rust
fn main() -> anyhow::Result<()> {
    #[allow(deprecated)] // TODO (QIT-21): Use centralized configuration approach.
    let log_format = vlog::log_format_from_env();
    let _guard = vlog::ObservabilityBuilder::new()
        .with_log_format(log_format)
        .build();

    MainNodeBuilder::new()
        .add_pools_layer()?
        .add_fee_input_layer()?
        .add_object_store_layer()?
        .add_metadata_calculator_layer()?
        .add_state_keeper_layer()?
        .add_eth_watch_layer()?
        .add_healthcheck_layer()?
        .build()
        .run()?;

    Ok(())
}
```
## What ❔

https://opensource.org/license/mit/ ->
https://opensource.org/blog/license/mit/

## Why ❔

CI is broken, seems like https://opensource.org changed the base URL for
the license pages

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `zk spellcheck`.
- [x] Linkcheck has been run via `zk linkcheck`.
## What ❔

Added a config flag to disable all filter-related methods. If turned on,
they report that the method is not implemented.

## Why ❔

If a node is behind a load balancer then there is no way for client to
reliably hit the same node where he/she created the filter. Hence
supporting this is a very awkward UX. Making this a flag since external
nodes might want to turn this on if they only have a single node mapped
to a static address.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `zk spellcheck`.
- [x] Linkcheck has been run via `zk linkcheck`.
## What ❔

* Added verification key and finalization hint file for 4844 circuit
* Actually start saving finalization hint when generating verification
keys
* Fixed witness vector generator for 4844 circuit - used by GPU prover

## Why ❔

* Necessary steps to start creating 4844 proofs
…labs#1080)

## What ❔

- Propagates I/O errors in the state keeper using `anyhow::Result`,
adding context to calls where appropriate.
- Propagates DB errors in `FactoryDepsDal`.

## Why ❔

Propagating errors makes it easier to attach additional context to them;
thus, it improves DevEx and code maintainability.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `zk spellcheck`.
- [x] Linkcheck has been run via `zk linkcheck`.
…#1205)

## What ❔

* Start using the new shivini function for  4844 circuit

## Why ❔

* Needed for GPU provers
ColoCarletti and others added 22 commits March 12, 2024 17:41
…ion is less than or equal to the expected gas price
…-labs/zksync-era into feat_pubdata_pricing_trait
…cing_trait

feat(Validium): `PubdataPricing` trait
…ation_mode"

This reverts commit dd73f50.
Both `async` functions and functions returning a `Result<_>` are already
`must_use` by default, so the annotation is redundant.
…s-price-calculation

fix(Validium): Fix broken integration test
…st_generator_mode_assertion

feat(Validium): unit tests for `ensure_l1_batch_commit_data_generation_mode`
…-labs/zksync-era into feat_test_casing_in_consistency_checker
…_in_consistency_checker

fix(validium): test casing in consistency checker
Copy link
Collaborator

@mationorato mationorato left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The demo works, so LGTM for demo purposes.
This is not intended to go into main ever.

@mationorato mationorato merged commit bd90d92 into validium_mode_example_crate Mar 20, 2024
4 of 11 checks passed
@mationorato mationorato deleted the update_validium_exampl branch March 20, 2024 17:46
mationorato added a commit that referenced this pull request Mar 20, 2024
mationorato added a commit that referenced this pull request Mar 20, 2024
@mationorato mationorato restored the update_validium_exampl branch March 20, 2024 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.