Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bedrock optimism deposits #6993

Merged
merged 23 commits into from Mar 16, 2023
Merged

Conversation

sl1depengwyn
Copy link
Collaborator

No description provided.

Copy link
Contributor

@varasev varasev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, view my comments, especially about the method of getting logs (events) and getting blocks timestamps by their numbers. I'm worried of using eth_newFilter and eth_getFilterChanges - not sure they will work in production.

@varasev
Copy link
Contributor

varasev commented Mar 16, 2023

Let's add logging to handle_new_logs when logs_to_parse list is not empty. I would do that as follows:

Optimism.log_blocks_chunk_handling(start_block, end_block, start_block, end_block, "#{Enum.count(logs_to_parse)} TransactionDeposited event(s)", "L1")

Here start_block is the min number of block found in blockNumber field of the RPC response, end_block is the max number of block found in blockNumber field of the RPC response.

Could you please clarify why we need one more message for the same action, we already have messages like this:

2023-03-16T11:24:36.867 application=indexer fetcher=optimism_deposits [info] Finish handling L1 block range 8552936..8553436. Found 111 TransactionDeposited event(s). Target range: 8381594..8663789. Progress: 60.89%

here we see how many events are in the range

This is for realtime mode logging. We still don't have logs for realtime

Copy link
Contributor

@varasev varasev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sl1depengwyn
Copy link
Collaborator Author

There are two things left to do before merging:

  1. #6993 (comment)
  2. #6993 (comment)

Done here 1392710

About case when from_block > to_block:
I decided to check it in init function and switch to realtime immediately if it is the case

@varasev varasev merged commit 3b7840a into va-bedrock-optimism Mar 16, 2023
5 checks passed
@varasev varasev deleted the mf-bedrock-optimism-deposits branch March 16, 2023 15:40
@varasev varasev temporarily deployed to Tests March 16, 2023 15:54 — with GitHub Actions Inactive
varasev added a commit that referenced this pull request Mar 19, 2023
* Create `op_deposits` table

* Add OptimismDeposit runner

* WIP Fetcher

* Finish fetcher

* Integrate deposits into APIv2

* Add envs

* Fix requests

* Remove debug

* Update envs names

* Rename `tx` -> `transaction`

* Reuse `decode_data/2`

* Fix review

* Add `uninstall_filter`

* Fix formatting

* Switch to realtime mode more carefully

* Fix review

Allow nil in timestamp
Add progress logging
Improve check_interval calculation

* Fix logging and env

* Fix Association.NotLoaded error

* Replace switching to realtime mode log

* Remove excess start_block

* Fix reorg logging

* Fix `from_block` > `to_block` and add realtime logging

* Fix block boundaries

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>
varasev added a commit that referenced this pull request Mar 21, 2023
* Create `op_deposits` table

* Add OptimismDeposit runner

* WIP Fetcher

* Finish fetcher

* Integrate deposits into APIv2

* Add envs

* Fix requests

* Remove debug

* Update envs names

* Rename `tx` -> `transaction`

* Reuse `decode_data/2`

* Fix review

* Add `uninstall_filter`

* Fix formatting

* Switch to realtime mode more carefully

* Fix review

Allow nil in timestamp
Add progress logging
Improve check_interval calculation

* Fix logging and env

* Fix Association.NotLoaded error

* Replace switching to realtime mode log

* Remove excess start_block

* Fix reorg logging

* Fix `from_block` > `to_block` and add realtime logging

* Fix block boundaries

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>
varasev added a commit that referenced this pull request Mar 23, 2023
* Create `op_deposits` table

* Add OptimismDeposit runner

* WIP Fetcher

* Finish fetcher

* Integrate deposits into APIv2

* Add envs

* Fix requests

* Remove debug

* Update envs names

* Rename `tx` -> `transaction`

* Reuse `decode_data/2`

* Fix review

* Add `uninstall_filter`

* Fix formatting

* Switch to realtime mode more carefully

* Fix review

Allow nil in timestamp
Add progress logging
Improve check_interval calculation

* Fix logging and env

* Fix Association.NotLoaded error

* Replace switching to realtime mode log

* Remove excess start_block

* Fix reorg logging

* Fix `from_block` > `to_block` and add realtime logging

* Fix block boundaries

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>
varasev added a commit that referenced this pull request Mar 31, 2023
* Create `op_deposits` table

* Add OptimismDeposit runner

* WIP Fetcher

* Finish fetcher

* Integrate deposits into APIv2

* Add envs

* Fix requests

* Remove debug

* Update envs names

* Rename `tx` -> `transaction`

* Reuse `decode_data/2`

* Fix review

* Add `uninstall_filter`

* Fix formatting

* Switch to realtime mode more carefully

* Fix review

Allow nil in timestamp
Add progress logging
Improve check_interval calculation

* Fix logging and env

* Fix Association.NotLoaded error

* Replace switching to realtime mode log

* Remove excess start_block

* Fix reorg logging

* Fix `from_block` > `to_block` and add realtime logging

* Fix block boundaries

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>
varasev added a commit that referenced this pull request Apr 17, 2023
* Create `op_deposits` table

* Add OptimismDeposit runner

* WIP Fetcher

* Finish fetcher

* Integrate deposits into APIv2

* Add envs

* Fix requests

* Remove debug

* Update envs names

* Rename `tx` -> `transaction`

* Reuse `decode_data/2`

* Fix review

* Add `uninstall_filter`

* Fix formatting

* Switch to realtime mode more carefully

* Fix review

Allow nil in timestamp
Add progress logging
Improve check_interval calculation

* Fix logging and env

* Fix Association.NotLoaded error

* Replace switching to realtime mode log

* Remove excess start_block

* Fix reorg logging

* Fix `from_block` > `to_block` and add realtime logging

* Fix block boundaries

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>
varasev added a commit that referenced this pull request Apr 27, 2023
* Create `op_deposits` table

* Add OptimismDeposit runner

* WIP Fetcher

* Finish fetcher

* Integrate deposits into APIv2

* Add envs

* Fix requests

* Remove debug

* Update envs names

* Rename `tx` -> `transaction`

* Reuse `decode_data/2`

* Fix review

* Add `uninstall_filter`

* Fix formatting

* Switch to realtime mode more carefully

* Fix review

Allow nil in timestamp
Add progress logging
Improve check_interval calculation

* Fix logging and env

* Fix Association.NotLoaded error

* Replace switching to realtime mode log

* Remove excess start_block

* Fix reorg logging

* Fix `from_block` > `to_block` and add realtime logging

* Fix block boundaries

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>
varasev added a commit that referenced this pull request May 11, 2023
* Create `op_deposits` table

* Add OptimismDeposit runner

* WIP Fetcher

* Finish fetcher

* Integrate deposits into APIv2

* Add envs

* Fix requests

* Remove debug

* Update envs names

* Rename `tx` -> `transaction`

* Reuse `decode_data/2`

* Fix review

* Add `uninstall_filter`

* Fix formatting

* Switch to realtime mode more carefully

* Fix review

Allow nil in timestamp
Add progress logging
Improve check_interval calculation

* Fix logging and env

* Fix Association.NotLoaded error

* Replace switching to realtime mode log

* Remove excess start_block

* Fix reorg logging

* Fix `from_block` > `to_block` and add realtime logging

* Fix block boundaries

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>
varasev added a commit that referenced this pull request Jun 7, 2023
* Create `op_deposits` table

* Add OptimismDeposit runner

* WIP Fetcher

* Finish fetcher

* Integrate deposits into APIv2

* Add envs

* Fix requests

* Remove debug

* Update envs names

* Rename `tx` -> `transaction`

* Reuse `decode_data/2`

* Fix review

* Add `uninstall_filter`

* Fix formatting

* Switch to realtime mode more carefully

* Fix review

Allow nil in timestamp
Add progress logging
Improve check_interval calculation

* Fix logging and env

* Fix Association.NotLoaded error

* Replace switching to realtime mode log

* Remove excess start_block

* Fix reorg logging

* Fix `from_block` > `to_block` and add realtime logging

* Fix block boundaries

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>
vbaranov pushed a commit that referenced this pull request Jun 8, 2023
…thdrawals) (#6980)

* Add op_output_roots table

* Add OptimismOutputRoots runner

* Add initial code for output roots fetcher

* Add checks to init function

* Partially add logs and L1 reorgs handling

* Add reorgs handling

* Add RPC retries

* Write output roots to database

* Log output roots handling

* Update indexer README

* Add API v2 for Optimism Output Roots

* Add op_withdrawals table

* Add OptimismWithdrawals runner

* Prepare realtime optimism withdrawals fetching

* Add realtime optimism withdrawals fetching

* Define checks in init function

* log.first_topic can be nil

* Show total count of output roots in API v2

* Add msg_nonce gaps filler

* Refactoring

* Intermediate refactoring

* Add historical withdrawals handling and refactor

* Finish op_withdrawals table filling

* Small refactoring

* Add op_withdrawal_events table

* Add OptimismWithdrawalEvents runner

* Add OptimismWithdrawalEvent fetcher

* Update indexer README

* Add API v2 for Optimism Withdrawals

* Add env variables to common-blockscout.env and Makefile

* Set `from` as address object instead of just address hash for withdrawal

* mix format

* Add op_transaction_batches table

* Add OptimismTxnBatches runner

* Add a draft for OptimismTxnBatch fetcher

* Add a draft for OptimismTxnBatch

* Extend a draft for OptimismTxnBatch

* Extend OptimismTxnBatch

* Finish OptimismTxnBatch (without reorgs handling yet)

* Optimize OptimismTxnBatch fetcher

* Remove duplicated txn batches

* Add zlib_inflate_handler for empty case

* Add reorgs handling for txn batches

* Fix reorgs handling for txn batches

* Small refactor

* Finish Indexer.Fetcher.OptimismTxnBatch (without refactoring yet)

* Apply new ex_rlp version

* Add API v2 for Optimism Txn Batches

* Add env variables to common-blockscout.env and Makefile

* Refactor OptimismTxnBatch fetcher for mix credo

* Replace binary_slice with binary_part function to run with Elixir 1.13

* Update changelog

* Update indexer readme

* Rename op_withdrawals.l2_tx_hash field to l2_transaction_hash

* Rename l1_tx_hash fields to l1_transaction_hash

* Rename *tx* fields to *transaction* fields

* Rename env variables

* Rename env variables

* Add an indexer helper

* Add an indexer helper

* Small refactoring

* Fix tx_count for txn batches view

* Use EthereumJSONRPC.Block.ByHash instead of the raw call

* Infinity timeout for blocks query

* Small refactoring

* Refactor init function for two modules

* Small refactoring

* Rename l1_transaction_timestamp field to l1_timestamp

* Rename withdrawal_hash field to hash

* Refactor for decode_data function

* Refactor for mix credo

* Add INDEXER_OPTIMISM_L1_BATCH_BLOCKS_CHUNK_SIZE env and small refactoring

* Add INDEXER_OPTIMISM_L1_BATCH_BLOCKS_CHUNK_SIZE env to other files

* Add an index for l1_block_number field

* Add an index for l1_block_number field

* Remove redundant :ok

* Use faster way to count rows in a table

* Refactor reorgs monitor functions

* Clarify frame structure

* Reduce storage consumption for optimism transaction batches

* Reuse CacheHelper.estimated_count_from function

* Bedrock optimism deposits (#6993)

* Create `op_deposits` table

* Add OptimismDeposit runner

* WIP Fetcher

* Finish fetcher

* Integrate deposits into APIv2

* Add envs

* Fix requests

* Remove debug

* Update envs names

* Rename `tx` -> `transaction`

* Reuse `decode_data/2`

* Fix review

* Add `uninstall_filter`

* Fix formatting

* Switch to realtime mode more carefully

* Fix review

Allow nil in timestamp
Add progress logging
Improve check_interval calculation

* Fix logging and env

* Fix Association.NotLoaded error

* Replace switching to realtime mode log

* Remove excess start_block

* Fix reorg logging

* Fix `from_block` > `to_block` and add realtime logging

* Fix block boundaries

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* mix format

* Return total count of L2 entity by a separate API request

* Filter by consensus blocks

* Parallelize tx count operation and small refactoring

* Use read replica for L2 entities in API

* Parse block_number and tx_hash for Optimism Deposits module

* Return page_size back to 50

* Small fixes and refactoring

* Update apps/block_scout_web/lib/block_scout_web/api_router.ex

Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>

* Small optimization

* Use ecto association instead of explicit join for txn batches

* Refactoring

* Use Stream inspead of Enum

* Small refactoring

* Add assoc for transaction batches in OptimismFrameSequence

* Use common reorg monitor for Optimism modules

* Rename Explorer.Helpers to Explorer.Helper

* Don't start an optimism module unless the main optimism module is not started

* Don't start reorg monitor for optimism modules when it is not needed

* Small refactoring

* Remove debug broadcasting

* Add Optimism BedRock Deposits to the main page in API (#7200)

* Add Optimism BedRock Deposits to the main page in API

* Update changelog

* Pass the number of deposits instead of only one item per once

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Refactor for credo

* Output L1 fields in API v2 for transaction page

* Update changelog

* Use helper

* Refactor Indexer.Fetcher.Optimism

* Fix l1_timestamp issue in OptimismTxnBatch fetcher

* Reset Logger metadata before Indexer.Transform.OptimismWithdrawals.parse function finishes

* Fix IDs ordering in remove_duplicates function of Indexer.Fetcher.OptimismTxnBatch

* Consider rewriting of the first frame in Indexer.Fetcher.OptimismTxnBatch

* Fix Indexer.Fetcher.OptimismTxnBatch (consider chunking)

* Fix Indexer.Fetcher.OptimismTxnBatch

* Fix handling invalid frame sequences in Indexer.Fetcher.OptimismTxnBatch

* Read Optimism finalization period from a smart contract

* Fixes for dialyzer

* Fix for EthereumJSONRPC tests

* Fixes for Explorer tests

* Fixes for Explorer tests

* Fix of block/realtime/fetcher_test.exs

* mix format and small fixes for block_scout_web tests

* Reset GA cache

* Fix handling nil in PendingBlockOperation.estimated_count()

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>
Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>
vbaranov pushed a commit that referenced this pull request Feb 23, 2024
…thdrawals) (#6980)

* Add op_output_roots table

* Add OptimismOutputRoots runner

* Add initial code for output roots fetcher

* Add checks to init function

* Partially add logs and L1 reorgs handling

* Add reorgs handling

* Add RPC retries

* Write output roots to database

* Log output roots handling

* Update indexer README

* Add API v2 for Optimism Output Roots

* Add op_withdrawals table

* Add OptimismWithdrawals runner

* Prepare realtime optimism withdrawals fetching

* Add realtime optimism withdrawals fetching

* Define checks in init function

* log.first_topic can be nil

* Show total count of output roots in API v2

* Add msg_nonce gaps filler

* Refactoring

* Intermediate refactoring

* Add historical withdrawals handling and refactor

* Finish op_withdrawals table filling

* Small refactoring

* Add op_withdrawal_events table

* Add OptimismWithdrawalEvents runner

* Add OptimismWithdrawalEvent fetcher

* Update indexer README

* Add API v2 for Optimism Withdrawals

* Add env variables to common-blockscout.env and Makefile

* Set `from` as address object instead of just address hash for withdrawal

* mix format

* Add op_transaction_batches table

* Add OptimismTxnBatches runner

* Add a draft for OptimismTxnBatch fetcher

* Add a draft for OptimismTxnBatch

* Extend a draft for OptimismTxnBatch

* Extend OptimismTxnBatch

* Finish OptimismTxnBatch (without reorgs handling yet)

* Optimize OptimismTxnBatch fetcher

* Remove duplicated txn batches

* Add zlib_inflate_handler for empty case

* Add reorgs handling for txn batches

* Fix reorgs handling for txn batches

* Small refactor

* Finish Indexer.Fetcher.OptimismTxnBatch (without refactoring yet)

* Apply new ex_rlp version

* Add API v2 for Optimism Txn Batches

* Add env variables to common-blockscout.env and Makefile

* Refactor OptimismTxnBatch fetcher for mix credo

* Replace binary_slice with binary_part function to run with Elixir 1.13

* Update changelog

* Update indexer readme

* Rename op_withdrawals.l2_tx_hash field to l2_transaction_hash

* Rename l1_tx_hash fields to l1_transaction_hash

* Rename *tx* fields to *transaction* fields

* Rename env variables

* Rename env variables

* Add an indexer helper

* Add an indexer helper

* Small refactoring

* Fix tx_count for txn batches view

* Use EthereumJSONRPC.Block.ByHash instead of the raw call

* Infinity timeout for blocks query

* Small refactoring

* Refactor init function for two modules

* Small refactoring

* Rename l1_transaction_timestamp field to l1_timestamp

* Rename withdrawal_hash field to hash

* Refactor for decode_data function

* Refactor for mix credo

* Add INDEXER_OPTIMISM_L1_BATCH_BLOCKS_CHUNK_SIZE env and small refactoring

* Add INDEXER_OPTIMISM_L1_BATCH_BLOCKS_CHUNK_SIZE env to other files

* Add an index for l1_block_number field

* Add an index for l1_block_number field

* Remove redundant :ok

* Use faster way to count rows in a table

* Refactor reorgs monitor functions

* Clarify frame structure

* Reduce storage consumption for optimism transaction batches

* Reuse CacheHelper.estimated_count_from function

* Bedrock optimism deposits (#6993)

* Create `op_deposits` table

* Add OptimismDeposit runner

* WIP Fetcher

* Finish fetcher

* Integrate deposits into APIv2

* Add envs

* Fix requests

* Remove debug

* Update envs names

* Rename `tx` -> `transaction`

* Reuse `decode_data/2`

* Fix review

* Add `uninstall_filter`

* Fix formatting

* Switch to realtime mode more carefully

* Fix review

Allow nil in timestamp
Add progress logging
Improve check_interval calculation

* Fix logging and env

* Fix Association.NotLoaded error

* Replace switching to realtime mode log

* Remove excess start_block

* Fix reorg logging

* Fix `from_block` > `to_block` and add realtime logging

* Fix block boundaries

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* mix format

* Return total count of L2 entity by a separate API request

* Filter by consensus blocks

* Parallelize tx count operation and small refactoring

* Use read replica for L2 entities in API

* Parse block_number and tx_hash for Optimism Deposits module

* Return page_size back to 50

* Small fixes and refactoring

* Update apps/block_scout_web/lib/block_scout_web/api_router.ex

Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>

* Small optimization

* Use ecto association instead of explicit join for txn batches

* Refactoring

* Use Stream inspead of Enum

* Small refactoring

* Add assoc for transaction batches in OptimismFrameSequence

* Use common reorg monitor for Optimism modules

* Rename Explorer.Helpers to Explorer.Helper

* Don't start an optimism module unless the main optimism module is not started

* Don't start reorg monitor for optimism modules when it is not needed

* Small refactoring

* Remove debug broadcasting

* Add Optimism BedRock Deposits to the main page in API (#7200)

* Add Optimism BedRock Deposits to the main page in API

* Update changelog

* Pass the number of deposits instead of only one item per once

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Refactor for credo

* Output L1 fields in API v2 for transaction page

* Update changelog

* Use helper

* Refactor Indexer.Fetcher.Optimism

* Fix l1_timestamp issue in OptimismTxnBatch fetcher

* Reset Logger metadata before Indexer.Transform.OptimismWithdrawals.parse function finishes

* Fix IDs ordering in remove_duplicates function of Indexer.Fetcher.OptimismTxnBatch

* Consider rewriting of the first frame in Indexer.Fetcher.OptimismTxnBatch

* Fix Indexer.Fetcher.OptimismTxnBatch (consider chunking)

* Fix Indexer.Fetcher.OptimismTxnBatch

* Fix handling invalid frame sequences in Indexer.Fetcher.OptimismTxnBatch

* Read Optimism finalization period from a smart contract

* Fixes for dialyzer

* Fix for EthereumJSONRPC tests

* Fixes for Explorer tests

* Fixes for Explorer tests

* Fix of block/realtime/fetcher_test.exs

* mix format and small fixes for block_scout_web tests

* Reset GA cache

* Fix handling nil in PendingBlockOperation.estimated_count()

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>
Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>
vbaranov pushed a commit that referenced this pull request Feb 29, 2024
…thdrawals) (#6980)

* Add op_output_roots table

* Add OptimismOutputRoots runner

* Add initial code for output roots fetcher

* Add checks to init function

* Partially add logs and L1 reorgs handling

* Add reorgs handling

* Add RPC retries

* Write output roots to database

* Log output roots handling

* Update indexer README

* Add API v2 for Optimism Output Roots

* Add op_withdrawals table

* Add OptimismWithdrawals runner

* Prepare realtime optimism withdrawals fetching

* Add realtime optimism withdrawals fetching

* Define checks in init function

* log.first_topic can be nil

* Show total count of output roots in API v2

* Add msg_nonce gaps filler

* Refactoring

* Intermediate refactoring

* Add historical withdrawals handling and refactor

* Finish op_withdrawals table filling

* Small refactoring

* Add op_withdrawal_events table

* Add OptimismWithdrawalEvents runner

* Add OptimismWithdrawalEvent fetcher

* Update indexer README

* Add API v2 for Optimism Withdrawals

* Add env variables to common-blockscout.env and Makefile

* Set `from` as address object instead of just address hash for withdrawal

* mix format

* Add op_transaction_batches table

* Add OptimismTxnBatches runner

* Add a draft for OptimismTxnBatch fetcher

* Add a draft for OptimismTxnBatch

* Extend a draft for OptimismTxnBatch

* Extend OptimismTxnBatch

* Finish OptimismTxnBatch (without reorgs handling yet)

* Optimize OptimismTxnBatch fetcher

* Remove duplicated txn batches

* Add zlib_inflate_handler for empty case

* Add reorgs handling for txn batches

* Fix reorgs handling for txn batches

* Small refactor

* Finish Indexer.Fetcher.OptimismTxnBatch (without refactoring yet)

* Apply new ex_rlp version

* Add API v2 for Optimism Txn Batches

* Add env variables to common-blockscout.env and Makefile

* Refactor OptimismTxnBatch fetcher for mix credo

* Replace binary_slice with binary_part function to run with Elixir 1.13

* Update changelog

* Update indexer readme

* Rename op_withdrawals.l2_tx_hash field to l2_transaction_hash

* Rename l1_tx_hash fields to l1_transaction_hash

* Rename *tx* fields to *transaction* fields

* Rename env variables

* Rename env variables

* Add an indexer helper

* Add an indexer helper

* Small refactoring

* Fix tx_count for txn batches view

* Use EthereumJSONRPC.Block.ByHash instead of the raw call

* Infinity timeout for blocks query

* Small refactoring

* Refactor init function for two modules

* Small refactoring

* Rename l1_transaction_timestamp field to l1_timestamp

* Rename withdrawal_hash field to hash

* Refactor for decode_data function

* Refactor for mix credo

* Add INDEXER_OPTIMISM_L1_BATCH_BLOCKS_CHUNK_SIZE env and small refactoring

* Add INDEXER_OPTIMISM_L1_BATCH_BLOCKS_CHUNK_SIZE env to other files

* Add an index for l1_block_number field

* Add an index for l1_block_number field

* Remove redundant :ok

* Use faster way to count rows in a table

* Refactor reorgs monitor functions

* Clarify frame structure

* Reduce storage consumption for optimism transaction batches

* Reuse CacheHelper.estimated_count_from function

* Bedrock optimism deposits (#6993)

* Create `op_deposits` table

* Add OptimismDeposit runner

* WIP Fetcher

* Finish fetcher

* Integrate deposits into APIv2

* Add envs

* Fix requests

* Remove debug

* Update envs names

* Rename `tx` -> `transaction`

* Reuse `decode_data/2`

* Fix review

* Add `uninstall_filter`

* Fix formatting

* Switch to realtime mode more carefully

* Fix review

Allow nil in timestamp
Add progress logging
Improve check_interval calculation

* Fix logging and env

* Fix Association.NotLoaded error

* Replace switching to realtime mode log

* Remove excess start_block

* Fix reorg logging

* Fix `from_block` > `to_block` and add realtime logging

* Fix block boundaries

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* mix format

* Return total count of L2 entity by a separate API request

* Filter by consensus blocks

* Parallelize tx count operation and small refactoring

* Use read replica for L2 entities in API

* Parse block_number and tx_hash for Optimism Deposits module

* Return page_size back to 50

* Small fixes and refactoring

* Update apps/block_scout_web/lib/block_scout_web/api_router.ex

Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>

* Small optimization

* Use ecto association instead of explicit join for txn batches

* Refactoring

* Use Stream inspead of Enum

* Small refactoring

* Add assoc for transaction batches in OptimismFrameSequence

* Use common reorg monitor for Optimism modules

* Rename Explorer.Helpers to Explorer.Helper

* Don't start an optimism module unless the main optimism module is not started

* Don't start reorg monitor for optimism modules when it is not needed

* Small refactoring

* Remove debug broadcasting

* Add Optimism BedRock Deposits to the main page in API (#7200)

* Add Optimism BedRock Deposits to the main page in API

* Update changelog

* Pass the number of deposits instead of only one item per once

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Refactor for credo

* Output L1 fields in API v2 for transaction page

* Update changelog

* Use helper

* Refactor Indexer.Fetcher.Optimism

* Fix l1_timestamp issue in OptimismTxnBatch fetcher

* Reset Logger metadata before Indexer.Transform.OptimismWithdrawals.parse function finishes

* Fix IDs ordering in remove_duplicates function of Indexer.Fetcher.OptimismTxnBatch

* Consider rewriting of the first frame in Indexer.Fetcher.OptimismTxnBatch

* Fix Indexer.Fetcher.OptimismTxnBatch (consider chunking)

* Fix Indexer.Fetcher.OptimismTxnBatch

* Fix handling invalid frame sequences in Indexer.Fetcher.OptimismTxnBatch

* Read Optimism finalization period from a smart contract

* Fixes for dialyzer

* Fix for EthereumJSONRPC tests

* Fixes for Explorer tests

* Fixes for Explorer tests

* Fix of block/realtime/fetcher_test.exs

* mix format and small fixes for block_scout_web tests

* Reset GA cache

* Fix handling nil in PendingBlockOperation.estimated_count()

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>
Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>
vbaranov pushed a commit that referenced this pull request Mar 4, 2024
…thdrawals) (#6980)

* Add op_output_roots table

* Add OptimismOutputRoots runner

* Add initial code for output roots fetcher

* Add checks to init function

* Partially add logs and L1 reorgs handling

* Add reorgs handling

* Add RPC retries

* Write output roots to database

* Log output roots handling

* Update indexer README

* Add API v2 for Optimism Output Roots

* Add op_withdrawals table

* Add OptimismWithdrawals runner

* Prepare realtime optimism withdrawals fetching

* Add realtime optimism withdrawals fetching

* Define checks in init function

* log.first_topic can be nil

* Show total count of output roots in API v2

* Add msg_nonce gaps filler

* Refactoring

* Intermediate refactoring

* Add historical withdrawals handling and refactor

* Finish op_withdrawals table filling

* Small refactoring

* Add op_withdrawal_events table

* Add OptimismWithdrawalEvents runner

* Add OptimismWithdrawalEvent fetcher

* Update indexer README

* Add API v2 for Optimism Withdrawals

* Add env variables to common-blockscout.env and Makefile

* Set `from` as address object instead of just address hash for withdrawal

* mix format

* Add op_transaction_batches table

* Add OptimismTxnBatches runner

* Add a draft for OptimismTxnBatch fetcher

* Add a draft for OptimismTxnBatch

* Extend a draft for OptimismTxnBatch

* Extend OptimismTxnBatch

* Finish OptimismTxnBatch (without reorgs handling yet)

* Optimize OptimismTxnBatch fetcher

* Remove duplicated txn batches

* Add zlib_inflate_handler for empty case

* Add reorgs handling for txn batches

* Fix reorgs handling for txn batches

* Small refactor

* Finish Indexer.Fetcher.OptimismTxnBatch (without refactoring yet)

* Apply new ex_rlp version

* Add API v2 for Optimism Txn Batches

* Add env variables to common-blockscout.env and Makefile

* Refactor OptimismTxnBatch fetcher for mix credo

* Replace binary_slice with binary_part function to run with Elixir 1.13

* Update changelog

* Update indexer readme

* Rename op_withdrawals.l2_tx_hash field to l2_transaction_hash

* Rename l1_tx_hash fields to l1_transaction_hash

* Rename *tx* fields to *transaction* fields

* Rename env variables

* Rename env variables

* Add an indexer helper

* Add an indexer helper

* Small refactoring

* Fix tx_count for txn batches view

* Use EthereumJSONRPC.Block.ByHash instead of the raw call

* Infinity timeout for blocks query

* Small refactoring

* Refactor init function for two modules

* Small refactoring

* Rename l1_transaction_timestamp field to l1_timestamp

* Rename withdrawal_hash field to hash

* Refactor for decode_data function

* Refactor for mix credo

* Add INDEXER_OPTIMISM_L1_BATCH_BLOCKS_CHUNK_SIZE env and small refactoring

* Add INDEXER_OPTIMISM_L1_BATCH_BLOCKS_CHUNK_SIZE env to other files

* Add an index for l1_block_number field

* Add an index for l1_block_number field

* Remove redundant :ok

* Use faster way to count rows in a table

* Refactor reorgs monitor functions

* Clarify frame structure

* Reduce storage consumption for optimism transaction batches

* Reuse CacheHelper.estimated_count_from function

* Bedrock optimism deposits (#6993)

* Create `op_deposits` table

* Add OptimismDeposit runner

* WIP Fetcher

* Finish fetcher

* Integrate deposits into APIv2

* Add envs

* Fix requests

* Remove debug

* Update envs names

* Rename `tx` -> `transaction`

* Reuse `decode_data/2`

* Fix review

* Add `uninstall_filter`

* Fix formatting

* Switch to realtime mode more carefully

* Fix review

Allow nil in timestamp
Add progress logging
Improve check_interval calculation

* Fix logging and env

* Fix Association.NotLoaded error

* Replace switching to realtime mode log

* Remove excess start_block

* Fix reorg logging

* Fix `from_block` > `to_block` and add realtime logging

* Fix block boundaries

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* mix format

* Return total count of L2 entity by a separate API request

* Filter by consensus blocks

* Parallelize tx count operation and small refactoring

* Use read replica for L2 entities in API

* Parse block_number and tx_hash for Optimism Deposits module

* Return page_size back to 50

* Small fixes and refactoring

* Update apps/block_scout_web/lib/block_scout_web/api_router.ex

Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>

* Small optimization

* Use ecto association instead of explicit join for txn batches

* Refactoring

* Use Stream inspead of Enum

* Small refactoring

* Add assoc for transaction batches in OptimismFrameSequence

* Use common reorg monitor for Optimism modules

* Rename Explorer.Helpers to Explorer.Helper

* Don't start an optimism module unless the main optimism module is not started

* Don't start reorg monitor for optimism modules when it is not needed

* Small refactoring

* Remove debug broadcasting

* Add Optimism BedRock Deposits to the main page in API (#7200)

* Add Optimism BedRock Deposits to the main page in API

* Update changelog

* Pass the number of deposits instead of only one item per once

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Refactor for credo

* Output L1 fields in API v2 for transaction page

* Update changelog

* Use helper

* Refactor Indexer.Fetcher.Optimism

* Fix l1_timestamp issue in OptimismTxnBatch fetcher

* Reset Logger metadata before Indexer.Transform.OptimismWithdrawals.parse function finishes

* Fix IDs ordering in remove_duplicates function of Indexer.Fetcher.OptimismTxnBatch

* Consider rewriting of the first frame in Indexer.Fetcher.OptimismTxnBatch

* Fix Indexer.Fetcher.OptimismTxnBatch (consider chunking)

* Fix Indexer.Fetcher.OptimismTxnBatch

* Fix handling invalid frame sequences in Indexer.Fetcher.OptimismTxnBatch

* Read Optimism finalization period from a smart contract

* Fixes for dialyzer

* Fix for EthereumJSONRPC tests

* Fixes for Explorer tests

* Fixes for Explorer tests

* Fix of block/realtime/fetcher_test.exs

* mix format and small fixes for block_scout_web tests

* Reset GA cache

* Fix handling nil in PendingBlockOperation.estimated_count()

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>
Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>
vbaranov added a commit that referenced this pull request Mar 4, 2024
* Transaction page L1 fields

* Path fix

* Reduce the number of files from 19 to 5 in logs rotate config

* Customize optimism-goerli deployment

* Optimism branding

* Remove testnet logo text. OG uses customized label

* Fix Circles theme

* L1 tx fields fix for Optimism BedRock update

* Remove redundant line

* Add gas_price handling for elixir_to_params and change function ordering

* Remove l1TxOrigin handling for another version of RPC

* Add GA

* Fix realtime fetcher test

* Update Changelog

* Fix internal transactions processing for non-consensus blocks

* Lose consensus only for consensus=true blocks

* Fix handling transaction RPC responses without some fields

* Fix tests except for indexer module

* Add Optimism BedRock support (Txn Batches, Output Roots, Deposits, Withdrawals) (#6980)

* Add op_output_roots table

* Add OptimismOutputRoots runner

* Add initial code for output roots fetcher

* Add checks to init function

* Partially add logs and L1 reorgs handling

* Add reorgs handling

* Add RPC retries

* Write output roots to database

* Log output roots handling

* Update indexer README

* Add API v2 for Optimism Output Roots

* Add op_withdrawals table

* Add OptimismWithdrawals runner

* Prepare realtime optimism withdrawals fetching

* Add realtime optimism withdrawals fetching

* Define checks in init function

* log.first_topic can be nil

* Show total count of output roots in API v2

* Add msg_nonce gaps filler

* Refactoring

* Intermediate refactoring

* Add historical withdrawals handling and refactor

* Finish op_withdrawals table filling

* Small refactoring

* Add op_withdrawal_events table

* Add OptimismWithdrawalEvents runner

* Add OptimismWithdrawalEvent fetcher

* Update indexer README

* Add API v2 for Optimism Withdrawals

* Add env variables to common-blockscout.env and Makefile

* Set `from` as address object instead of just address hash for withdrawal

* mix format

* Add op_transaction_batches table

* Add OptimismTxnBatches runner

* Add a draft for OptimismTxnBatch fetcher

* Add a draft for OptimismTxnBatch

* Extend a draft for OptimismTxnBatch

* Extend OptimismTxnBatch

* Finish OptimismTxnBatch (without reorgs handling yet)

* Optimize OptimismTxnBatch fetcher

* Remove duplicated txn batches

* Add zlib_inflate_handler for empty case

* Add reorgs handling for txn batches

* Fix reorgs handling for txn batches

* Small refactor

* Finish Indexer.Fetcher.OptimismTxnBatch (without refactoring yet)

* Apply new ex_rlp version

* Add API v2 for Optimism Txn Batches

* Add env variables to common-blockscout.env and Makefile

* Refactor OptimismTxnBatch fetcher for mix credo

* Replace binary_slice with binary_part function to run with Elixir 1.13

* Update changelog

* Update indexer readme

* Rename op_withdrawals.l2_tx_hash field to l2_transaction_hash

* Rename l1_tx_hash fields to l1_transaction_hash

* Rename *tx* fields to *transaction* fields

* Rename env variables

* Rename env variables

* Add an indexer helper

* Add an indexer helper

* Small refactoring

* Fix tx_count for txn batches view

* Use EthereumJSONRPC.Block.ByHash instead of the raw call

* Infinity timeout for blocks query

* Small refactoring

* Refactor init function for two modules

* Small refactoring

* Rename l1_transaction_timestamp field to l1_timestamp

* Rename withdrawal_hash field to hash

* Refactor for decode_data function

* Refactor for mix credo

* Add INDEXER_OPTIMISM_L1_BATCH_BLOCKS_CHUNK_SIZE env and small refactoring

* Add INDEXER_OPTIMISM_L1_BATCH_BLOCKS_CHUNK_SIZE env to other files

* Add an index for l1_block_number field

* Add an index for l1_block_number field

* Remove redundant :ok

* Use faster way to count rows in a table

* Refactor reorgs monitor functions

* Clarify frame structure

* Reduce storage consumption for optimism transaction batches

* Reuse CacheHelper.estimated_count_from function

* Bedrock optimism deposits (#6993)

* Create `op_deposits` table

* Add OptimismDeposit runner

* WIP Fetcher

* Finish fetcher

* Integrate deposits into APIv2

* Add envs

* Fix requests

* Remove debug

* Update envs names

* Rename `tx` -> `transaction`

* Reuse `decode_data/2`

* Fix review

* Add `uninstall_filter`

* Fix formatting

* Switch to realtime mode more carefully

* Fix review

Allow nil in timestamp
Add progress logging
Improve check_interval calculation

* Fix logging and env

* Fix Association.NotLoaded error

* Replace switching to realtime mode log

* Remove excess start_block

* Fix reorg logging

* Fix `from_block` > `to_block` and add realtime logging

* Fix block boundaries

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* mix format

* Return total count of L2 entity by a separate API request

* Filter by consensus blocks

* Parallelize tx count operation and small refactoring

* Use read replica for L2 entities in API

* Parse block_number and tx_hash for Optimism Deposits module

* Return page_size back to 50

* Small fixes and refactoring

* Update apps/block_scout_web/lib/block_scout_web/api_router.ex

Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>

* Small optimization

* Use ecto association instead of explicit join for txn batches

* Refactoring

* Use Stream inspead of Enum

* Small refactoring

* Add assoc for transaction batches in OptimismFrameSequence

* Use common reorg monitor for Optimism modules

* Rename Explorer.Helpers to Explorer.Helper

* Don't start an optimism module unless the main optimism module is not started

* Don't start reorg monitor for optimism modules when it is not needed

* Small refactoring

* Remove debug broadcasting

* Add Optimism BedRock Deposits to the main page in API (#7200)

* Add Optimism BedRock Deposits to the main page in API

* Update changelog

* Pass the number of deposits instead of only one item per once

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Refactor for credo

* Output L1 fields in API v2 for transaction page

* Update changelog

* Use helper

* Refactor Indexer.Fetcher.Optimism

* Fix l1_timestamp issue in OptimismTxnBatch fetcher

* Reset Logger metadata before Indexer.Transform.OptimismWithdrawals.parse function finishes

* Fix IDs ordering in remove_duplicates function of Indexer.Fetcher.OptimismTxnBatch

* Consider rewriting of the first frame in Indexer.Fetcher.OptimismTxnBatch

* Fix Indexer.Fetcher.OptimismTxnBatch (consider chunking)

* Fix Indexer.Fetcher.OptimismTxnBatch

* Fix handling invalid frame sequences in Indexer.Fetcher.OptimismTxnBatch

* Read Optimism finalization period from a smart contract

* Fixes for dialyzer

* Fix for EthereumJSONRPC tests

* Fixes for Explorer tests

* Fixes for Explorer tests

* Fix of block/realtime/fetcher_test.exs

* mix format and small fixes for block_scout_web tests

* Reset GA cache

* Fix handling nil in PendingBlockOperation.estimated_count()

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>
Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>

* Fix autocomplete

* Fix merging conflicts

* Add exit handler to Indexer.Fetcher.OptimismWithdrawal

* Fix transactions ordering in Indexer.Fetcher.OptimismTxnBatch

* Update changelog

* Refactor to fix credo

* Mix credo fix

* Fix transaction batches module for L2 OP stack (#7827)

* Fix mixed transactions handling in Indexer.Fetcher.OptimismTxnBatch

* Ignore duplicated frame

* Update changelog

* Add sorting to the future frames list

* Change list order

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Remove unused aliases

* Ignore previously handled frame by OP transaction batches module (#8122)

* Ignore duplicated frame

* Update changelog

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Return alias for Explorer.Chain.Cache.Helper in chain.ex

* Ignore invalid frame by OP transaction batches module (#8208)

* Update changelog

* Ignore invalid frame

* Update changelog

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Fix Indexer.Fetcher.OptimismTxnBatch

* Fix API v2 for OP Withdrawals

* Refactor optimism fetchers init

* Add log for switching from fallback url

* Fix for Indexer.Fetcher.OptimismTxnBatch

* Add OP withdrawal status to transaction page in API (#8702)

* Add OP withdrawal status to transaction page in API

* Update changelog

* Small refactoring

* Update .dialyzer-ignore

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Add start pause to `Indexer.Fetcher.OptimismTxnBatch`

* Small refactor of `Indexer.Fetcher.OptimismTxnBatch`

* Consider consensus block only when retrieving OP withdrawal transaction status (#8811)

* Consider consensus block only when retrieving OP withdrawal transaction status

* Update changelog

* Clear GA cache

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Hotfix for optimism_withdrawal_transaction_status function

* Return all OP Withdrawals bound to L2 transaction

* Try to import config

* Remove unused functions from Explorer.Chain

* Refactor for mix credo

* Fix order of proxy standards: 1167, 1967

* Fixes in Optimism due to changed log topics type

* Fix for EthereumJSONRPC tests

* Clear GA cache and update cspell.json

* Fix indexer tests

* Return current exchange rate in api/v2/stats

* Fix log decoding bug

* Temp disable build of image for arm64

* Rewrite Indexer.Fetcher.OptimismTxnBatch module

* Add handling of span batches

* Add support of latest block for Optimism modules

* Update changelog and spelling

* Rewrite Indexer.Fetcher.OptimismTxnBatch module

* Add handling of span batches

* Add support of latest block for Optimism modules

* Refactoring

* Partially add specs and docs for public functions

* Refactoring

* add an entry to CHANEGELOG.md

* apply review (use origin entity instead of joined entity in with tx status)

* Fixes after rebase

* Remove old UI sustomizations

* Optimism chain type

* Change structure of folders

* Fixes after review

* Fix CHANGELOG

* Fixes after 2nd review

* Process 3d review: add tests for fee/2 function

* Process 4th review

* Review fix: move Op related functions from chain.ex

* Review fix: make OptimismFinalizationPeriod configurable

* Process review comment

* System.get_env("CHAIN_TYPE") => Application.get_env(:explorer, :chain_type)

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>
Co-authored-by: Qwerty5Uiop <alex000010@bk.ru>
Co-authored-by: varasev <33550681+varasev@users.noreply.github.com>
Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>
Co-authored-by: rlgns98kr <rlgns98kr@gmail.com>
sl1depengwyn added a commit that referenced this pull request Mar 4, 2024
…thdrawals) (#6980)

* Add op_output_roots table

* Add OptimismOutputRoots runner

* Add initial code for output roots fetcher

* Add checks to init function

* Partially add logs and L1 reorgs handling

* Add reorgs handling

* Add RPC retries

* Write output roots to database

* Log output roots handling

* Update indexer README

* Add API v2 for Optimism Output Roots

* Add op_withdrawals table

* Add OptimismWithdrawals runner

* Prepare realtime optimism withdrawals fetching

* Add realtime optimism withdrawals fetching

* Define checks in init function

* log.first_topic can be nil

* Show total count of output roots in API v2

* Add msg_nonce gaps filler

* Refactoring

* Intermediate refactoring

* Add historical withdrawals handling and refactor

* Finish op_withdrawals table filling

* Small refactoring

* Add op_withdrawal_events table

* Add OptimismWithdrawalEvents runner

* Add OptimismWithdrawalEvent fetcher

* Update indexer README

* Add API v2 for Optimism Withdrawals

* Add env variables to common-blockscout.env and Makefile

* Set `from` as address object instead of just address hash for withdrawal

* mix format

* Add op_transaction_batches table

* Add OptimismTxnBatches runner

* Add a draft for OptimismTxnBatch fetcher

* Add a draft for OptimismTxnBatch

* Extend a draft for OptimismTxnBatch

* Extend OptimismTxnBatch

* Finish OptimismTxnBatch (without reorgs handling yet)

* Optimize OptimismTxnBatch fetcher

* Remove duplicated txn batches

* Add zlib_inflate_handler for empty case

* Add reorgs handling for txn batches

* Fix reorgs handling for txn batches

* Small refactor

* Finish Indexer.Fetcher.OptimismTxnBatch (without refactoring yet)

* Apply new ex_rlp version

* Add API v2 for Optimism Txn Batches

* Add env variables to common-blockscout.env and Makefile

* Refactor OptimismTxnBatch fetcher for mix credo

* Replace binary_slice with binary_part function to run with Elixir 1.13

* Update changelog

* Update indexer readme

* Rename op_withdrawals.l2_tx_hash field to l2_transaction_hash

* Rename l1_tx_hash fields to l1_transaction_hash

* Rename *tx* fields to *transaction* fields

* Rename env variables

* Rename env variables

* Add an indexer helper

* Add an indexer helper

* Small refactoring

* Fix tx_count for txn batches view

* Use EthereumJSONRPC.Block.ByHash instead of the raw call

* Infinity timeout for blocks query

* Small refactoring

* Refactor init function for two modules

* Small refactoring

* Rename l1_transaction_timestamp field to l1_timestamp

* Rename withdrawal_hash field to hash

* Refactor for decode_data function

* Refactor for mix credo

* Add INDEXER_OPTIMISM_L1_BATCH_BLOCKS_CHUNK_SIZE env and small refactoring

* Add INDEXER_OPTIMISM_L1_BATCH_BLOCKS_CHUNK_SIZE env to other files

* Add an index for l1_block_number field

* Add an index for l1_block_number field

* Remove redundant :ok

* Use faster way to count rows in a table

* Refactor reorgs monitor functions

* Clarify frame structure

* Reduce storage consumption for optimism transaction batches

* Reuse CacheHelper.estimated_count_from function

* Bedrock optimism deposits (#6993)

* Create `op_deposits` table

* Add OptimismDeposit runner

* WIP Fetcher

* Finish fetcher

* Integrate deposits into APIv2

* Add envs

* Fix requests

* Remove debug

* Update envs names

* Rename `tx` -> `transaction`

* Reuse `decode_data/2`

* Fix review

* Add `uninstall_filter`

* Fix formatting

* Switch to realtime mode more carefully

* Fix review

Allow nil in timestamp
Add progress logging
Improve check_interval calculation

* Fix logging and env

* Fix Association.NotLoaded error

* Replace switching to realtime mode log

* Remove excess start_block

* Fix reorg logging

* Fix `from_block` > `to_block` and add realtime logging

* Fix block boundaries

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* mix format

* Return total count of L2 entity by a separate API request

* Filter by consensus blocks

* Parallelize tx count operation and small refactoring

* Use read replica for L2 entities in API

* Parse block_number and tx_hash for Optimism Deposits module

* Return page_size back to 50

* Small fixes and refactoring

* Update apps/block_scout_web/lib/block_scout_web/api_router.ex

Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>

* Small optimization

* Use ecto association instead of explicit join for txn batches

* Refactoring

* Use Stream inspead of Enum

* Small refactoring

* Add assoc for transaction batches in OptimismFrameSequence

* Use common reorg monitor for Optimism modules

* Rename Explorer.Helpers to Explorer.Helper

* Don't start an optimism module unless the main optimism module is not started

* Don't start reorg monitor for optimism modules when it is not needed

* Small refactoring

* Remove debug broadcasting

* Add Optimism BedRock Deposits to the main page in API (#7200)

* Add Optimism BedRock Deposits to the main page in API

* Update changelog

* Pass the number of deposits instead of only one item per once

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Refactor for credo

* Output L1 fields in API v2 for transaction page

* Update changelog

* Use helper

* Refactor Indexer.Fetcher.Optimism

* Fix l1_timestamp issue in OptimismTxnBatch fetcher

* Reset Logger metadata before Indexer.Transform.OptimismWithdrawals.parse function finishes

* Fix IDs ordering in remove_duplicates function of Indexer.Fetcher.OptimismTxnBatch

* Consider rewriting of the first frame in Indexer.Fetcher.OptimismTxnBatch

* Fix Indexer.Fetcher.OptimismTxnBatch (consider chunking)

* Fix Indexer.Fetcher.OptimismTxnBatch

* Fix handling invalid frame sequences in Indexer.Fetcher.OptimismTxnBatch

* Read Optimism finalization period from a smart contract

* Fixes for dialyzer

* Fix for EthereumJSONRPC tests

* Fixes for Explorer tests

* Fixes for Explorer tests

* Fix of block/realtime/fetcher_test.exs

* mix format and small fixes for block_scout_web tests

* Reset GA cache

* Fix handling nil in PendingBlockOperation.estimated_count()

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>
Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>
istoril added a commit to cryptonomic0n/blockscout that referenced this pull request Apr 26, 2024
* fix: not found page for unknown blobs

* chore: changelog

* Fix tabs counter cache bug

* Changelog

* Fix no function clause matching in Integer.parse/2 (#9484)

* Fix no function clause matching in Integer.parse/2

* Changelog

* 6.2.1

* Add env vars for NFT sanitize migration

* 6.2.2

* Docker-compose 2.24.6 compatibility

* Add tsvector index on smart_contracts.name (#9487)

* Add tsvector index on smart_contracts.name

* Changelog

* Optimism chain type (#9460)

* Transaction page L1 fields

* Path fix

* Reduce the number of files from 19 to 5 in logs rotate config

* Customize optimism-goerli deployment

* Optimism branding

* Remove testnet logo text. OG uses customized label

* Fix Circles theme

* L1 tx fields fix for Optimism BedRock update

* Remove redundant line

* Add gas_price handling for elixir_to_params and change function ordering

* Remove l1TxOrigin handling for another version of RPC

* Add GA

* Fix realtime fetcher test

* Update Changelog

* Fix internal transactions processing for non-consensus blocks

* Lose consensus only for consensus=true blocks

* Fix handling transaction RPC responses without some fields

* Fix tests except for indexer module

* Add Optimism BedRock support (Txn Batches, Output Roots, Deposits, Withdrawals) (#6980)

* Add op_output_roots table

* Add OptimismOutputRoots runner

* Add initial code for output roots fetcher

* Add checks to init function

* Partially add logs and L1 reorgs handling

* Add reorgs handling

* Add RPC retries

* Write output roots to database

* Log output roots handling

* Update indexer README

* Add API v2 for Optimism Output Roots

* Add op_withdrawals table

* Add OptimismWithdrawals runner

* Prepare realtime optimism withdrawals fetching

* Add realtime optimism withdrawals fetching

* Define checks in init function

* log.first_topic can be nil

* Show total count of output roots in API v2

* Add msg_nonce gaps filler

* Refactoring

* Intermediate refactoring

* Add historical withdrawals handling and refactor

* Finish op_withdrawals table filling

* Small refactoring

* Add op_withdrawal_events table

* Add OptimismWithdrawalEvents runner

* Add OptimismWithdrawalEvent fetcher

* Update indexer README

* Add API v2 for Optimism Withdrawals

* Add env variables to common-blockscout.env and Makefile

* Set `from` as address object instead of just address hash for withdrawal

* mix format

* Add op_transaction_batches table

* Add OptimismTxnBatches runner

* Add a draft for OptimismTxnBatch fetcher

* Add a draft for OptimismTxnBatch

* Extend a draft for OptimismTxnBatch

* Extend OptimismTxnBatch

* Finish OptimismTxnBatch (without reorgs handling yet)

* Optimize OptimismTxnBatch fetcher

* Remove duplicated txn batches

* Add zlib_inflate_handler for empty case

* Add reorgs handling for txn batches

* Fix reorgs handling for txn batches

* Small refactor

* Finish Indexer.Fetcher.OptimismTxnBatch (without refactoring yet)

* Apply new ex_rlp version

* Add API v2 for Optimism Txn Batches

* Add env variables to common-blockscout.env and Makefile

* Refactor OptimismTxnBatch fetcher for mix credo

* Replace binary_slice with binary_part function to run with Elixir 1.13

* Update changelog

* Update indexer readme

* Rename op_withdrawals.l2_tx_hash field to l2_transaction_hash

* Rename l1_tx_hash fields to l1_transaction_hash

* Rename *tx* fields to *transaction* fields

* Rename env variables

* Rename env variables

* Add an indexer helper

* Add an indexer helper

* Small refactoring

* Fix tx_count for txn batches view

* Use EthereumJSONRPC.Block.ByHash instead of the raw call

* Infinity timeout for blocks query

* Small refactoring

* Refactor init function for two modules

* Small refactoring

* Rename l1_transaction_timestamp field to l1_timestamp

* Rename withdrawal_hash field to hash

* Refactor for decode_data function

* Refactor for mix credo

* Add INDEXER_OPTIMISM_L1_BATCH_BLOCKS_CHUNK_SIZE env and small refactoring

* Add INDEXER_OPTIMISM_L1_BATCH_BLOCKS_CHUNK_SIZE env to other files

* Add an index for l1_block_number field

* Add an index for l1_block_number field

* Remove redundant :ok

* Use faster way to count rows in a table

* Refactor reorgs monitor functions

* Clarify frame structure

* Reduce storage consumption for optimism transaction batches

* Reuse CacheHelper.estimated_count_from function

* Bedrock optimism deposits (#6993)

* Create `op_deposits` table

* Add OptimismDeposit runner

* WIP Fetcher

* Finish fetcher

* Integrate deposits into APIv2

* Add envs

* Fix requests

* Remove debug

* Update envs names

* Rename `tx` -> `transaction`

* Reuse `decode_data/2`

* Fix review

* Add `uninstall_filter`

* Fix formatting

* Switch to realtime mode more carefully

* Fix review

Allow nil in timestamp
Add progress logging
Improve check_interval calculation

* Fix logging and env

* Fix Association.NotLoaded error

* Replace switching to realtime mode log

* Remove excess start_block

* Fix reorg logging

* Fix `from_block` > `to_block` and add realtime logging

* Fix block boundaries

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* mix format

* Return total count of L2 entity by a separate API request

* Filter by consensus blocks

* Parallelize tx count operation and small refactoring

* Use read replica for L2 entities in API

* Parse block_number and tx_hash for Optimism Deposits module

* Return page_size back to 50

* Small fixes and refactoring

* Update apps/block_scout_web/lib/block_scout_web/api_router.ex

Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>

* Small optimization

* Use ecto association instead of explicit join for txn batches

* Refactoring

* Use Stream inspead of Enum

* Small refactoring

* Add assoc for transaction batches in OptimismFrameSequence

* Use common reorg monitor for Optimism modules

* Rename Explorer.Helpers to Explorer.Helper

* Don't start an optimism module unless the main optimism module is not started

* Don't start reorg monitor for optimism modules when it is not needed

* Small refactoring

* Remove debug broadcasting

* Add Optimism BedRock Deposits to the main page in API (#7200)

* Add Optimism BedRock Deposits to the main page in API

* Update changelog

* Pass the number of deposits instead of only one item per once

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Refactor for credo

* Output L1 fields in API v2 for transaction page

* Update changelog

* Use helper

* Refactor Indexer.Fetcher.Optimism

* Fix l1_timestamp issue in OptimismTxnBatch fetcher

* Reset Logger metadata before Indexer.Transform.OptimismWithdrawals.parse function finishes

* Fix IDs ordering in remove_duplicates function of Indexer.Fetcher.OptimismTxnBatch

* Consider rewriting of the first frame in Indexer.Fetcher.OptimismTxnBatch

* Fix Indexer.Fetcher.OptimismTxnBatch (consider chunking)

* Fix Indexer.Fetcher.OptimismTxnBatch

* Fix handling invalid frame sequences in Indexer.Fetcher.OptimismTxnBatch

* Read Optimism finalization period from a smart contract

* Fixes for dialyzer

* Fix for EthereumJSONRPC tests

* Fixes for Explorer tests

* Fixes for Explorer tests

* Fix of block/realtime/fetcher_test.exs

* mix format and small fixes for block_scout_web tests

* Reset GA cache

* Fix handling nil in PendingBlockOperation.estimated_count()

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>
Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>

* Fix autocomplete

* Fix merging conflicts

* Add exit handler to Indexer.Fetcher.OptimismWithdrawal

* Fix transactions ordering in Indexer.Fetcher.OptimismTxnBatch

* Update changelog

* Refactor to fix credo

* Mix credo fix

* Fix transaction batches module for L2 OP stack (#7827)

* Fix mixed transactions handling in Indexer.Fetcher.OptimismTxnBatch

* Ignore duplicated frame

* Update changelog

* Add sorting to the future frames list

* Change list order

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Remove unused aliases

* Ignore previously handled frame by OP transaction batches module (#8122)

* Ignore duplicated frame

* Update changelog

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Return alias for Explorer.Chain.Cache.Helper in chain.ex

* Ignore invalid frame by OP transaction batches module (#8208)

* Update changelog

* Ignore invalid frame

* Update changelog

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Fix Indexer.Fetcher.OptimismTxnBatch

* Fix API v2 for OP Withdrawals

* Refactor optimism fetchers init

* Add log for switching from fallback url

* Fix for Indexer.Fetcher.OptimismTxnBatch

* Add OP withdrawal status to transaction page in API (#8702)

* Add OP withdrawal status to transaction page in API

* Update changelog

* Small refactoring

* Update .dialyzer-ignore

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Add start pause to `Indexer.Fetcher.OptimismTxnBatch`

* Small refactor of `Indexer.Fetcher.OptimismTxnBatch`

* Consider consensus block only when retrieving OP withdrawal transaction status (#8811)

* Consider consensus block only when retrieving OP withdrawal transaction status

* Update changelog

* Clear GA cache

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Hotfix for optimism_withdrawal_transaction_status function

* Return all OP Withdrawals bound to L2 transaction

* Try to import config

* Remove unused functions from Explorer.Chain

* Refactor for mix credo

* Fix order of proxy standards: 1167, 1967

* Fixes in Optimism due to changed log topics type

* Fix for EthereumJSONRPC tests

* Clear GA cache and update cspell.json

* Fix indexer tests

* Return current exchange rate in api/v2/stats

* Fix log decoding bug

* Temp disable build of image for arm64

* Rewrite Indexer.Fetcher.OptimismTxnBatch module

* Add handling of span batches

* Add support of latest block for Optimism modules

* Update changelog and spelling

* Rewrite Indexer.Fetcher.OptimismTxnBatch module

* Add handling of span batches

* Add support of latest block for Optimism modules

* Refactoring

* Partially add specs and docs for public functions

* Refactoring

* add an entry to CHANEGELOG.md

* apply review (use origin entity instead of joined entity in with tx status)

* Fixes after rebase

* Remove old UI sustomizations

* Optimism chain type

* Change structure of folders

* Fixes after review

* Fix CHANGELOG

* Fixes after 2nd review

* Process 3d review: add tests for fee/2 function

* Process 4th review

* Review fix: move Op related functions from chain.ex

* Review fix: make OptimismFinalizationPeriod configurable

* Process review comment

* System.get_env("CHAIN_TYPE") => Application.get_env(:explorer, :chain_type)

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>
Co-authored-by: Qwerty5Uiop <alex000010@bk.ru>
Co-authored-by: varasev <33550681+varasev@users.noreply.github.com>
Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>
Co-authored-by: rlgns98kr <rlgns98kr@gmail.com>

* Fix missing `0x` prefix in `eth_getLogs` response  (#9514)

* Remove `transactionLogIndex` from `eth_getLogs` response

* Fix missing `0x` prefix for `blockNumber`, `logIndex`, `transactionIndex` in `eth_getLogs` response

* Fix logs pagination test

* Fix docs for `index` field in chain log

* Update smart contract ABI wiki link

* Add regression test for missing `0x` in `eth_getLogs`

* Add blob transaction count and transaction type filter to block view (#9490)

* feat: add blob tx count and filter to block view

* chore: changelog

* Add stability validators (#9390)

* Add stability validators

* Process review comments

* Fix tests

* ETH JSON RPC extension (#9409)

* Add new ETH JSON RPC methods

* Add new ETH JSON RPC API methods; Add ETH_JSON_RPC_MAX_BATCH_SIZE env

* Process review comments

* Bump logger_json from 5.1.3 to 5.1.4

Bumps [logger_json](https://github.com/Nebo15/logger_json) from 5.1.3 to 5.1.4.
- [Release notes](https://github.com/Nebo15/logger_json/releases)
- [Commits](https://github.com/Nebo15/logger_json/compare/5.1.3...5.1.4)

---
updated-dependencies:
- dependency-name: logger_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump credo from 1.7.4 to 1.7.5

Bumps [credo](https://github.com/rrrene/credo) from 1.7.4 to 1.7.5.
- [Release notes](https://github.com/rrrene/credo/releases)
- [Changelog](https://github.com/rrrene/credo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rrrene/credo/compare/v1.7.4...v1.7.5)

---
updated-dependencies:
- dependency-name: credo
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump floki from 0.35.4 to 0.36.0

Bumps [floki](https://github.com/philss/floki) from 0.35.4 to 0.36.0.
- [Release notes](https://github.com/philss/floki/releases)
- [Changelog](https://github.com/philss/floki/blob/main/CHANGELOG.md)
- [Commits](https://github.com/philss/floki/compare/v0.35.4...v0.36.0)

---
updated-dependencies:
- dependency-name: floki
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump postgrex from 0.17.4 to 0.17.5

Bumps [postgrex](https://github.com/elixir-ecto/postgrex) from 0.17.4 to 0.17.5.
- [Release notes](https://github.com/elixir-ecto/postgrex/releases)
- [Changelog](https://github.com/elixir-ecto/postgrex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/elixir-ecto/postgrex/compare/v0.17.4...v0.17.5)

---
updated-dependencies:
- dependency-name: postgrex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump phoenix_ecto from 4.4.3 to 4.5.0

Bumps [phoenix_ecto](https://github.com/phoenixframework/phoenix_ecto) from 4.4.3 to 4.5.0.
- [Changelog](https://github.com/phoenixframework/phoenix_ecto/blob/main/CHANGELOG.md)
- [Commits](https://github.com/phoenixframework/phoenix_ecto/compare/v4.4.3...v4.5.0)

---
updated-dependencies:
- dependency-name: phoenix_ecto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump ueberauth from 0.10.7 to 0.10.8

Bumps [ueberauth](https://github.com/ueberauth/ueberauth) from 0.10.7 to 0.10.8.
- [Release notes](https://github.com/ueberauth/ueberauth/releases)
- [Changelog](https://github.com/ueberauth/ueberauth/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ueberauth/ueberauth/compare/v0.10.7...v0.10.8)

---
updated-dependencies:
- dependency-name: ueberauth
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump @babel/core from 7.23.9 to 7.24.0 in /apps/block_scout_web/assets

Bumps [@babel/core](https://github.com/babel/babel/tree/HEAD/packages/babel-core) from 7.23.9 to 7.24.0.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.24.0/packages/babel-core)

---
updated-dependencies:
- dependency-name: "@babel/core"
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump chart.js from 4.4.1 to 4.4.2 in /apps/block_scout_web/assets

Bumps [chart.js](https://github.com/chartjs/Chart.js) from 4.4.1 to 4.4.2.
- [Release notes](https://github.com/chartjs/Chart.js/releases)
- [Commits](https://github.com/chartjs/Chart.js/compare/v4.4.1...v4.4.2)

---
updated-dependencies:
- dependency-name: chart.js
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump mini-css-extract-plugin in /apps/block_scout_web/assets

Bumps [mini-css-extract-plugin](https://github.com/webpack-contrib/mini-css-extract-plugin) from 2.8.0 to 2.8.1.
- [Release notes](https://github.com/webpack-contrib/mini-css-extract-plugin/releases)
- [Changelog](https://github.com/webpack-contrib/mini-css-extract-plugin/blob/master/CHANGELOG.md)
- [Commits](https://github.com/webpack-contrib/mini-css-extract-plugin/compare/v2.8.0...v2.8.1)

---
updated-dependencies:
- dependency-name: mini-css-extract-plugin
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump postcss-loader from 8.1.0 to 8.1.1 in /apps/block_scout_web/assets

Bumps [postcss-loader](https://github.com/webpack-contrib/postcss-loader) from 8.1.0 to 8.1.1.
- [Release notes](https://github.com/webpack-contrib/postcss-loader/releases)
- [Changelog](https://github.com/webpack-contrib/postcss-loader/blob/master/CHANGELOG.md)
- [Commits](https://github.com/webpack-contrib/postcss-loader/compare/v8.1.0...v8.1.1)

---
updated-dependencies:
- dependency-name: postcss-loader
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump @amplitude/analytics-browser in /apps/block_scout_web/assets

Bumps [@amplitude/analytics-browser](https://github.com/amplitude/Amplitude-TypeScript) from 2.5.1 to 2.5.2.
- [Release notes](https://github.com/amplitude/Amplitude-TypeScript/releases)
- [Commits](https://github.com/amplitude/Amplitude-TypeScript/compare/@amplitude/analytics-browser@2.5.1...@amplitude/analytics-browser@2.5.2)

---
updated-dependencies:
- dependency-name: "@amplitude/analytics-browser"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update dependabot.yml

* Bump @babel/preset-env in /apps/block_scout_web/assets

Bumps [@babel/preset-env](https://github.com/babel/babel/tree/HEAD/packages/babel-preset-env) from 7.23.9 to 7.24.0.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.24.0/packages/babel-preset-env)

---
updated-dependencies:
- dependency-name: "@babel/preset-env"
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump xss from 1.0.14 to 1.0.15 in /apps/block_scout_web/assets

Bumps [xss](https://github.com/leizongmin/js-xss) from 1.0.14 to 1.0.15.
- [Changelog](https://github.com/leizongmin/js-xss/blob/master/CHANGELOG.md)
- [Commits](https://github.com/leizongmin/js-xss/compare/v1.0.14...v1.0.15)

---
updated-dependencies:
- dependency-name: xss
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump autoprefixer in /apps/block_scout_web/assets

Bumps [autoprefixer](https://github.com/postcss/autoprefixer) from 10.4.17 to 10.4.18.
- [Release notes](https://github.com/postcss/autoprefixer/releases)
- [Changelog](https://github.com/postcss/autoprefixer/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/autoprefixer/compare/10.4.17...10.4.18)

---
updated-dependencies:
- dependency-name: autoprefixer
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* - change txpool_besuTransactions to txpool_besuPendingTransactions

- change runtime config enable pending transaction for besu client

* adding comment in pending transaction besu

* add PR tag

* Add cancun evm version

* Change value txpool_besuPendingTransactions to 512

* Fix get_blocks_by_events function

* Update changelog

* Prerelease workflow for Ethereum

* Fix timestamp handler for unfinalized zkEVM batches

* Update changelog

* Handle non-existent timestamp

* update log info

* Aff fallback for OTP_VERSION and ELIXIR_VERSION in the main CI workflow

* Fix format and cspell tests

* Fix EIP-1967 beacon proxy pattern detection

* Processing of eth_getStorageAt might return less 32 bytes

* Fix case for less 20 bytes in response

* Refactoring

* Fix MultipleResultsError in smart_contract_creation_tx_bytecode/1

* Changelog

* Support Optimism Ecotone upgrade by Indexer.Fetcher.Optimism.TxnBatch module (#9571)

* Draft for Ecotone support by Indexer.Fetcher.Optimism.TxnBatch module

* Small refactoring

* Use RollupL1ReorgMonitor for Optimism

* Move EIP-4844 decode function to Explorer.Chain.Optimism.TxnBatch

* Small refactoring

* Update changelog

* Update cspell.json

* Extend logs

* Fix init_continue in Indexer.Fetcher.Optimism

* mix format

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Bump ex_abi from 0.6.4 to 0.7.0 (#9329)

* Bump ex_abi from 0.6.4 to 0.7.0

Bumps [ex_abi](https://github.com/poanetwork/ex_abi) from 0.6.4 to 0.7.0.
- [Release notes](https://github.com/poanetwork/ex_abi/releases)
- [Changelog](https://github.com/poanetwork/ex_abi/blob/master/CHANGELOG.md)
- [Commits](https://github.com/poanetwork/ex_abi/compare/0.6.4...0.7.0)

---
updated-dependencies:
- dependency-name: ex_abi
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Shorten event signatures to first 4 bytes for consistency

* Renamed variables and improved documentation for clearness

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Fedor Ivanov <ivnfedor@gmail.com>

* Bump phoenix_ecto from 4.5.0 to 4.5.1

Bumps [phoenix_ecto](https://github.com/phoenixframework/phoenix_ecto) from 4.5.0 to 4.5.1.
- [Changelog](https://github.com/phoenixframework/phoenix_ecto/blob/main/CHANGELOG.md)
- [Commits](https://github.com/phoenixframework/phoenix_ecto/compare/v4.5.0...v4.5.1)

---
updated-dependencies:
- dependency-name: phoenix_ecto
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump ex_doc from 0.31.1 to 0.31.2

Bumps [ex_doc](https://github.com/elixir-lang/ex_doc) from 0.31.1 to 0.31.2.
- [Release notes](https://github.com/elixir-lang/ex_doc/releases)
- [Changelog](https://github.com/elixir-lang/ex_doc/blob/main/CHANGELOG.md)
- [Commits](https://github.com/elixir-lang/ex_doc/compare/v0.31.1...v0.31.2)

---
updated-dependencies:
- dependency-name: ex_doc
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump ecto from 3.11.1 to 3.11.2

Bumps [ecto](https://github.com/elixir-ecto/ecto) from 3.11.1 to 3.11.2.
- [Release notes](https://github.com/elixir-ecto/ecto/releases)
- [Changelog](https://github.com/elixir-ecto/ecto/blob/master/CHANGELOG.md)
- [Commits](https://github.com/elixir-ecto/ecto/compare/v3.11.1...v3.11.2)

---
updated-dependencies:
- dependency-name: ecto
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump redix from 1.3.0 to 1.4.1

Bumps [redix](https://github.com/whatyouhide/redix) from 1.3.0 to 1.4.1.
- [Changelog](https://github.com/whatyouhide/redix/blob/main/CHANGELOG.md)
- [Commits](https://github.com/whatyouhide/redix/compare/v1.3.0...v1.4.1)

---
updated-dependencies:
- dependency-name: redix
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update external-backend.yml

Removed "repends_on: backend"

* Fix duplicate read methods (#9591)

* Fix the typo in docs

* Fix duplicated results in `methods-read` endpoint

* Add regression test ensuring read-methods are not duplicated

* Update `CHANGELOG.md`

* chore: remove repetitive words

Signed-off-by: carrychair <linghuchong404@gmail.com>

* Fix token instance transform panic (#9601)

* fix: token instance transform panic

* chore: changelog

* Update token transfers block_consensus by block_number

* Fix logging

* Fix Geth block internal transactions fetching

* Fix skipped read methods (#9621)

* Bump ex_abi from 0.7.0 to 0.7.1

* Add regression test for function ABI missing `outputs` field

* chore: fix some comments

Signed-off-by: one230six <723682061@qq.com>

* Improve gasprice oracle time estimation fallback (#9582)

* Improve gasprice oracle time estimation fallback

* Fix review comments

* API v1 bridgedtokenlist endpoint (#9506)

* bridgedtokenlist API endpoint

* Change spec for bridged_token_usd_cap function

* Fix review comments

* Return bridged_token_usd_cap as string

* Fix typos in CHANGELOG

* Find a single creation tx for smart-contract

* Convert outputs to string in smart_contract_view.ex (#9529)

* Convert integers and bools to string in smart_contract_view.ex

* Changelog

* Fix tests

* Fix test to support new numbers format

---------

Co-authored-by: Fedor Ivanov <ivnfedor@gmail.com>

* Add user_op interpretation (#9473)

* Add /api/v2/proxy/account-abstraction/operations/{operation_hash_param}/summary endpoint

* Add changelog, fix test

* Process review comments

* Format .eex files

* Setup alternative hex.pm mirrors (#9622)

* Setup alternative hex.pm mirrors

* Update `CHANGELOG.md`

* Release workflow for Gnosis chain

* Massive blocks fetcher (#9486)

* Massive blocks fetcher

* Improve massive blocks fetcher log

* Update apps/explorer/lib/explorer/utility/massive_block.ex

Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>

* Add low priority queue for MassiveBlocksFetcher

---------

Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>

* zksync chain type support (#9631)

* zkSync customizations

* Insert placeholders instead of deriving current token balances

* ZkSync Batches status tracking (#9080)

* initial version of batch tracking

* missed file added

* attempt to add DB migration

* Finalized L1 txs tracking

* keep batches in DB

* Batches statuses tracker introduction

* rpc endponts to get batches data

* extended views for blocks and transactions

* Refactoring of fetchers

* Fetch historical blocks

* handle_info calls simplified

* Ability to recover missed blocks

* zksync info in a separate sub-map

* added doc comments, part 1

* finalized doc comments

* actual bathes count instead of the last imported batch

* fix formatting

* credo fixes

* Address dialyzer warnings

* Fix spelling

* remaining issues with spelling and dialyzer

* Attempt to address BlockScout Web Tests issue

* review comments addressed, part 1

* review comments addressed, part 2

* collection all_options for import module reworked to get rid of dialyzer findings

* removed unnecessary functionality

* proper import

* Credo fixes

* Add CHAIN_TYPE=zksync to image generation workflow

* Proper handling of empty transactions list in etc_getBlockByNumber

* Merge master

* Address merge issues

* Fix format

* Refactoring of chain type specific code for block and transaction views

* Consistent name for functions

* add exceptions for Credo.Check.Design.AliasUsage

* Fix rebasing conflicts

* Fix rebase conflicts

* fix issue with stability fees in tx view

* make Stability related tests dependent on chain type in compile time

* move zksync related migration

* Changelog updated

* removal of duplicated migration

* List r,s,v as optional attributes for transaction

---------

Co-authored-by: Viktor Baranov <baranov.viktor.27@gmail.com>
Co-authored-by: Qwerty5Uiop <alex000010@bk.ru>

* Don't insert pbo for not inserted blocks (#9629)

* Rewrite query for token transfers on address to eliminate "or" (#9576)

* Rewrite query for token transfers on address to eliminate "or"

* Add token_transfers [:to_address_hash, :block_number] index

* Review processing

* Review processing #2

* Update apps/explorer/lib/explorer/chain/token_transfer.ex

Co-authored-by: Kirill Fedoseev <kirill@blockscout.com>

---------

Co-authored-by: Kirill Fedoseev <kirill@blockscout.com>

* Add last output root size counter (#9532)

* Fix no function clause matching in BENS.item_to_address_hash_strings/1 (#9640)

* Re-add missing blob transactions count (#9644)

* fix: add missing blob tx count

* chore: refactor

* chore: changelog

* Fix fetch_coin_balance query to compare between balances with values (#9638)

* Fix fetch_coin_balance query to compare between balances with non-nil values

* Review fix

* Update CHANGELOG record

* Reset missing ranges collector to max number after the cycle is done (#9635)

* Fix infinite retries for orphaned blobs (#9620)

* fix: implement finite retries for orphaned blobs

* chore: changelog

* Separate errors by type in EndpointAvailabilityObserver (#9511)

* Separate errors by type in EndpointAvailabilityObserver

* Add eth_call fallback url

* Update url unavailable warning message

* Update switching back from fallback url message

* Fix zero balances coming via WS (#9510)

* Add secondary coin and transaction stats (#9483)

* Add volume_24h

* Add secondary coin and transactions stats

* Process review comments

* Allow different source for secondary coin

* Fix exchange_rates_secondary_coin_price_source

---------

Co-authored-by: Nikita Pozdniakov <nikitosing4@mail.ru>

* ERC-404 basic support (#9407)

* ERC-404 basic support

* rename nft_token_ to nft_

* ERC-404 support additions

* Cover with token transfer parsing tests

* Cover ERC-404 with token balance tests

* Cover ERC-404 with current token balance tests

* Notification summary tests

* Some more tests

* Update apps/block_scout_web/lib/block_scout_web/views/tokens/helper.ex

Co-authored-by: Qwerty5Uiop <105209995+Qwerty5Uiop@users.noreply.github.com>

* Process review comments

* Process review comment

* Format changes

---------

Co-authored-by: Qwerty5Uiop <105209995+Qwerty5Uiop@users.noreply.github.com>

* Hotfix for Optimism Ecotone batch blobs indexing (#9646)

* Hotfix for Optimism Ecotone batch blobs indexing

* Update changelog

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* 6.3.0

* Remove duplicated tx hashes while indexing OP batches (#9652)

* Remove duplicated tx hashes while indexing OP batches

* Update changelog

* Update changelog

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* Send timeout param in debug_traceBlockByNumber request (#9654)

* Tokens import improvements (#9653)

* Tokens import improvements

* Fix review comment

* Remove debug code

* Update CHANGELOG, add zksync release workflow

* Allow to manually trigger zksync workflow

* Disallow zksync release workflow dispatch

* chore: Use git-cliff changelog generator (#9687)

* chore: Use git-cliff changelog generator

* Update cliff.toml

Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>

* Update cliff.toml

Co-authored-by: Fedor Ivanov <ivnfedor@gmail.com>

---------

Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>
Co-authored-by: Fedor Ivanov <ivnfedor@gmail.com>

* chore: Bump floki from 0.36.0 to 0.36.1 (#9670)

Bumps [floki](https://github.com/philss/floki) from 0.36.0 to 0.36.1.
- [Release notes](https://github.com/philss/floki/releases)
- [Changelog](https://github.com/philss/floki/blob/main/CHANGELOG.md)
- [Commits](https://github.com/philss/floki/compare/v0.36.0...v0.36.1)

---
updated-dependencies:
- dependency-name: floki
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump follow-redirects from 1.15.4 to 1.15.6 in /apps/explorer (#9648)

Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.15.4 to 1.15.6.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.15.4...v1.15.6)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: fix typos (#9693)

* chore: Bump bureaucrat from 0.2.9 to 0.2.10 (#9669)

Bumps [bureaucrat](https://github.com/api-hogs/bureaucrat) from 0.2.9 to 0.2.10.
- [Release notes](https://github.com/api-hogs/bureaucrat/releases)
- [Commits](https://github.com/api-hogs/bureaucrat/commits)

---
updated-dependencies:
- dependency-name: bureaucrat
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: erc-404 type stored in token balances tables (#9700)

* refactor: `Enum.count` to `Enum.empty?` (#9666)

* refactor Enum.count to Enum.empty?.

* use `not` instead of `!`

* fix typo.

* improve readability.

* refactor if negate branching.

* use unless in template.

* format code.

* remove CHANGELOG record.

* chore: Bump express in /apps/block_scout_web/assets (#9725)

Bumps [express](https://github.com/expressjs/express) from 4.18.2 to 4.19.2.
- [Release notes](https://github.com/expressjs/express/releases)
- [Changelog](https://github.com/expressjs/express/blob/master/History.md)
- [Commits](https://github.com/expressjs/express/compare/4.18.2...4.19.2)

---
updated-dependencies:
- dependency-name: express
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump cldr_utils from 2.24.2 to 2.25.0 (#9723)

Bumps [cldr_utils](https://github.com/elixir-cldr/cldr_utils) from 2.24.2 to 2.25.0.
- [Release notes](https://github.com/elixir-cldr/cldr_utils/releases)
- [Changelog](https://github.com/elixir-cldr/cldr_utils/blob/main/CHANGELOG.md)
- [Commits](https://github.com/elixir-cldr/cldr_utils/compare/v2.24.2...v2.25.0)

---
updated-dependencies:
- dependency-name: cldr_utils
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: erc-404 token transfers null value (#9698)

* fix: erc-404 token transfers null value

* Process review comment

* fix: Fix Geth block tracing errors handling (#9672)

* fix: Change CoinGecko token image attribute priority (#9671)

* fix: Add missing preloads (#9520)

* Add missing preloads

* Changelog

* Refactor flat_1155_batch_token_transfers/2

* Process review comments

* chore: Temporarily ignore OP batches written to Celestia (#9734)

* Ignore OP batches written to Celestia

* Add Celestia to cspell.json

* Fix typo in Indexer.Transform.Optimism.Withdrawals

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* perf: Add EIP4844 blob transactions index (#9661)

* fix: add blob transactions index

* fix: migration lock

* fix: Separate ZkSync and ZkEvm readers in API controller (#9749)

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* fix: Fix UTF-8 json handling in NFT metadata fetching (#9707)

* fix: Fix UTF-8 json handling in NFT metadata fetching

* Fix cspell

* Process review comment

* Fix tests

* feat: Add trying to decode internal calldata for user ops (#9675)

* Add trying to decode internal calldata for user ops

* Changelog

* Put proper address structure

* Revert CHANGELOG.md

* Update apps/block_scout_web/lib/block_scout_web/microservice_interfaces/transaction_interpretation.ex

Co-authored-by: Kirill Fedoseev <kirill@blockscout.com>

* Process review comments

* Update typespec

---------

Co-authored-by: Kirill Fedoseev <kirill@blockscout.com>

* feat: Improvements in zksync batch related transactions requests (#9680)

* pagination for /api/v2/transactions/zkevm-batch/:batch_number

* omit transactions list in /api/v2/zksync/batches/:batch_number

* changelog updated

* Copy&paste issue fixed

* Rollback changes in CHANGELOG

* chore: Enable Rust sc-verifier microservice by default (#9752)

* chore: Enable Rust sc-verifier microservice by default

* Fix explorer tests

* Fix block_scout_web tests

* chore: Bump core-js in /apps/block_scout_web/assets (#9781)

Bumps [core-js](https://github.com/zloirock/core-js/tree/HEAD/packages/core-js) from 3.36.0 to 3.36.1.
- [Release notes](https://github.com/zloirock/core-js/releases)
- [Changelog](https://github.com/zloirock/core-js/blob/master/CHANGELOG.md)
- [Commits](https://github.com/zloirock/core-js/commits/v3.36.1/packages/core-js)

---
updated-dependencies:
- dependency-name: core-js
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump @babel/preset-env in /apps/block_scout_web/assets (#9784)

Bumps [@babel/preset-env](https://github.com/babel/babel/tree/HEAD/packages/babel-preset-env) from 7.24.0 to 7.24.3.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.24.3/packages/babel-preset-env)

---
updated-dependencies:
- dependency-name: "@babel/preset-env"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump sweetalert2 in /apps/block_scout_web/assets (#9783)

Bumps [sweetalert2](https://github.com/sweetalert2/sweetalert2) from 11.10.5 to 11.10.7.
- [Release notes](https://github.com/sweetalert2/sweetalert2/releases)
- [Changelog](https://github.com/sweetalert2/sweetalert2/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sweetalert2/sweetalert2/compare/v11.10.5...v11.10.7)

---
updated-dependencies:
- dependency-name: sweetalert2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump solc from 0.8.24 to 0.8.25 in /apps/explorer (#9789)

Bumps [solc](https://github.com/ethereum/solc-js) from 0.8.24 to 0.8.25.
- [Commits](https://github.com/ethereum/solc-js/compare/v0.8.24...v0.8.25)

---
updated-dependencies:
- dependency-name: solc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump @amplitude/analytics-browser in /apps/block_scout_web/assets (#9788)

Bumps [@amplitude/analytics-browser](https://github.com/amplitude/Amplitude-TypeScript) from 2.5.2 to 2.6.1.
- [Release notes](https://github.com/amplitude/Amplitude-TypeScript/releases)
- [Commits](https://github.com/amplitude/Amplitude-TypeScript/compare/@amplitude/analytics-browser@2.5.2...@amplitude/analytics-browser@2.6.1)

---
updated-dependencies:
- dependency-name: "@amplitude/analytics-browser"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump postcss in /apps/block_scout_web/assets (#9785)

Bumps [postcss](https://github.com/postcss/postcss) from 8.4.35 to 8.4.38.
- [Release notes](https://github.com/postcss/postcss/releases)
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/postcss/compare/8.4.35...8.4.38)

---
updated-dependencies:
- dependency-name: postcss
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump webpack in /apps/block_scout_web/assets (#9787)

Bumps [webpack](https://github.com/webpack/webpack) from 5.90.3 to 5.91.0.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.90.3...v5.91.0)

---
updated-dependencies:
- dependency-name: webpack
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump @babel/core in /apps/block_scout_web/assets (#9782)

Bumps [@babel/core](https://github.com/babel/babel/tree/HEAD/packages/babel-core) from 7.24.0 to 7.24.3.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.24.3/packages/babel-core)

---
updated-dependencies:
- dependency-name: "@babel/core"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feat: GraphQL management env vars (#9751)

* feat: GraphQL management env vars

* Process review comment

* feat: Support the 2nd version of L2<->L1 Polygon zkEVM Bridge (#9637)

* Add support for Polygon zkEVM bridge v2 (draft version)

* Refactoring for mix credo

* Fix config/runtime.exs

* Support arbitrary network ids

* Additional env variables

* Fixes for the bridge

* Update changelog

* Fix spelling

* mix format

* Sanitize token decimals in zkevm and tx action modules

* Small refactoring

* Update spelling

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* chore: Bump sass from 1.71.1 to 1.72.0 in /apps/block_scout_web/assets (#9780)

Bumps [sass](https://github.com/sass/dart-sass) from 1.71.1 to 1.72.0.
- [Release notes](https://github.com/sass/dart-sass/releases)
- [Changelog](https://github.com/sass/dart-sass/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sass/dart-sass/compare/1.71.1...1.72.0)

---
updated-dependencies:
- dependency-name: sass
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Remove /api/account/v1 path (#9660)

* chore: Bump autoprefixer in /apps/block_scout_web/assets (#9786)

Bumps [autoprefixer](https://github.com/postcss/autoprefixer) from 10.4.18 to 10.4.19.
- [Release notes](https://github.com/postcss/autoprefixer/releases)
- [Changelog](https://github.com/postcss/autoprefixer/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/autoprefixer/compare/10.4.18...10.4.19)

---
updated-dependencies:
- dependency-name: autoprefixer
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: Update token's holder_count in the db from ETS module (#9623)

* fix: Update holder_count n db from ETS module

* Update apps/explorer/lib/explorer/chain/token.ex

Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>

* Update apps/explorer/lib/explorer/counters/token_holders_counter.ex

Co-authored-by: nikitosing <32202610+nikitosing@users.noreply.github.com>

* update token holders cache in async mode

* Fixing tests

---------

Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>
Co-authored-by: nikitosing <32202610+nikitosing@users.noreply.github.com>

* feat: Add /api/v2/tokens/:address_hash_param/holders/csv endpoint (#9722)

* feat: Add /api/v2/tokens/:address_hash_param/holders/csv endpoint

* Remove divide_decimals/2 in BlockScoutWeb.CurrencyHelper

* fix: Add block range check into OP Withdrawals fetcher (#9770)

* fix: Add block range check into OP Withdrawals fetcher

* Refactor

* Add the same check for Polygon Edge

* Fix spec for fill_block_range function

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* chore: Reduce default API v1 limit by key 50 -> 10 (#9799)

* chore: Exclude latest tag update from alpha releases (#9800)

* feat: Contract code on-demand fetcher (#9708)

* Contract code on-demand fetcher

* Review comments #1

* Review comments #2: ignore addresses with nonce

* Review comments 3: fix threshold calculation

* Refine trigger_fetch function

* Skip updating retries number in case of failed eth_getCode response

* fix: fetching GraphQL schema by GraphiQL IDE (#9630)

* feat: Allow for custom base_url for fetching prices (#9679)

* update env for new coingecko

* fix(cmc env) - add cmc to env

* nit(add changelog)

* fix(base_url override) fix comments

* fix(changelog) remove changelog changes

* ci: rework chain type matrix in CI runs (#9704)

* ci: remove polygon edge and suave from CI runs

* ci: run chain type workflows based on pr labels

* chore: keep suave and polygon_edge as optional

* ci: update matrix selection logic

* ci: re-run ci if PR is relabeled

* ci: fix default chain type

* ci: refactor default chain types

* feat: internal txs fetching for Arbitrum (#9737)

* Arbitrum internal transactions fetching support

* Remove arbitrum JSON RPC variant

* CHAIN_TYPE=arbitrum

* Add indexer test

* Update apps/explorer/lib/explorer/chain/internal_transaction/call_type.ex

Co-authored-by: Kirill Fedoseev <kirill@blockscout.com>

* Update apps/explorer/lib/explorer/chain/internal_transaction/type.ex

Co-authored-by: Kirill Fedoseev <kirill@blockscout.com>

* Process review comments

---------

Co-authored-by: Kirill Fedoseev <kirill@blockscout.com>

* fix: Replace tx gas_price with effectiveGasPrice from receipt (#9733)

* fix: Replace tx gas_price with effectiveGasPrice from receipt

* Fix format

* feat: support for internal user operation calldata decoded by microservice (#9776)

* feat: Add rate limits to graphQL API (#9771)

* feat: Add rate limits to graphQL API

* Fix tests

* Process review comments

* Update common-blockscout.env

* feat: Support verifier alliance and eth-bytecode-db v1.7.0 changes (#9724)

* feat: Support verifier alliance and eth-bytecode-db v1.7.0 changes

* Fix bug and tests

* Add tests

* Process review comments

* feat: Integrate Metadata microservice (#9706)

* feat: Integrate Metadata microservice

* Fix formatting

* feat: Add /api/v2/blocks/{hash_or_number}/internal-transactions endpoint (#9668)

* Add /api/v2/blocks/{hash_or_number}/internal-transactions endpoint

* Fix tests

* Fix tests

* Add tests

* Changelog

* Revert CHANGELOG.md

* Process review comment

* Fix credo

* 6.4.0

* feat: Secondary coin price in `api/v2/stats` (#9777)

* Update CHANGELOG

* fix: Apply quantity_to_integer/1 to effectiveGasPrice (#9812)

* Update CHANGELOG

* Add zksync to pre-release CI

* chore: Bump plug_cowboy from 2.7.0 to 2.7.1 (#9844)

Bumps [plug_cowboy](https://github.com/elixir-plug/plug_cowboy) from 2.7.0 to 2.7.1.
- [Changelog](https://github.com/elixir-plug/plug_cowboy/blob/master/CHANGELOG.md)
- [Commits](https://github.com/elixir-plug/plug_cowboy/compare/v2.7.0...v2.7.1)

---
updated-dependencies:
- dependency-name: plug_cowboy
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump absinthe_phoenix from 2.0.2 to 2.0.3 (#9840)

Bumps [absinthe_phoenix](https://github.com/absinthe-graphql/absinthe_phoenix) from 2.0.2 to 2.0.3.
- [Changelog](https://github.com/absinthe-graphql/absinthe_phoenix/blob/main/CHANGELOG.md)
- [Commits](https://github.com/absinthe-graphql/absinthe_phoenix/commits)

---
updated-dependencies:
- dependency-name: absinthe_phoenix
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump number from 1.0.4 to 1.0.5 (#9843)

Bumps [number](https://github.com/danielberkompas/number) from 1.0.4 to 1.0.5.
- [Changelog](https://github.com/danielberkompas/number/blob/master/CHANGELOG.md)
- [Commits](https://github.com/danielberkompas/number/compare/v1.0.4...v1.0.5)

---
updated-dependencies:
- dependency-name: number
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump junit_formatter from 3.3.1 to 3.4.0 (#9842)

Bumps [junit_formatter](https://github.com/victorolinasc/junit-formatter) from 3.3.1 to 3.4.0.
- [Release notes](https://github.com/victorolinasc/junit-formatter/releases)
- [Changelog](https://github.com/victorolinasc/junit-formatter/blob/master/CHANGELOG.md)
- [Commits](https://github.com/victorolinasc/junit-formatter/compare/v3.3.1...v3.4.0)

---
updated-dependencies:
- dependency-name: junit_formatter
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Remove /config/json-rpc-url API endpoint (#9798)

* feat: Allow the use of Coingecko demo account (#9835)

Co-authored-by: GimluCom <>

* chore: Bump ex_abi from 0.7.1 to 0.7.2 (#9841)

Bumps [ex_abi](https://github.com/poanetwork/ex_abi) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/poanetwork/ex_abi/releases)
- [Changelog](https://github.com/poanetwork/ex_abi/blob/master/CHANGELOG.md)
- [Commits](https://github.com/poanetwork/ex_abi/compare/0.7.1...0.7.2)

---
updated-dependencies:
- dependency-name: ex_abi
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: fix some typos in conments (#9838)

Signed-off-by: wanshuangcheng <wanshuangcheng@outlook.com>

* chore: Reduce number of warnings in web tests (#9851)

* fix: Add non-unique log_index support in update_token_instances_owner (#9862)

* chore: Increase MissingRangesCollector past check interval after the first cycle (#9872)

* fix: Disable BlockReward fetcher for unsupported variants (#9859)

* fix: Disable BlockReward fetcher for unsupported variants

* Update Application envs

* process review comment

* chore: Add metrics for realtime event handlers queue length (#9822)

* fix: Broadcast realtime coin balances (#9804)

* chore: move `has_methods_*` fields to `/smart-contracts` endpoint response (#9599)

* chore: move `methods`-related fields to `/api/v2/smart-contracts/{address_hash}` endpoint response`

* fix: add custom abi predicate fields to unverified contract response

* refactor: do not put not necessary `conn`

* chore: add RPC-call expectations in tests

* fix: Correct processing of sized array to view in API v2 (#9854)

* Fix sized array read

* Rewrite render_json to correctly process tuple types

* Cover with regression test

* fix: handle transactions with `gas_price` set to `nil` in `transaction_revert_reason/2` (#9647)

* refactor: use `Wei.hex_format` for `gas`

* fix: handle `nil` values in `Wei.hex_format`

* chore: change `@spec` of `hex_format` function to allow `nil` as input value

* chore: Bump ex_cldr_units from 3.16.4 to 3.16.5 (#9884)

Bumps [ex_cldr_units](https://github.com/elixir-cldr/cldr_units) from 3.16.4 to 3.16.5.
- [Release notes](https://github.com/elixir-cldr/cldr_units/releases)
- [Changelog](https://github.com/elixir-cldr/cldr_units/blob/main/CHANGELOG.md)
- [Commits](https://github.com/elixir-cldr/cldr_units/commits)

---
updated-dependencies:
- dependency-name: ex_cldr_units
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump ex_secp256k1 from 0.7.2 to 0.7.3 (#9888)

Bumps [ex_secp256k1](https://github.com/ayrat555/ex_secp256k1) from 0.7.2 to 0.7.3.
- [Release notes](https://github.com/ayrat555/ex_secp256k1/releases)
- [Changelog](https://github.com/ayrat555/ex_secp256k1/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ayrat555/ex_secp256k1/compare/v0.7.2...v0.7.3)

---
updated-dependencies:
- dependency-name: ex_secp256k1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump cloak_ecto from 1.2.0 to 1.3.0 (#9890)

Bumps [cloak_ecto](https://github.com/danielberkompas/cloak_ecto) from 1.2.0 to 1.3.0.
- [Release notes](https://github.com/danielberkompas/cloak_ecto/releases)
- [Changelog](https://github.com/danielberkompas/cloak_ecto/blob/master/CHANGELOG.md)
- [Commits](https://github.com/danielberkompas/cloak_ecto/compare/v1.2.0...v1.3.0)

---
updated-dependencies:
- dependency-name: cloak_ecto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump logger_file_backend from 0.0.13 to 0.0.14 (#9885)

Bumps [logger_file_backend](https://github.com/onkel-dirtus/logger_file_backend) from 0.0.13 to 0.0.14.
- [Changelog](https://github.com/onkel-dirtus/logger_file_backend/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onkel-dirtus/logger_file_backend/commits/v0.0.14)

---
updated-dependencies:
- dependency-name: logger_file_backend
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump tesla from 1.8.0 to 1.9.0 (#9886)

Bumps [tesla](https://github.com/teamon/tesla) from 1.8.0 to 1.9.0.
- [Release notes](https://github.com/teamon/tesla/releases)
- [Commits](https://github.com/teamon/tesla/compare/v1.8.0...v1.9.0)

---
updated-dependencies:
- dependency-name: tesla
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Unknown token in email template (#9883)

* fix: Refactor catchup rudimentaries + fix graceful shutdown (#9729)

* fix: Refactor catchup rudimentaries + fix graceful shutdown

* Add INDEXER_GRACEFUL_SHUTDOWN_PERIOD env var + docs

* chore: Separate reorgs from blocks that just need refetch (#9674)

* chore: Bump ex_doc from 0.31.2 to 0.32.1 (#9889)

Bumps [ex_doc](https://github.com/elixir-lang/ex_doc) from 0.31.2 to 0.32.1.
- [Release notes](https://github.com/elixir-lang/ex_doc/releases)
- [Changelog](https://github.com/elixir-lang/ex_doc/blob/main/CHANGELOG.md)
- [Commits](https://github.com/elixir-lang/ex_doc/compare/v0.31.2...v0.32.1)

---
updated-dependencies:
- dependency-name: ex_doc
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Victor Baranov <baranov.viktor.27@gmail.com>

* chore: Add queue expanding logic to memory monitor (#9870)

* fix: Coin balances fetcher error logging (#9902)

* Zerion API proxy (#9896)

* Zerion API proxy

* Add envs to common-blockscout.env

* Change proxy endpoint path

* IPFS gateway URL extra params (#9898)

* IPFS gateway URL authentication params

* Fix cspell

* Fix explorer tests

* Add IPFS gateway URL custom header support

* Parse cataloged value of IPFS_GATEWAY_URL_PARAM_LOCATION

* Update apps/indexer/lib/indexer/fetcher/token_instance/metadata_retriever.ex

Co-authored-by: Qwerty5Uiop <105209995+Qwerty5Uiop@users.noreply.github.com>

* Update apps/indexer/lib/indexer/fetcher/token_instance/metadata_retriever.ex

Co-authored-by: Qwerty5Uiop <105209995+Qwerty5Uiop@users.noreply.github.com>

* Process review comments

* Return exit(:shutdown)

---------

Co-authored-by: Qwerty5Uiop <105209995+Qwerty5Uiop@users.noreply.github.com>

* fix: Expand memory only if it was shrunk (#9907)

* chore: fix some typos in comments (#9900)

Signed-off-by: TechVest <techdashen@qq.com>

* feat: Return number of days in address's coin-balance-history-by-day API v2 endpoint (#9806)

* Return days in coin-balance-history-by-day API v2 endpoint

* Actualize test

* feat: Support Optimism Fault Proofs (#9892)

* Add game_index field to op_withdrawal_events db table

* Draft for dispute games indexer

* Add extra data indexing into Indexer.Fetcher.Optimism.DisputeGame

* Add game statuses fetching into Indexer.Fetcher.Optimism.DisputeGame

* Almost final Indexer.Fetcher.Optimism.DisputeGame

* Setting constant values in Indexer.Fetcher.Optimism.DisputeGame

* Extend Optimism Withdrawal statuses

* Add API for Optimism dispute games

* Handle a case of non-existent game

* Add specs and docs

* DisputeGame fetcher waits infinitely until OptimismPortal is updated

* Fix spelling

* Small refactoring

---------

Co-authored-by: POA <33550681+poa@users.noreply.github.com>

* perf: Improve performance of token page transfers tab (#9809)

* Improve performance of token page transfers tab

* mix format

* Separate index creation and deletion to defferent migrations

* Rename migration

* fix: Remove backend dependency in microservices.yml (#9905)

* perf: Enhance index for token holders list (#9816)

* Enhance index for token holders list

* Separate index creation and deletion to different migrations

* perf: Adjust unfetched_address_token_balances_index to fit all bound query conditions (#9912)

* fix: Adjust unfetched_address_token_balances_index to fit all bound query conditions

* Rewrite indexes in Ecto meta-language

* mix format

* feat: Exit on provided invalid CHAIN_TYPE (#9904)

* fix: `coin_price_change_percentage` calculation (#9774)

* perf: Paging function edge cases fix (#9820)

* Paging function edge cases fix

* Process other edge cases of zeros in pagination function input

* Update apps/explorer/lib/explorer/chain.ex

Co-authored-by: Qwerty5Uiop <105209995+Qwerty5Uiop@users.noreply.github.com>

* Update apps/explorer/lib/explorer/chain.ex

Co-authored-by: Qwerty5Uiop <105209995+Qwerty5Uiop@users.noreply.github.com>

* Fix tests

* Process review comment

* fix formatting

---------

Co-authored-by: Qwerty5Uiop <105209995+Qwerty5Uiop@users.noreply.github.com>

* feat: Certified smart contracts (#9910)

* Certified smart-contracts

* Prioritize certified smart-contracts in the search

* Refactoring: remove CustomContractsHelper

* Return certified in the list and in the search

* mix format

* Fix tests

* Process review comment

* 6.5.0

* Add zksync to supported_chain_types

* fix: Fix metadata preload (#9925)

* fix: Fix metadata preload

* Add regression test

* fix: handle "null" filter in api/v1/logs-csv (#9933)

* Update CHANGELOG

* fix: `GAS_PRICE_ORACLE_NUM_OF_BLOCKS` calculation (#9943)

* Update CHANGELOG

* fix: Set refetch_needed: false on block import (#9953)

* Update CHANGELOG

* Add Optimism to pre-release workflow

* Fix CI workflow name

* feat: Add ENS category to search result; Add ENS to check-redirect (#9779)

* feat: Add ENS category to search result; Add ENS to check-redirect

* Fix missed type in search result

* Add @spec and @doc

* chore: (old UI) Replace old Twitter icon with new 'X' (#9641)

"This update involves replacing the old Twitter icon with the new 'X' icon from Font Awesome."

Co-authored-by: bishalghimire1 <80529795+bishalghimire1@users.noreply.github.com>

* fix: Fix incorrect image_url parsing from NFT meta (#9956)

* fix: Fix incorrect image_url parsing from NFT meta

* Add regression test

* fix: Fix WebSocketClient reconnect (#9937)

* Extract Shibarium to a separate CI workflow

* Extract Shibarium to a separate CI workflow

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: carrychair <linghuchong404@gmail.com>
Signed-off-by: one230six <723682061@qq.com>
Signed-off-by: wanshuangcheng <wanshuangcheng@outlook.com>
Signed-off-by: TechVest <techdashen@qq.com>
Co-authored-by: Kirill Fedoseev <kirill@blockscout.com>
Co-authored-by: Victor Baranov <baranov.viktor.27@gmail.com>
Co-authored-by: Nikita Pozdniakov <nikitosing4@mail.ru>
Co-authored-by: nikitosing <32202610+nikitosing@users.noreply.github.com>
Co-authored-by: Qwerty5Uiop <alex000010@bk.ru>
Co-authored-by: POA <33550681+poa@users.noreply.github.com>
Co-authored-by: varasev <33550681+varasev@users.noreply.github.com>
Co-authored-by: Maxim Filonov <53992153+sl1depengwyn@users.noreply.github.com>
Co-authored-by: rlgns98kr <rlgns98kr@gmail.com>
Co-authored-by: Fedor Ivanov <ivnfedor@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: MASDXI <sirawitt42@gmai.com>
Co-authored-by: MASDXI <sirawitt42@gmail.com>
Co-authored-by: zjb0807 <zjb0807@qq.com>
Co-authored-by: GimluCom <79271880+GimluCom@users.noreply.github.com>
Co-authored-by: carrychair <linghuchong404@gmail.com>
Co-authored-by: one230six <723682061@qq.com>
Co-authored-by: Qwerty5Uiop <105209995+Qwerty5Uiop@users.noreply.github.com>
Co-authored-by: Alexander Kolotov <alexandr.kolotov@gmail.com>
Co-authored-by: Snoppy <michaleli@foxmail.com>
Co-authored-by: Mohammed Zeglam <56762707+mohammedzeglam-pg@users.noreply.github.com>
Co-authored-by: William X <24722014+mergd@users.noreply.github.com>
Co-authored-by: wanshuangcheng <166296003+wanshuangcheng@users.noreply.github.com>
Co-authored-by: TechVest <166724172+TechVest@users.noreply.github.com>
Co-authored-by: Thomas Cabrit <thomas@kcalb.org>
Co-authored-by: bishalghimire1 <80529795+bishalghimire1@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants