Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement l2 transactions preprocessing #422

Merged
merged 31 commits into from Jul 20, 2023

Conversation

torztomasz
Copy link
Contributor

Resolves L2B-1752

@linear
Copy link

linear bot commented Jul 11, 2023

L2B-1752 Update preprocessors for l2_transactions

L2 Statistics

For statistics in state update view:

  • to the preprocessed_state_details table add columns
    • l2_transaction_count
    • l2_replaced_transaction_count
    • l2_multitransaction_count
  async preprocessFeederGatewayStatistics(stateUpdate) {
    const lastPreprocessedFeedarGatewayStateUpdateId = 
      "SELECT MAX(state_update_id) FROM preprocessed_state_details WHERE l2_transaction_count IS NOT NULL"

    const lastDownloadedFeedarGatewayStateUpdateId =
      "SELECT MAX(state_update_id) FROM l2_transactions WHERE state_update_id IS NOT NULL"

    for (stateUpdateId = lastPreprocessedFeedarGatewayStateUpdateId;
      stateUpdateId <= min(lastDownloadedFeedarGatewayStateUpdateId, stateUpdate.id);
      i++) {

        const l2TxCount, l2ReplacedTxCount, l2MultiTxCount = 
          L2TransactionRepository.getByStateUpdateId(stateUpdate.id, trx)

        await this.preprocessedStateDetailsRepository.update(
          l2TxCount, ..
        )
      }
    }
  }

For statistics in user view:

  • to preprocessed_user_statistics add columns as above
  • implement similarily as above

For home page: same but for preproceessed_state_update

@torztomasz torztomasz force-pushed the add-l2-transactions-preprocessing branch from dc7c34d to a11cbb0 Compare July 12, 2023 10:34
@torztomasz torztomasz force-pushed the add-l2-transactions-preprocessing branch from a1e21da to 919ff3b Compare July 12, 2023 10:58
@torztomasz torztomasz force-pushed the add-l2-transactions-preprocessing branch from f288df1 to 589c945 Compare July 19, 2023 12:52
@torztomasz torztomasz force-pushed the add-l2-transactions-preprocessing branch from 589c945 to e7ccddd Compare July 19, 2023 12:54
table.string('stark_key').notNullable()
table.jsonb('l2_transactions_statistics').notNullable()
table.jsonb('cumulative_l2_transactions_statistics').notNullable()
table.index(['stark_key', 'state_update_id'])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The order of columns in the index is important. Since we often select by state_update_id only, it should come first, because if it comes second, this index won't be used (unless something changed since I learned about pg indexes :)).

BTW, we still need to spend some time afterwards and make sure we're not missing any indexes, and the ones we have are utilized and don't have too much data.

@torztomasz torztomasz added this pull request to the merge queue Jul 20, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 20, 2023
@torztomasz torztomasz added this pull request to the merge queue Jul 20, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 20, 2023
@torztomasz torztomasz added this pull request to the merge queue Jul 20, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 20, 2023
@torztomasz torztomasz added this pull request to the merge queue Jul 20, 2023
Merged via the queue into master with commit ee9c1ee Jul 20, 2023
4 checks passed
@torztomasz torztomasz deleted the add-l2-transactions-preprocessing branch July 20, 2023 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants