Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

op-batcher,op-node,batch_decoder: add logging of compression algo #10589

Merged
merged 6 commits into from
May 22, 2024

Conversation

sebastianst
Copy link
Member

Description

  • Adds logging of the used compression algo to the op-batcher and op-node.
    • To get the compression algo info to the right places in op-node, I added a Batch wrapper type batchWithMetadata that appends the algo to the log context.
  • Adds the algo to the batch decoder tool's output data.
  • Fixing some naming around compression algos.
  • Adds brotli as a valid default selector for brotli-10 compression

Tests

Added the default brotli compression algo string to tests. Remaining changes are logging (or the untested batch_decoder)

Additional context

There's currently no way to know from batcher or node logs what compression algo is actually in use for (de)compression.

Copy link
Contributor

coderabbitai bot commented May 20, 2024

Walkthrough

Walkthrough

The changes primarily focus on updating the handling of compression algorithms across various components. Key modifications include replacing CompressionAlgoTypes with CompressionAlgos, adding logging for compression settings, and introducing new fields and methods to manage compression algorithms more effectively. Additionally, the changes enhance error handling, improve test cases, and streamline batch processing by encapsulating metadata.

Changes

Files Change Summaries
op-batcher/batcher/channel_builder_test.go, op-batcher/batcher/config.go, ... Replaced CompressionAlgoTypes with CompressionAlgos in loops, function parameters, and usage messages.
op-batcher/batcher/channel_manager.go, op-batcher/batcher/service.go Added logging for compression_algo and target_num_frames in ensureChannelWithSpace and initChannelConfig.
op-node/cmd/batch_decoder/reassemble/reassemble.go Added ComprAlgos field to ChannelWithMetadata struct and updated processFrames to populate this field.
op-node/rollup/derive/batch.go Added ComprAlgo field to BatchData struct and introduced batchWithMetadata type for enhanced logging.
op-node/rollup/derive/batch_queue.go, op-node/rollup/derive/batches.go Refactored type assertions to use AsSingularBatch and AsSpanBatch methods, improved error handling.
op-node/rollup/derive/channel.go, op-node/rollup/derive/channel_in_reader.go Introduced comprAlgo variable and updated batch processing to use batchWithMetadata struct.
op-node/rollup/derive/types.go, op-node/rollup/derive/types_test.go Added new Brotli compression algorithm, updated regex for brotli, and adjusted related functions and tests.
op-e2e/e2eutils/geth/wait.go Added strings import, updated error handling in WaitForTransaction function.

Recent Review Details

Configuration used: .coderabbit.yml
Review profile: CHILL

Commits Files that changed from the base of the PR and between 88e9a09 and 0e18f4b.
Files selected for processing (1)
  • op-node/rollup/derive/channel_test.go (3 hunks)
Additional comments not posted (2)
op-node/rollup/derive/channel_test.go (2)

58-95: The adjustments to the test cases in TestFrameValidity effectively address frame duplication and closing frame issues.


Line range hint 156-233: The test cases in TestBatchReader are well-structured and effectively test the handling of different compression algorithms in various scenarios. Good use of require for assertions enhances the clarity and reliability of the tests.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

semgrep-app bot commented May 20, 2024

Semgrep found 4 ban-common-hex2bytes findings:

Found banned use of common.Hex2Bytes. Use common.FromHex instead.

Ignore this finding from ban-common-hex2bytes.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

op-batcher/batcher/service.go Outdated Show resolved Hide resolved
op-e2e/e2eutils/geth/wait.go Outdated Show resolved Hide resolved
Copy link
Contributor

@trianglesphere trianglesphere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

derivation + logging changes LGTM. Just some Q's about the re-assemble tool

@sebastianst sebastianst force-pushed the seb/batcher-compr-log branch 2 times, most recently from 84a4a14 to 654f269 Compare May 21, 2024 17:44
@sebastianst
Copy link
Member Author

I removed the op-e2e receipt confirmation change, will handle in a separate PR.

Since we're dealing now with wrapped types around Batch implementations,
this let's us transparently unwrap the underlying batch types. It makes
sense to add this to the interface, because getting the underlying types
from the interface is done in several places, so it's part of the
interface's contract.
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Outside diff range and nitpick comments (2)
op-node/rollup/derive/batch_queue.go (1)

Line range hint 180-201: Refactor to improve readability and error handling in batch type processing.

- switch typ := batch.GetBatchType(); typ {
- case SingularBatchType:
-   singularBatch, ok := batch.AsSingularBatch()
-   if !ok {
-     return nil, false, NewCriticalError(errors.New("failed type assertion to SingularBatch"))
-   }
-   nextBatch = singularBatch
- case SpanBatchType:
-   spanBatch, ok := batch.AsSpanBatch()
-   if !ok {
-     return nil, false, NewCriticalError(errors.New("failed type assertion to SpanBatch"))
-   }
-   singularBatches, err := spanBatch.GetSingularBatches(bq.l1Blocks, parent)
-   if err != nil {
-     return nil, false, NewCriticalError(err)
-   }
-   bq.nextSpan = singularBatches
-   nextBatch = bq.popNextBatch(parent)
- default:
-   return nil, false, NewCriticalError(fmt.Errorf("unrecognized batch type: %d", typ))
- }
+ if nextBatch, err := processBatchByType(batch, bq, parent); err != nil {
+   return nil, false, err
+ }
op-node/rollup/derive/span_batch.go (1)

425-427: Ensure consistency in method documentation.

The methods AsSingularBatch and AsSpanBatch lack documentation. It's a good practice to provide comments explaining the purpose and behavior of public methods, especially in a complex domain like blockchain.

op-node/rollup/derive/batches.go Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

op-node/rollup/derive/channel_test.go Show resolved Hide resolved
@sebastianst sebastianst added this pull request to the merge queue May 22, 2024
Merged via the queue into develop with commit 01d3a17 May 22, 2024
69 checks passed
@sebastianst sebastianst deleted the seb/batcher-compr-log branch May 22, 2024 21:31
tarunkhasnavis pushed a commit that referenced this pull request May 28, 2024
…0589)

* op-batcher: add logging of compression algo

* add node logging of algo, add brotli default algo

* fix typos

* only log compression algo if present

* add type conversion abstraction to Batch interface

Since we're dealing now with wrapped types around Batch implementations,
this let's us transparently unwrap the underlying batch types. It makes
sense to add this to the interface, because getting the underlying types
from the interface is done in several places, so it's part of the
interface's contract.

* adapt BatchReader test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants