feat: fetch included da height from store #2880
Conversation
|
The latest Buf updates on your PR. Results from workflow CI / buf-check (pull_request).
|
|
Claude finished @julienrbrt's task —— View job Code Review CompleteI've completed a comprehensive review of this PR. The implementation correctly addresses the need to fetch DA height from persistent storage on startup, which improves resilience when the cache is cleared. The changes are well-structured with good test coverage. Below are my findings organized by focus area: 1. Code Quality & Architecture ✅Strengths:
Observations:
2. Error Handling & Resilience
|
Summary of ChangesHello @julienrbrt, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request improves the reliability of the system's Data Availability (DA) height tracking. It ensures that upon startup, the node correctly identifies the highest known DA height by consulting its persistent storage, rather than solely relying on potentially volatile cached values. This change prevents potential synchronization issues and provides a more robust foundation for block processing, alongside standardizing how DA height metadata keys are constructed and accessed across different components. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request refactors the generation of store keys for DA height mappings into dedicated functions within the store package, which improves code consistency and maintainability. It also enhances the syncer's initialization logic to read the highest known DA height from the store, making it more resilient to cache clears on restart. The changes are well-implemented, but I have one suggestion to improve the new getHighestStoredDAHeight function for better robustness and readability.
| func (s *Syncer) getHighestStoredDAHeight() uint64 { | ||
| // Get the DA included height from store | ||
| daIncludedHeightBytes, err := s.store.GetMetadata(s.ctx, store.DAIncludedHeightKey) | ||
| if err != nil || len(daIncludedHeightBytes) != 8 { | ||
| return 0 | ||
| } | ||
| daIncludedHeight := binary.LittleEndian.Uint64(daIncludedHeightBytes) | ||
| if daIncludedHeight == 0 { | ||
| return 0 | ||
| } | ||
|
|
||
| var highestDAHeight uint64 | ||
|
|
||
| // Get header DA height for the last included height | ||
| headerKey := store.GetHeightToDAHeightHeaderKey(daIncludedHeight) | ||
| if headerBytes, err := s.store.GetMetadata(s.ctx, headerKey); err == nil && len(headerBytes) == 8 { | ||
| headerDAHeight := binary.LittleEndian.Uint64(headerBytes) | ||
| highestDAHeight = max(highestDAHeight, headerDAHeight) | ||
| } | ||
|
|
||
| // Get data DA height for the last included height | ||
| dataKey := store.GetHeightToDAHeightDataKey(daIncludedHeight) | ||
| if dataBytes, err := s.store.GetMetadata(s.ctx, dataKey); err == nil && len(dataBytes) == 8 { | ||
| dataDAHeight := binary.LittleEndian.Uint64(dataBytes) | ||
| highestDAHeight = max(highestDAHeight, dataDAHeight) | ||
| } | ||
|
|
||
| return highestDAHeight | ||
| } |
There was a problem hiding this comment.
This function can be improved for robustness and maintainability.
- Error Handling: Errors from
s.store.GetMetadataare silently ignored. It's better to log these errors (unless it's adatastore.ErrNotFound) to help debug potential issues with the data store. This applies to all threeGetMetadatacalls in this function. - Code Duplication: The logic to fetch and decode the header and data DA heights is duplicated. Extracting this into a helper function or a local closure would reduce redundancy.
Here is an example of how it could be refactored:
func (s *Syncer) getHighestStoredDAHeight() uint64 {
daIncludedHeightBytes, err := s.store.GetMetadata(s.ctx, store.DAIncludedHeightKey)
if err != nil {
if !errors.Is(err, datastore.ErrNotFound) {
s.logger.Warn().Err(err).Msg("Failed to get DA included height from store")
}
return 0
}
if len(daIncludedHeightBytes) != 8 {
// Potentially log this as it indicates data corruption
return 0
}
daIncludedHeight := binary.LittleEndian.Uint64(daIncludedHeightBytes)
if daIncludedHeight == 0 {
return 0
}
// Helper to fetch and decode a DA height from a given key.
getDAHeight := func(key string) uint64 {
bz, err := s.store.GetMetadata(s.ctx, key)
if err != nil {
if !errors.Is(err, datastore.ErrNotFound) {
s.logger.Warn().Err(err).Str("key", key).Msg("Failed to get DA height from store")
}
return 0
}
if len(bz) != 8 {
// Potentially log this as it indicates data corruption
return 0
}
return binary.LittleEndian.Uint64(bz)
}
headerDAHeight := getDAHeight(store.GetHeightToDAHeightHeaderKey(daIncludedHeight))
dataDAHeight := getDAHeight(store.GetHeightToDAHeightDataKey(daIncludedHeight))
return max(headerDAHeight, dataDAHeight)
}There was a problem hiding this comment.
it is shorter the way it currently is, and we do want to skip errors and not clutter the logs
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2880 +/- ##
==========================================
- Coverage 64.87% 64.81% -0.06%
==========================================
Files 81 81
Lines 7330 7350 +20
==========================================
+ Hits 4755 4764 +9
- Misses 2034 2045 +11
Partials 541 541
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
* main: chore: remove extra github action yml file (#2882) fix(execution/evm): verify payload status (#2863) feat: fetch included da height from store (#2880) chore: better output on errors (#2879) refactor!: create da client and split cache interface (#2878) chore!: rename `evm-single` and `grpc-single` (#2839) build(deps): Bump golang.org/x/crypto from 0.42.0 to 0.45.0 in /tools/da-debug in the go_modules group across 1 directory (#2876) chore: parallel cache de/serialization (#2868) chore: bump blob size (#2877)
Overview
Fetch once startup da height from store metadata (included da height from inclusion loop).
It is useful when a user cleared the cache.
The cache remains updated and is faster. The cache value == the store metadata value. Except it is cheaper to get.