Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EN-7433: storer to elastic - fix miniblock not found #2260

Merged

Conversation

bogdan-rosianu
Copy link
Contributor

  • fixed a bug where a miniblock was not found because it was stored in the previous epoch's storage.
  • removed the chain ID path from config/flag and fetched it from genesis nodes setup
  • skip indexing intra-shard smart contract results (excepting metachain intra-shard)
  • replaced the indexing with go routines with the serial one, because due to too many requests, some data wouldn't have been indexed.

Testing procedure is the same as for #2249

@bogdan-rosianu bogdan-rosianu added the type:bug Something isn't working label Aug 31, 2020
@bogdan-rosianu bogdan-rosianu self-assigned this Aug 31, 2020
@iulianpascalau iulianpascalau self-requested a review August 31, 2020 12:59
@@ -156,16 +156,18 @@ func (dp *dataProcessor) indexData(data *storer2ElasticData.HeaderData) error {
notarizedHeaders := dp.computeNotarizedHeaders(data.Header)
newBody := &block.Body{MiniBlocks: make([]*block.MiniBlock, 0)}
for _, mb := range data.Body.MiniBlocks {
if mb.Type == block.ReceiptBlock { // don't index receipt miniblocks
if mb.Type == block.ReceiptBlock ||
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe extract into a variable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@bogdan-rosianu bogdan-rosianu marked this pull request as ready for review August 31, 2020 13:51
@@ -81,7 +81,9 @@ func (dr *databaseReader) GetDatabaseInfo() ([]*DatabaseInfo, error) {
epochStr := re.FindString(dirname)
epoch, errParseInt := strconv.ParseInt(epochStr, 10, 64)
if errParseInt != nil {
log.Warn("cannot parse epoch number from directory name", "directory name", dirname)
if dirname != "Static" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

magic string?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done. also in other places

iulianpascalau
iulianpascalau previously approved these changes Aug 31, 2020
sasurobert
sasurobert previously approved these changes Aug 31, 2020
Copy link
Contributor

@LucianMincu LucianMincu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@LucianMincu LucianMincu merged commit eaf7352 into development Sep 5, 2020
@LucianMincu LucianMincu deleted the EN-7433-storer2elastic-fix-miniblock-prev-epoch branch September 5, 2020 10:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants