Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Hyperblock coordinates in GetTxByHash() #2226

Merged
merged 38 commits into from
Aug 23, 2020

Conversation

andreibancioiu
Copy link
Collaborator

@andreibancioiu andreibancioiu commented Aug 17, 2020

  • store extra fields in the miniblock metadata storage (used for API). These extra fields refer to the metablock that included the miniblock in question at source / at destination.
  • return these extra fields on the API (get transaction)
  • the fields NotarizedAtDestinationInMeta* are used to populate the hyperblock coordinates at proxy-side.
  • fix status for invalid transactions (within miniblock of type invalidBlock)
  • refactoring on the API indexing aka full history feature.

How to test:

  • send a transaction
  • get transaction by hash, /transaction/abracadabra, and inspect the fields:
NotarizedAtSourceInMetaNonce
NotarizedAtSourceInMetaHash
NotarizedAtDestinationInMetaNonce
NotarizedAtDestinationInMetaHash
  • run the test for several different epochs.
  • look for reward transactions as well

Limitations:

cannot populate NotarizedAtSourceInMetaNonce / NotarizedAtSourceInMetaHash when running in destination shard if the notification of "notarized at source" comes before committing the block (at destination) - since there will be no miniblock metadata in the storer yet. Can be easily fixed though. Fixed.

@andreibancioiu andreibancioiu self-assigned this Aug 17, 2020
@andreibancioiu andreibancioiu marked this pull request as ready for review August 17, 2020 15:23
@andreibancioiu andreibancioiu changed the title [WIP] Add Hyperblock coordinates in GetTxByHash() Add Hyperblock coordinates in GetTxByHash() Aug 17, 2020
@sasurobert sasurobert self-requested a review August 17, 2020 15:29
Capacity = 20000
Type = "LRU"
[FullHistory.HashEpochStorageConfig.DB]
[FullHistory.MiniblockHashByTxHashStorageConfig.DB]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if we use the INDEXER interface in order to save all kinds of mapping like this? The node would work the same with the same storage and data, but the saved transaction structure in case of indexer would have all this metadata ? Not for current PR, but for the long term. food for taught.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, as discussed, we have to do that on the long term.

}

func newErrCannotSaveMiniblockMetadata(hash []byte, originalErr error) error {
return fmt.Errorf("cannot save miniblock metadata, hash [%s]: %w", hex.EncodeToString(hash), originalErr)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you really need this errors ? returning original err is not enough ? keep it simple

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to have some correlation info in case there's an issue. Can we keep them for the moment?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is really low possibility for this to happen

@@ -56,11 +56,12 @@ func (hpf *historyRepositoryFactory) Create() (fullHistory.HistoryRepository, er
}

historyRepArgs := fullHistory.HistoryRepositoryArguments{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is fullHistory in the core package ? it should not be here. core is only for simple stuff.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. Will be moved in a future PR.

if err != nil {
return fmt.Errorf("cannot save in storage history transaction %w", err)
return newErrCannotSaveMiniblockMetadata(hash, err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just return a constant error here - no need to add hash. this will never happen. keep it simple.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to have some correlation info in case there's an issue. Can we keep it for the moment?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i do not see this usable - it is pretty impossible for this to happen.

return hp.epochByHashIndex.getEpochByHash(hash)
}

func (hp *historyProcessor) RegisterToBlockTracker(blockTracker BlockTracker) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add comment to public function

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

metaBlock, ok := header.(*block.MetaBlock)
if !ok {
// Question for review: why, on every round, there's a header of type = *block.Header (even if shardID = metachain)?
log.Debug("onNotarizedBlocks(): cannot convert to *block.Metablock", "type", fmt.Sprintf("%T", header))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete this log - no need for it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

return
}

for i := 0; i < len(headers); i++ {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for i, header := range headers

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

return
}

log.Trace("onNotarizedBlocks()", "index", i, "len(ShardInfo)", len(metaBlock.ShardInfo))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we really need these log trace's ? if they will not be used, delete them.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.

}

func (hp *historyProcessor) onNotarizedBlock(metaBlockNonce uint64, metaBlockHash []byte, blockHeader block.ShardData) {
log.Trace("onNotarizedBlock()",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logging this data already exist - please delete

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.

storer storage.Storer
}

func newMiniblockHashByTxHashIndex(storer storage.Storer) *miniblockHashByTxHashIndex {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is no need for this class. this in only a storer. Add a named variable which says miniblockByHashIndexStorer and you are fine.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, removed.

AdoAdoAdo
AdoAdoAdo previously approved these changes Aug 20, 2020
sasurobert
sasurobert previously approved these changes Aug 20, 2020
sasurobert
sasurobert previously approved these changes Aug 20, 2020
AdoAdoAdo
AdoAdoAdo previously approved these changes Aug 20, 2020
Copy link
Contributor

@LucianMincu LucianMincu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

System tests passed

@LucianMincu LucianMincu merged commit 24ab55f into development Aug 23, 2020
@LucianMincu LucianMincu deleted the hyperblock-coordinates branch August 23, 2020 16:50
@andreibancioiu andreibancioiu restored the hyperblock-coordinates branch August 23, 2020 19:54
@andreibancioiu andreibancioiu deleted the hyperblock-coordinates branch August 26, 2020 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants