Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: refactored tx submit checker initialisation #44

Merged
merged 19 commits into from
Nov 17, 2022

Conversation

zavgorodnii
Copy link
Collaborator

@zavgorodnii zavgorodnii commented Oct 10, 2022

This PR refactors the submit checker:

  1. Removes the redundant workers (we can later simply instantiate multiple instances of the submit checker to speed things up);
  2. Adds proper shared storage management (storage is instantiated in the main routine, closed only after all subroutines exit).

HOW TO TEST THIS CODE?

Tun the tx and kv integration tests using the audit fix branches for neutron and neutron-contracts, this branch for the relayer, and the main branch for the integration tests.

@zavgorodnii zavgorodnii marked this pull request as ready for review October 17, 2022 08:04
Copy link
Collaborator

@pr0n00gler pr0n00gler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sort imports, please:

goimports -local github.com/neutron-org -w .

internal/config/config.go Outdated Show resolved Hide resolved
cmd/neutron_query_relayer/main.go Outdated Show resolved Hide resolved
cmd/neutron_query_relayer/main.go Outdated Show resolved Hide resolved
internal/app/app.go Outdated Show resolved Hide resolved
cmd/neutron_query_relayer/main.go Outdated Show resolved Hide resolved
cmd/neutron_query_relayer/main.go Outdated Show resolved Hide resolved
internal/app/app.go Outdated Show resolved Hide resolved
internal/relay/relayer.go Show resolved Hide resolved
Andrei Zavgorodnii added 4 commits October 27, 2022 12:27
func (tc *TxSubmitChecker) Run(ctx context.Context) error {
// we don't want to start jobs before we read all pending txs from database,
// hence we block on this read operation right in the beginning
func (tc *TxSubmitChecker) Run(ctx context.Context, submittedTxsTasksQueue <-chan relay.PendingSubmittedTxInfo) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this is good idea to move submittedTxsTasksQueue into Run(), cause we kinda give idea that you can use this function with multiple queues on the same object. But you're not, cause it is tied to concrete storage that is in TxSubmitChecker struct

Copy link
Collaborator Author

@zavgorodnii zavgorodnii Nov 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure what your point here is; you can, for example, run multiple instances of TxSubmitChecker in separate goroutines to enable parallel processing of pending txs

logger.Fatal("Failed to create NewDefaultStorage", zap.Error(err))
}
defer func(storage relay.Storage) {
if cfg.AllowTxQueries {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now I reailse that this suggestion was a piece of shit #44 (comment): storage is also used by KV processor, so we are not able to have a nil storage on AllowTxQueries==false and therefore should close storage regardless of tx query allowance

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, does that mean we always need storage to be correctly initialised in any workflow? Why wouldn't we just always initialise it then? That will allow us to remove dummy storage implementation completely, and also remove some useless logic around it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exactly, that's the idea

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, i'll revert that

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dummy storage implementation is still present though, don't we need to remove it since we don't use it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

func NewDefaultTxSubmitChecker(cfg config.NeutronQueryRelayerConfig, logRegistry *nlogger.Registry,
storage relay.Storage) (relay.TxSubmitChecker, error) {
neutronClient, err := raw.NewRPCClient(cfg.NeutronChain.RPCAddr, cfg.NeutronChain.Timeout,
logRegistry.Get(TargetChainRPCClientContext))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrongly passed context: should be NeutronChainRPCClientContext

QueryID: queryID,
SubmittedTxHash: hash,
NeutronHash: neutronTxHash,
SubmitTime: time.Now(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this submission time 1) is not correct; 2) is not used by the submit checker. wyt we can do with it? I'd remove it cuz it misleads

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup, i don't see any usages anywhere in the code. removed this field

foxpy
foxpy previously approved these changes Nov 3, 2022
Copy link
Contributor

@foxpy foxpy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks great and all tests are passing. Can you please resolve merge conflicts now?

Copy link
Contributor

@NeverHappened NeverHappened left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I'm very interested in merging this ASAP for my task! (but there are conflicts)

internal/relay/relayer.go Show resolved Hide resolved
Andrei Zavgorodnii added 3 commits November 15, 2022 17:15
# Conflicts:
#	.env.example
#	.env.example.dev
#	cmd/neutron_query_relayer/main.go
#	internal/app/app.go
#	internal/relay/relayer.go
#	internal/txprocessor/txprocessor.go
sotnikov-s
sotnikov-s previously approved these changes Nov 17, 2022
Copy link
Contributor

@sotnikov-s sotnikov-s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests pass. lgtm

pr0n00gler
pr0n00gler previously approved these changes Nov 17, 2022
Copy link
Collaborator

@pr0n00gler pr0n00gler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, untested

Copy link

@oldremez oldremez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests pass

internal/config/config.go Outdated Show resolved Hide resolved
@zavgorodnii zavgorodnii merged commit 8dc793c into main Nov 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants