Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: parallelize ensuring active pools on startup #1525

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

EnriqueL8
Copy link
Contributor

When a large number of token pools are active and namespace is restarted, it take a long time to activate one by one each token pool. This PR spins a new go routine for each API call.

Signed-off-by: Enrique Lacal <enrique.lacal@kaleido.io>
@EnriqueL8 EnriqueL8 requested a review from a team as a code owner June 17, 2024 15:29
// by one of the goroutines.
if err := g.Wait(); err != nil {
// The above handleMessage will retry if errors occur
return err
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a change in behavior - the previous ft.EnsureTokenPoolActive(ctx, currentPool) code did not crash the namespace start in the case it failed to activate a token pool.

It might be a positive change in behavior, but it would need some analysis/thinking (probably from @awrichar?) as to why the previous code intentionally did not fail on this condition before.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I thought about that, it felt weird to me that it didn't fail on startup... Would love to hear @awrichar thoughts on this one

@@ -716,16 +717,31 @@ func (ft *FFTokens) handleNamespaceStarted(ctx context.Context, data fftypes.JSO
// Make sure any pools that are marked as active in our DB are indeed active
namespace := data.GetString("namespace")
log.L(ctx).Debugf("Token connector '%s' started namespace '%s'. Ensuring all token pools active.", ft.Name(), namespace)

g, ctx := errgroup.WithContext(ctx)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like using `SetLimit() on the concurrency would be better than a completely unbounded parallelism against the token connector:
https://pkg.go.dev/golang.org/x/sync/errgroup#Group.SetLimit

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea - will set that to the number of pools to activate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants