Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

domain: Clean up the goroutine after closing a domain #5163

Merged
merged 5 commits into from
Nov 22, 2017

Conversation

zimulala
Copy link
Contributor

If we close the domain, maybe the channel of syncer.Done and do.exit are both notified.
If it selects the channel of syncer.Done and the NewSession function encounters an error like grpc: the client connection is closing(the error isn’t the context done error), then the goroutine can't quit. So this domain will leave some dirty data after quitting.

@zimulala
Copy link
Contributor Author

/run-all-tests

domain/domain.go Outdated
@@ -317,6 +317,7 @@ func (do *Domain) Reload() error {
}

func (do *Domain) loadSchemaInLoop(lease time.Duration) {
defer do.wg.Done()
// Lease renewal can run at any frequency.
// Use lease/2 here as recommend by paper.
// TODO: Reset ticker or make interval longer.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the TODO.

@ngaut
Copy link
Member

ngaut commented Nov 21, 2017

PTAL @siddontang

domain/domain.go Outdated
if err == nil {
return nil
}
// If the domain exits, we return this function.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/exits/has stopped/

domain/domain.go Outdated
if err == nil {
return nil
}
// If the domain exits, we return this function.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/this function/error immediately/

@zimulala
Copy link
Contributor Author

PTAL @tiancaiamao @coocood

@@ -474,6 +496,7 @@ func NewDomain(store kv.Storage, ddlLease time.Duration, statsLease time.Duratio
// Only when the store is local that the lease value is 0.
// If the store is local, it doesn't need loadSchemaInLoop.
if ddlLease > 0 {
d.wg.Add(1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why need to wait for this rountine exit?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normally the structure stops, then the gorountine will exit.

@coocood
Copy link
Member

coocood commented Nov 21, 2017

@zimulala
The syncer.Restart use an unlimited retry count` can we just use limited retry count instead?

@tiancaiamao
Copy link
Contributor

SchemaSyncer belongs to ddl object, how about move it's logic to other place, out of loadSchemaInLoop?

When domain.Close() is called, do.ddl.Stop() should already handle all things, hence we won't suffer the problem here. @zimulala

@zimulala
Copy link
Contributor Author

@coocood
The syncer.Restart use a context to handle when need to stop.
If we use a limited retry count, we also need mustRestartSyncer function. The different is that one uses the context with timeout,another uses a limited retry count.

@zimulala
Copy link
Contributor Author

@tiancaiamao
But SchemaSyncer is used to sync schema info, and SchemaValidator also in the domain.


for {
childCtx, cancel := goctx.WithTimeout(ctx, timeout)
err := syncer.Restart(childCtx)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we log this error?

domain/domain.go Outdated
@@ -352,6 +352,28 @@ func (do *Domain) loadSchemaInLoop(lease time.Duration) {
}
}

func (do *Domain) mustRestartSyncer() error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add comments for this method.

@zimulala
Copy link
Contributor Author

PTAL @coocood @winkyao @tiancaiamao

Copy link
Contributor

@winkyao winkyao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@coocood
Copy link
Member

coocood commented Nov 22, 2017

LGTM

@coocood coocood merged commit 9c36002 into pingcap:master Nov 22, 2017
@zz-jason zz-jason added the status/LGT2 Indicates that a PR has LGTM 2. label Nov 22, 2017
@zimulala zimulala deleted the restart-syncer branch December 16, 2017 04:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants