On Fleet Server reboot gives a grace period #1605
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ph
added
the
Team:Elastic-Agent-Control-Plane
ph
requested review from
aleksmaus,
narph and
joshdover
and removed request for
a team
|
@joshdover I think this will help api key issues we saw. |
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
💚 Flaky test reportTests succeeded. 🤖 GitHub commentsTo re-run your PR in the CI, just comment with:
|
|
@Mergifyio update |
✅ Branch has been successfully updated |
|
@ph i have relabelled the issue appropriately to make it land in 8.4. |
|
Test is fixed and it's ready for review. |
aleksmaus
reviewed
Jun 30, 2022
internal/pkg/coordinator/monitor.go
Outdated
| @@ -453,6 +453,23 @@ func runCoordinatorOutput(ctx context.Context, cord Coordinator, bulker bulk.Bul | |||
| } | |||
|
|
|||
| func runUnenroller(ctx context.Context, bulker bulk.Bulk, policyID string, unenrollTimeout time.Duration, l zerolog.Logger, checkInterval time.Duration, agentsIndex string) { | |||
| // When fleet-server is offline for a long period and finally recover, it means that the connected | |||
When Fleet Server is offline for a period bigger or equal to the unenrollTimeout, on reboot the Server will start to automatically unenroll Elastic Agent without giving them a change to communicate with the system. Instead on reboot we will give a grace period equivalent to the unenrollTimeout this will give enough time to the Elastic Agent to communicate back to Fleet Server and update their last checkin time. Fixes: elastic#1500
Consider the grace period when executing the test.
aleksmaus
reviewed
Jun 30, 2022
aleksmaus
reviewed
Jun 30, 2022
| Msg("giving a grace period to Elastic Agent before enforcing unenrollTimeout monitor") | ||
|
|
||
| if err := waitWithContext(ctx, unenrollTimeout); err != nil { | ||
| l.Err(err).Dur("unenroll_timeout", unenrollTimeout). |
There was a problem hiding this comment.
nit: maybe log "context canceled" at debug level?
ph
added a commit
that referenced
this pull request
Jun 30, 2022
[8.3](backport #1605) On Fleet Server reboot gives a grace period
7 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When Fleet Server is offline for a period bigger or equal to the
unenrollTimeout, on reboot the Server will start to automatically
unenroll Elastic Agent without giving them a chance to communicate with
the system. Instead on reboot we will give a grace period equivalent to
the unenrollTimeout this will give enough time to the Elastic Agent to
communicate back to Fleet Server and update their last checkin time.
Fixes: #1500
What is the problem this PR solves?
// Please do not just reference an issue. Explain WHAT the problem this PR solves here.
How does this PR solve the problem?
// Explain HOW you solved the problem in your code. It is possible that during PR reviews this changes and then this section should be updated.
How to test this PR locally
Checklist
CHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.Related issues