-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adds actors healthz integration tests #7394
Adds actors healthz integration tests #7394
Conversation
Mark actors as always healthy when there is no app channel or their have been no actor entities (actor types) provided by the application config endpoint. This is important for enabling actor related APIs such as workflows, which don't rely on the application implementing actors itself. Signed-off-by: joshvanl <me@joshvanl.dev>
Most importantly, ensuring that actors are marked as always healthy when the both the application healthz endpoint is disabled, and the application reports no actor entities. Signed-off-by: joshvanl <me@joshvanl.dev>
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #7394 +/- ##
==========================================
+ Coverage 62.32% 62.34% +0.02%
==========================================
Files 240 240
Lines 22154 22154
==========================================
+ Hits 13807 13812 +5
+ Misses 7190 7182 -8
- Partials 1157 1160 +3 ☔ View full report in Codecov by Sentry. |
srvNoHealthz := prochttp.New(t, | ||
prochttp.WithHandlerFunc("/dapr/config", func(w http.ResponseWriter, r *http.Request) { | ||
w.Write([]byte(`{}`)) | ||
}), | ||
prochttp.WithHandlerFunc("/healthz", func(w http.ResponseWriter, r *http.Request) { | ||
w.WriteHeader(http.StatusOK) | ||
onceNoHealthz.Do(func() { | ||
close(n.noHealthzCalled) | ||
}) | ||
}), | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
qq when there are no actor types and health check is disabled, why is a healthz endpoint needed and is being called?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We are verifying this doesn't get called further down.
@JoshVanL The IT seem to failing in the specific test that was written in this PR. |
Thanks @mukundansundar, it's clear this PR is causing some kind of unexpected side effect somewhere. I'm taking a look |
pkg/actors/placement/placement.go
Outdated
@@ -180,6 +180,7 @@ func (p *actorPlacement) Start(ctx context.Context) error { | |||
defer p.shutdownConnLoop.Done() | |||
ch := p.appHealthFn(ctx) | |||
if ch == nil { | |||
p.appHealthy.Store(true) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems a bit off. I don't think we can do this because if the app has a HTTP app channel, ch
will be non-nil.
Can you check pkg/actors/actors.go
? Specifically, where a.checker
is initialized:
Perhaps the way to fix this issue is to simply not initialize it if len(hat) == 0
?
if len(hat) > 0 {
a.checker, err = a.getAppHealthChecker()
if err != nil {
return fmt.Errorf("actors: couldn't create health check: %w", err)
}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it is OK because ch will be nil
if entities is empty
Lines 317 to 319 in ecd3798
if len(a.actorsConfig.Config.HostedActorTypes.ListActorTypes()) == 0 || a.appChannel == nil { | |
return nil, nil | |
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should be covering this scenario in the noentities
test.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hold on, then if ch
is already nil
when entities is empty.... There's nothing we need to do in this PR? Was the issue already fixed? (And just needed tests?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When we start placement, we already set appHealthy to true:
And if ch
is nil, then this can never be set to false
(And if that weren't the case, workflow tests would have failed already sinced they don't implement /healthz)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Without the code change made in this PR, the new test noentities
will fail. i.e. actors will never become ready.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It works on my machine without this line. Please try it!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@JoshVanL Please, double check this because we already have E2E to validate that sidecars that don't serve actors types can still invoke other actors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
BTW, happy to take the integration test part of it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done! I have just included the integration tests now. I have also updated the PR title and description.
This is a regression because a sidecar that does not host actors can always use actors that are hosted by other instances. If this is the case, this is a hotfix. |
tests/integration/suite/actors/healthz/endpoint/noappentities.go
Outdated
Show resolved
Hide resolved
Signed-off-by: joshvanl <me@joshvanl.dev>
Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: Artur Souza <asouza.pro@gmail.com>
Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: Artur Souza <asouza.pro@gmail.com>
* Mark actors healthy on no app channel or actor entities Mark actors as always healthy when there is no app channel or their have been no actor entities (actor types) provided by the application config endpoint. This is important for enabling actor related APIs such as workflows, which don't rely on the application implementing actors itself. Signed-off-by: joshvanl <me@joshvanl.dev> * Adds integration tests for actor healthz endpoints Most importantly, ensuring that actors are marked as always healthy when the both the application healthz endpoint is disabled, and the application reports no actor entities. Signed-off-by: joshvanl <me@joshvanl.dev> * Remove uneeded app health check store Signed-off-by: joshvanl <me@joshvanl.dev> * Update tests/integration/suite/actors/healthz/endpoint/noapp.go Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: Artur Souza <asouza.pro@gmail.com> * Update tests/integration/suite/actors/healthz/endpoint/noappentities.go Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: Artur Souza <asouza.pro@gmail.com> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Signed-off-by: Artur Souza <asouza.pro@gmail.com> Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
* Mark actors healthy on no app channel or actor entities Mark actors as always healthy when there is no app channel or their have been no actor entities (actor types) provided by the application config endpoint. This is important for enabling actor related APIs such as workflows, which don't rely on the application implementing actors itself. Signed-off-by: joshvanl <me@joshvanl.dev> * Adds integration tests for actor healthz endpoints Most importantly, ensuring that actors are marked as always healthy when the both the application healthz endpoint is disabled, and the application reports no actor entities. Signed-off-by: joshvanl <me@joshvanl.dev> * Remove uneeded app health check store Signed-off-by: joshvanl <me@joshvanl.dev> * Update tests/integration/suite/actors/healthz/endpoint/noapp.go Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: Artur Souza <asouza.pro@gmail.com> * Update tests/integration/suite/actors/healthz/endpoint/noappentities.go Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: Artur Souza <asouza.pro@gmail.com> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Signed-off-by: Artur Souza <asouza.pro@gmail.com> Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
This reverts commit 0b61910.
* Mark actors healthy on no app channel or actor entities Mark actors as always healthy when there is no app channel or their have been no actor entities (actor types) provided by the application config endpoint. This is important for enabling actor related APIs such as workflows, which don't rely on the application implementing actors itself. Signed-off-by: joshvanl <me@joshvanl.dev> * Adds integration tests for actor healthz endpoints Most importantly, ensuring that actors are marked as always healthy when the both the application healthz endpoint is disabled, and the application reports no actor entities. Signed-off-by: joshvanl <me@joshvanl.dev> * Remove uneeded app health check store Signed-off-by: joshvanl <me@joshvanl.dev> * Update tests/integration/suite/actors/healthz/endpoint/noapp.go Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: Artur Souza <asouza.pro@gmail.com> * Update tests/integration/suite/actors/healthz/endpoint/noappentities.go Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: Artur Souza <asouza.pro@gmail.com> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Signed-off-by: Artur Souza <asouza.pro@gmail.com> Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> Signed-off-by: Elena Kolevska <elena@kolevska.com>
This reverts commit 0b61910. Signed-off-by: Elena Kolevska <elena@kolevska.com>
* Mark actors healthy on no app channel or actor entities Mark actors as always healthy when there is no app channel or their have been no actor entities (actor types) provided by the application config endpoint. This is important for enabling actor related APIs such as workflows, which don't rely on the application implementing actors itself. Signed-off-by: joshvanl <me@joshvanl.dev> * Adds integration tests for actor healthz endpoints Most importantly, ensuring that actors are marked as always healthy when the both the application healthz endpoint is disabled, and the application reports no actor entities. Signed-off-by: joshvanl <me@joshvanl.dev> * Remove uneeded app health check store Signed-off-by: joshvanl <me@joshvanl.dev> * Update tests/integration/suite/actors/healthz/endpoint/noapp.go Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: Artur Souza <asouza.pro@gmail.com> * Update tests/integration/suite/actors/healthz/endpoint/noappentities.go Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: Artur Souza <asouza.pro@gmail.com> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Signed-off-by: Artur Souza <asouza.pro@gmail.com> Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
This reverts commit 80c746a.
* Mark actors healthy on no app channel or actor entities Mark actors as always healthy when there is no app channel or their have been no actor entities (actor types) provided by the application config endpoint. This is important for enabling actor related APIs such as workflows, which don't rely on the application implementing actors itself. Signed-off-by: joshvanl <me@joshvanl.dev> * Adds integration tests for actor healthz endpoints Most importantly, ensuring that actors are marked as always healthy when the both the application healthz endpoint is disabled, and the application reports no actor entities. Signed-off-by: joshvanl <me@joshvanl.dev> * Remove uneeded app health check store Signed-off-by: joshvanl <me@joshvanl.dev> * Update tests/integration/suite/actors/healthz/endpoint/noapp.go Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: Artur Souza <asouza.pro@gmail.com> * Update tests/integration/suite/actors/healthz/endpoint/noappentities.go Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: Artur Souza <asouza.pro@gmail.com> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Signed-off-by: Artur Souza <asouza.pro@gmail.com> Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
This reverts commit 8ff62ac.
* Mark actors healthy on no app channel or actor entities Mark actors as always healthy when there is no app channel or their have been no actor entities (actor types) provided by the application config endpoint. This is important for enabling actor related APIs such as workflows, which don't rely on the application implementing actors itself. Signed-off-by: joshvanl <me@joshvanl.dev> * Adds integration tests for actor healthz endpoints Most importantly, ensuring that actors are marked as always healthy when the both the application healthz endpoint is disabled, and the application reports no actor entities. Signed-off-by: joshvanl <me@joshvanl.dev> * Remove uneeded app health check store Signed-off-by: joshvanl <me@joshvanl.dev> * Update tests/integration/suite/actors/healthz/endpoint/noapp.go Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: Artur Souza <asouza.pro@gmail.com> * Update tests/integration/suite/actors/healthz/endpoint/noappentities.go Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: Artur Souza <asouza.pro@gmail.com> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Signed-off-by: Artur Souza <asouza.pro@gmail.com> Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> Signed-off-by: Cassandra Coyle <cassie@diagrid.io>
Adds integration tests for testing actor healthz behaviour when entities and or app healthz is defined.
Closes #7355
Mark actors as always healthy when there is no app channel or their have beenno actor entities (actor types) provided by the application config endpoint.This is important for enabling actor related APIs such as workflows, whichdon't rely on the application implementing actors itself.Adds integration tests for actor healthz endpoints Most importantly, ensuringthat actors are marked as always healthy when the both the application healthzendpoint is disabled, and the application reports no actor entities.