Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wip] - do-not-merge - Add test PlacementTableVersionNotEventuallyIncreased #7571

Open
wants to merge 106 commits into
base: master
Choose a base branch
from

Conversation

fowlerlee
Copy link

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved.

Description

Add a new test called PlacementTableVersionNotEventuallyIncreased

Signed-off-by: Lee Fowler fowler.lee8@gmail.com

Issue reference

Issue #7490_#7490 (comment)

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

ItalyPaleAle and others added 12 commits February 27, 2024 21:58
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Lee Fowler <fowler.lee8@gmail.com>
Signed-off-by: Lee Fowler <fowler.lee8@gmail.com>
…lerlee/dapr into fowlerlee-7490-placement-api
@fowlerlee fowlerlee requested review from a team as code owners February 29, 2024 09:18
@fowlerlee fowlerlee marked this pull request as draft February 29, 2024 09:18
gspadotto and others added 3 commits March 4, 2024 09:38
Signed-off-by: Guido Spadotto <guido.spadotto@profesia.it>
Co-authored-by: Guido Spadotto <guido.spadotto@profesia.it>
kiterrors "github.com/dapr/kit/errors"
)

func PlacementTableVersionNotEventuallyIncreased(tableVersion int, msg string) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey @fowlerlee - so you'll want to call this func in place of where the relevant err is thrown. Here is a direct example of me doing that for the pubsub api, but youll need to find where this error is thrown in dapr and do the same thing. You'll want to pass in all relevant err details like maybe the appID (which you can typically find off the universal struct) and any other metadata that may be relevant to a user. Specifically, for the kiterrors code - you'll need to open a PR to dapr/kit to add the placement kiterrors.CodePrefixPlacement similar to this PR. Then you'll want to use that kit error prefix for the .WithErrorInfo().

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cicoyle : I added the error handling to this PR, and fixed some typos I had that were not conforming to the format tests. The issue still requires integration tests, but I'm not certain the form that these should take, since many already exist? I also 'git rebase' to get the master into the feature branch, hence the many commits from the master to this issue PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mikeee and others added 11 commits March 5, 2024 22:15
* ci: add 'mikeee' to dapr-bot owners

Signed-off-by: mikeee <hey@mike.ee>

* fix: alphabetise list of bot owner gh handles

Signed-off-by: mikeee <hey@mike.ee>

---------

Signed-off-by: mikeee <hey@mike.ee>
* Make injector resilient to sentry unavailability (dapr#7507)

* make injector resilient to sentry unavailability

Signed-off-by: yaron2 <schneider.yaron@live.com>

* remove redundant line

Signed-off-by: yaron2 <schneider.yaron@live.com>

---------

Signed-off-by: yaron2 <schneider.yaron@live.com>

* sentry retry up to 30s (dapr#7508)

Signed-off-by: yaron2 <schneider.yaron@live.com>

* Update contrib to 1.13.0-rc.3 (dapr#7509)

* update contrib to 1.13.0-rc.2

Signed-off-by: yaron2 <schneider.yaron@live.com>

* update to rc.3

Signed-off-by: yaron2 <schneider.yaron@live.com>

---------

Signed-off-by: yaron2 <schneider.yaron@live.com>

* Injector: add option to add `DAPR_HOST_IP` env var to daprd (dapr#7511)

The `DAPR_HOST_IP` env var is used in various places in Dapr for a sidecar to know its own IP address, for example for service invocation or actor invocation.

When using the Dapr injector to add the daprd container, we can use the downstream APIs to add the `DAPR_HOST_IP` env var based on data from the controller

This option can be enabled by setting the Helm option `dapr_sidecar_injector.enableK8sDownwardAPIs=true`

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fix state encryption regression + add integration test (dapr#7517)

* fix state encryption regression + add intg test

Signed-off-by: yaron2 <schneider.yaron@live.com>

* linter

Signed-off-by: yaron2 <schneider.yaron@live.com>

* review feedback

Signed-off-by: yaron2 <schneider.yaron@live.com>

* linter

Signed-off-by: yaron2 <schneider.yaron@live.com>

* just use sha256, remove nolint:gosec

Signed-off-by: yaron2 <schneider.yaron@live.com>

---------

Signed-off-by: yaron2 <schneider.yaron@live.com>

* Test Integration: speed up tests 10% (dapr#7528)

* Test Integration: speed up tests 10%

Speed up integration tests by changing poll intervals
`100*time.Millisecond` to `10*time.Millisecond`.

~4.50m to ~4.20m

Signed-off-by: joshvanl <me@joshvanl.dev>

* Wait for operator healthz before exiting to ensure no exit error

Signed-off-by: joshvanl <me@joshvanl.dev>

---------

Signed-off-by: joshvanl <me@joshvanl.dev>

* Revert selfhosted disk loader to not respect namespace (dapr#7527)

* Revert selfhosted disk loader to not respect namespace

Revert the selfhosted component disk loader to not respect the
namespace. This will allow the selfhosted disk loader to load components
from any namespace from file.

Fixes dapr#7523

Signed-off-by: joshvanl <me@joshvanl.dev>

* Revert component disk loader behaviour to respect component namespace
when NAMESPACE env var is set.

Signed-off-by: joshvanl <me@joshvanl.dev>

---------

Signed-off-by: joshvanl <me@joshvanl.dev>

* [1.13] Add warning that Dapr state store encryption could lead to catastrophic failures (dapr#7524)

* [1.13] Add warning that Dapr state store encryption could lead to catastrophic failures

If the same key is used to encrypt more than 2^32 values (ie. more than 2^32 "Save" operations), it can lead to the private keys being exposed.

See: dapr#6027
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Changed per review feedback

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

---------

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Add content-length to http channel (dapr#7537)

* add content-length to http channel

Signed-off-by: yaron2 <schneider.yaron@live.com>

* update tests

Signed-off-by: yaron2 <schneider.yaron@live.com>

---------

Signed-off-by: yaron2 <schneider.yaron@live.com>

* Updates components-contrib to 1.13.0-rc.4 (dapr#7538)

* Updates components-contrib to 1.13.0-rc.3

Signed-off-by: joshvanl <me@joshvanl.dev>

* mod-tidy-all

Signed-off-by: joshvanl <me@joshvanl.dev>

---------

Signed-off-by: joshvanl <me@joshvanl.dev>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>

* Subscriptions: Fix panic when match rule is empty (dapr#7539)

Prevents a panic resulting when a subscription route rule match
(`spec.routes.rules.match`) is empty. An empty match rule is valid and
is considered "default". Error occurs because of Go interface vs
implementation struct pointer nil check foot-gun.

Adds integration tests for both HTTP and gRPC subscribers.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Hot Reloading: don't watch files if not enabled (dapr#7521)

* Hot Reloading: don't watch files if not enabled

Update hot reloading so that we don't setup file watchers on the
component directory if hot reloading is not enabled.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Remove `Close` paradigm from hot reloader

Signed-off-by: joshvanl <me@joshvanl.dev>

* Linting

Signed-off-by: joshvanl <me@joshvanl.dev>

---------

Signed-off-by: joshvanl <me@joshvanl.dev>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>

* Remove pubsub content-length test which has been removed from (dapr#7550)

release-1.13

Signed-off-by: joshvanl <me@joshvanl.dev>

* Revert ApiLevel controlling vnodes back to context metadata (dapr#7547)

* Revert ApiLevel controlling vnodes back to context metadata

Signed-off-by: Elena Kolevska <elena@kolevska.com>

* Lint

Signed-off-by: Elena Kolevska <elena@kolevska.com>

* Fixes after review

Signed-off-by: Elena Kolevska <elena@kolevska.com>

* Set back api level to 10, just for completeness purposes

Signed-off-by: Elena Kolevska <elena@kolevska.com>

* Don’t specify APILevelSpecify api level

Signed-off-by: Elena Kolevska <elena@kolevska.com>

* Apply suggestions from code review

Co-authored-by: Josh van Leeuwen <me@joshvanl.dev>
Signed-off-by: Elena Kolevska <elena-kolevska@users.noreply.github.com>

* Cleanup after review

Signed-off-by: Elena Kolevska <elena@kolevska.com>

* Small refactor

Signed-off-by: Elena Kolevska <elena@kolevska.com>

* Refactor multiple parameters into a request object for placement table dissemination

Signed-off-by: Elena Kolevska <elena@kolevska.com>

---------

Signed-off-by: Elena Kolevska <elena@kolevska.com>
Signed-off-by: Elena Kolevska <elena-kolevska@users.noreply.github.com>
Co-authored-by: Josh van Leeuwen <me@joshvanl.dev>
Co-authored-by: Artur Souza <asouza.pro@gmail.com>

* Actor Reminders: Default JSON serialization. (dapr#7548)

* Actor Reminders: Default JSON serialization.

To support downgrades to 1.12 from 1.13, this PR changes the reminder
serialization storage format back to JSON by default. This means a 1.12
actor reminder client can read reminders written by 1.13 actors.

1.13 will continue to understand both JSON and protobuf. Protobuf
serialization can be enabled with the `ActorReminderStorageProtobuf`
feature gate. The actor "API Level" has been changed back to 10.

Adds test to ensure the default serialization is JSON.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Remove ActorReminderStorageProtobuf feature gate in favour of using API
level

Signed-off-by: joshvanl <me@joshvanl.dev>

* Fix api level tests

Signed-off-by: joshvanl <me@joshvanl.dev>

---------

Signed-off-by: joshvanl <me@joshvanl.dev>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>

* Update contrib to 1.13.0-rc.6 (dapr#7553)

* update contrib to 1.13.0-rc.5

Signed-off-by: yaron2 <schneider.yaron@live.com>

* update to rc.6

Signed-off-by: yaron2 <schneider.yaron@live.com>

---------

Signed-off-by: yaron2 <schneider.yaron@live.com>

* Updates components-contrib to 1.13.0-rc.7 (dapr#7562)

Signed-off-by: joshvanl <me@joshvanl.dev>

* update contrib to 1.13.0-rc.8 (dapr#7567)

* Add metadata in binding response even in case of error (dapr#7572)

* Add binding metadata even in case of error.

Signed-off-by: Artur Souza <asouza.pro@gmail.com>

* ADD IT.

Signed-off-by: Artur Souza <asouza.pro@gmail.com>

* Fix lint.

Signed-off-by: Artur Souza <asouza.pro@gmail.com>

---------

Signed-off-by: Artur Souza <asouza.pro@gmail.com>

* [Release-1.13] Upgrade to contrib `v1.13.0-rc.10` (dapr#7577)

* Upgrade to contrib 1.13.0-rc.9

Signed-off-by: Bernd Verst <github@bernd.dev>

* Use contrib 1.13.0-rc.10 for latest kafka sarama patch version

Signed-off-by: Bernd Verst <github@bernd.dev>

---------

Signed-off-by: Bernd Verst <github@bernd.dev>

* Fix for issue 7576 (dapr#7581) (dapr#7587)

Signed-off-by: Guido Spadotto <guido.spadotto@profesia.it>
Co-authored-by: Guido Spadotto <guido.spad8@gmail.com>
Co-authored-by: Guido Spadotto <guido.spadotto@profesia.it>

* Adds v1.13.0 release notes (dapr#7586)

* Adds v1.13.0 release notes

Adds docs/release_notes/v1.13.0.md

Signed-off-by: joshvanl <me@joshvanl.dev>

* Update docs/release_notes/v1.13.0.md

Co-authored-by: Paul Yuknewicz <paulyuk@microsoft.com>
Signed-off-by: Josh van Leeuwen <me@joshvanl.dev>

* Move Actor Reminder Performance higher in notes

Signed-off-by: joshvanl <me@joshvanl.dev>

* Update v1.13.0.md

Signed-off-by: Artur Souza <asouza.pro@gmail.com>

---------

Signed-off-by: joshvanl <me@joshvanl.dev>
Signed-off-by: Josh van Leeuwen <me@joshvanl.dev>
Signed-off-by: Artur Souza <asouza.pro@gmail.com>
Co-authored-by: Paul Yuknewicz <paulyuk@microsoft.com>
Co-authored-by: Artur Souza <asouza.pro@gmail.com>

* chore: bump go to 1.21.8 and protobuf lib to 1.33.0 (dapr#7591)

* ci: force go1.21.8 in workflows instead of go.mod

Signed-off-by: mikeee <hey@mike.ee>

* chore: bump google.golang.org/protobuf to 1.33.0

Signed-off-by: mikeee <hey@mike.ee>

* make modtidy-all

Signed-off-by: Artur Souza <asouza.pro@gmail.com>

---------

Signed-off-by: mikeee <hey@mike.ee>
Signed-off-by: Artur Souza <asouza.pro@gmail.com>
Co-authored-by: Artur Souza <asouza.pro@gmail.com>

---------

Signed-off-by: yaron2 <schneider.yaron@live.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: joshvanl <me@joshvanl.dev>
Signed-off-by: Elena Kolevska <elena@kolevska.com>
Signed-off-by: Elena Kolevska <elena-kolevska@users.noreply.github.com>
Signed-off-by: Artur Souza <asouza.pro@gmail.com>
Signed-off-by: Bernd Verst <github@bernd.dev>
Signed-off-by: Guido Spadotto <guido.spadotto@profesia.it>
Signed-off-by: Josh van Leeuwen <me@joshvanl.dev>
Signed-off-by: mikeee <hey@mike.ee>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Elena Kolevska <elena-kolevska@users.noreply.github.com>
Co-authored-by: Artur Souza <asouza.pro@gmail.com>
Co-authored-by: Bernd Verst <github@bernd.dev>
Co-authored-by: Guido Spadotto <guido.spad8@gmail.com>
Co-authored-by: Guido Spadotto <guido.spadotto@profesia.it>
Co-authored-by: Paul Yuknewicz <paulyuk@microsoft.com>
Co-authored-by: Mike Nguyen <hey@mike.ee>
* make injector resilient to sentry unavailability

Signed-off-by: yaron2 <schneider.yaron@live.com>

* remove redundant line

Signed-off-by: yaron2 <schneider.yaron@live.com>

---------

Signed-off-by: yaron2 <schneider.yaron@live.com>
Signed-off-by: yaron2 <schneider.yaron@live.com>
* update contrib to 1.13.0-rc.2

Signed-off-by: yaron2 <schneider.yaron@live.com>

* update to rc.3

Signed-off-by: yaron2 <schneider.yaron@live.com>

---------

Signed-off-by: yaron2 <schneider.yaron@live.com>
The `DAPR_HOST_IP` env var is used in various places in Dapr for a sidecar to know its own IP address, for example for service invocation or actor invocation.

When using the Dapr injector to add the daprd container, we can use the downstream APIs to add the `DAPR_HOST_IP` env var based on data from the controller

This option can be enabled by setting the Helm option `dapr_sidecar_injector.enableK8sDownwardAPIs=true`

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
daixiang0 and others added 28 commits March 25, 2024 09:28
chore: convert crypto APIs to net/http
Before v1.12, Daprd's would connect to the sentry service on port 80. This
was changed to 443 in v1.12, but we kept the port 80 open for backwards
compatibility. This PR removes the port 80 from the sentry service.

Signed-off-by: joshvanl <me@joshvanl.dev>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
* Refactor api/errors into builders

Refactors api/errors into functional builders. The current API for
api/errors is heavily string concatenation based with long parameter
arguments. This is fragile/error prone to the developer and difficult to
understand for the consumer.

This change should make it clearer to the reader/consumer what the
intended errors are and how to use them. Moving implementation details
of the error protocol codes into the error package also helps to clean
up API implementation code.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Remove use of `stderror` `errors` alias

Signed-off-by: joshvanl <me@joshvanl.dev>

---------

Signed-off-by: joshvanl <me@joshvanl.dev>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Test Integration: reserve ports in ordered blocks

Dapr opens a lot of ports. Integration tests use upwards of a 1000 ports
through a run. Tests are run in parallel and ports are reserved on
demand through port 0 then being released before being used by a test.
There was a high chance that a port would attempted to be used by
multiple tests/processes at the same time causing conflict errors and
false test failures.

To prevent these conflicts, the ports utils has been moved to a process
which will reserve 300 blocks of ports to be used by tests. These ports
are reserved by incrementing through port ranges 1024 to 65535 in a
ring. This always prevents conflicts on this test runner, and should
also decrease port conflicts across multiple simultaneous test runners.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Rename intPort to port

Signed-off-by: joshvanl <me@joshvanl.dev>

---------

Signed-off-by: joshvanl <me@joshvanl.dev>
Co-authored-by: Loong Dai <long.dai@intel.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
Adds youtube link to a talk about the integration tests which might be
useful for developers to learn more.

Signed-off-by: joshvanl <me@joshvanl.dev>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
* Runtime Manifest Loader Refactor

The current implementation of Component and HTTPEndpoint disk &
Kubernetes manifest loader spanned across disjointed packages with semi
generic functionality. PR continues the existing generic manifest loader
pattern with a shared internal package. Moves both Component and
HTTPEndpont loader implementations into shared internal generic package.

Existing Subscription, Configuration, and Resiliency loaders are left
as-is but should will be brought into this shared generic loader in
future PRs.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Adds namespace filter to disk manifest loader

Signed-off-by: joshvanl <me@joshvanl.dev>

* Fix disk loader unit test

Signed-off-by: joshvanl <me@joshvanl.dev>

---------

Signed-off-by: joshvanl <me@joshvanl.dev>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
It is sometimes the case that Dapr services are suspended for a period
of time, for example when developer closes their laptop over night,
before being resumed after a significant period of time. Due to the
renewal timer being a Go Timer, it will also be suspended during this
time and only resume after the process is resumed.

Because X.509 certificates expiration validity follow wall-clock time
and the renewal timer follows process up-time time, it is possible for
a Dapr identity certificates to expire and not be renewed until enough
observed time has passed for the renewal timer to trigger.

This commit introduces a safety check whereby the renewal timer will
always trigger either every minute or at renewal time, whichever is
sooner. Once ticked, the renew loop will check to ensure whether the
renewal time has actually been reached before renewing.

This change helps out developers by not needing to restart the Dapr
cluster when they suspend their Dapr clusters for a period of time.
While it is not expected that this change be useful in production, it is
possible users are suspending and resuming virtual machines in
production.

Adds a renewal integration test to ensure Dapr identity certificate is
renewed based on expiration.

Signed-off-by: joshvanl <me@joshvanl.dev>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
Co-authored-by: Loong Dai <long.dai@intel.com>
Adds support for injector process in integration tests.

Adds basic healthz & ports tests for injector.

Signed-off-by: joshvanl <me@joshvanl.dev>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
Use correct free port import and interface in injector integration test
process.

Signed-off-by: joshvanl <me@joshvanl.dev>
Signed-off-by: mikeee <hey@mike.ee>
Co-authored-by: Josh van Leeuwen <me@joshvanl.dev>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
…ders

chore: Convert actor HTTP APIs off fasthttp
…IRE TLS clients and servers. (dapr#7037)

* Removes legacy SPIFFE TLS clients and servers in favour of the new SPIRE TLS clients and servers.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Fix sentry int tests, and adds test to ensure legacy ID is not longer
accepted

Signed-off-by: joshvanl <me@joshvanl.dev>

* String match on sentry Kubernetes validator longname test

Signed-off-by: joshvanl <me@joshvanl.dev>

* Fix namespace of sentry in operator tests

Signed-off-by: joshvanl <me@joshvanl.dev>

* Linting

Signed-off-by: joshvanl <me@joshvanl.dev>

* Update integration kubernetes process to use leaf certificate with
cluster.local

Signed-off-by: joshvanl <me@joshvanl.dev>

* Fix setting correct control plane trust domain on daprd

Signed-off-by: joshvanl <me@joshvanl.dev>

* Remove SENTRY_LOCAL_IDENTITY form expected env var

Signed-off-by: joshvanl <me@joshvanl.dev>

* Fix control plane trust domain setting in test

Signed-off-by: joshvanl <me@joshvanl.dev>

* Fixes int version skew tests using legacy client/server

Signed-off-by: joshvanl <me@joshvanl.dev>

* Fix int version-skew patch on v1.13.0

Signed-off-by: joshvanl <me@joshvanl.dev>

* Use correct namespace for sentry in injector integration tests

Signed-off-by: joshvanl <me@joshvanl.dev>

---------

Signed-off-by: joshvanl <me@joshvanl.dev>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
…apr#7657)

When block shutdown is enabled, Dapr will block the shutdown sequence
for a configurable amount of time or until the app becomes unhealthy.
During this time the out-bound Dapr APIs remain available but all
in-bound requests are blocked. Currently, in-flight PubSub messages are
canceled when Dapr receives a SIGTERM, which can result in lost
messages.

Patch updates the pubsub subscribe message handler policy runner to
execute sending the message to the app channel in a background context.
This ensures that when Dapr cancels the Subscription due to a blocking
shutdown, the in-flight message is not effected.

Enables the `dapr_component_pubsub_egress_bulk_count` and
`dapr_component_pubsub_egress_bulk_latency` metrics for bulk pubsub
egress which were previously not enabled, as they are used to test that
an in-flight message is correctly sent after Dapr has begun to shutdown.

Signed-off-by: joshvanl <me@joshvanl.dev>
…pr#7659)

* Fix incorrect content-length being sent to HTTP published message

PR dapr#7537 reverse revered the change which removed the content-length
from being set on HTTP headers on sent messages. We still need to remove
the content-length from messages from pubsub subscriptions as the pubsub
may report a message with a content-length which does not actually
match the size of the delivered message to the app.

content-length is only removed on HTTP published messages.

Change should be backported to 1.13

Signed-off-by: joshvanl <me@joshvanl.dev>

* Adds grpc app subscriber to content-length tests

Signed-off-by: joshvanl <me@joshvanl.dev>

* Adds tests for content-length gRPC subscribed app

Signed-off-by: joshvanl <me@joshvanl.dev>

* framework/socket: skip if test is windows

Signed-off-by: joshvanl <me@joshvanl.dev>

* Fix socket runtime GOOS import

Signed-off-by: joshvanl <me@joshvanl.dev>

---------

Signed-off-by: joshvanl <me@joshvanl.dev>
On Darwin (MacOS), `os.TempDir()` nicely returns a truly ephemeral
process directory is `/var/folders/...` however in order to reap the
benefits of Go cache magic in integration tests we need a slightly more
permanent directory, i.e. `/tmp`. This directory will persist across
integration test runs and speed up the test execution boot times from
s to ms.

Signed-off-by: joshvanl <me@joshvanl.dev>
Signed-off-by: Lee Fowler <fowler.lee8@gmail.com>
Signed-off-by: Lee Fowler <fowler.lee8@gmail.com>
Signed-off-by: Lee Fowler <fowler.lee8@gmail.com>
Signed-off-by: Lee Fowler <fowler.lee8@gmail.com>
Signed-off-by: Lee Fowler <fowler.lee8@gmail.com>
@fowlerlee fowlerlee marked this pull request as ready for review April 2, 2024 14:35
@cicoyle
Copy link
Contributor

cicoyle commented May 7, 2024

It looks like the git rebase might have messed up. I am seeing a file diff of 364 files changed, which I wouldn't expect for this change. Can you try again, either a merge or rebase? I can better assist once I can see just the changes intended for this PR 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet