Flaky test: Aspire.Hosting.Garnet.Tests.GarnetFunctionalTests.VerifyGarnetResource

[radical]

Build Information

Build: https://github.com/dotnet/aspire/actions/runs/15200481758
Build error leg or test failing: Aspire.Hosting.Garnet.Tests.GarnetFunctionalTests.VerifyGarnetResource
Pull request:

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

StackExchange.Redis.RedisConnectionException : The message timed out in the backlog attempting to send because no connection became available (5000ms) - Last Connection Exception: SocketClosed (ReadEndOfStream, 0-read, last-recv: 0) on localhost:37049/Interactive, Flushed/MarkProcessed, last: ECHO, origin: ReadFromPipe, outstanding: 13, last-read: 0s ago, last-write: 3s ago, unanswered-write: 3s ago, keep-alive: 60s, state: ConnectedEstablishing, mgr: 8 of 10 available, last-heartbeat: never, global: 17s ago, v: 2.8.37.59676, command=SET, timeout: 5000, inst: 0, qu: 0, qs: 0, aw: False, bw: CheckingForTimeout, last-in: 0, cur-in: 0, sync-ops: 0, async-ops: 11, serverEndpoint: localhost:37049, conn-sec: n/a, aoc: 0, mc: 1/1/0, mgr: 10 of 10 available, clientName: fv-az1692-962(SE.Redis-v2.8.37.59676), IOCP: (Busy=0,Free=1000,Min=1,Max=1000), WORKER: (Busy=2,Free=32765,Min=4,Max=32767), POOL: (Threads=7,QueuedItems=0,CompletedItems=10567,Timers=12), v: 2.8.37.59676 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)
---- StackExchange.Redis.RedisConnectionException : SocketClosed (ReadEndOfStream, 0-read, last-recv: 0) on localhost:37049/Interactive, Flushed/MarkProcessed, last: ECHO, origin: ReadFromPipe, outstanding: 13, last-read: 0s ago, last-write: 3s ago, unanswered-write: 3s ago, keep-alive: 60s, state: ConnectedEstablishing, mgr: 8 of 10 available, last-heartbeat: never, global: 17s ago, v: 2.8.37.59676   at Aspire.Hosting.Garnet.Tests.GarnetFunctionalTests.<>c__DisplayClass3_0.<<VerifyGarnetResource>b__0>d.MoveNext() in /_/tests/Aspire.Hosting.Garnet.Tests/GarnetFunctionalTests.cs:line 89
--- End of stack trace from previous location ---
   at Polly.ResiliencePipeline.<>c.<<ExecuteAsync>b__3_0>d.MoveNext()
--- End of stack trace from previous location ---
   at Polly.Outcome`1.GetResultOrRethrow()
   at Polly.ResiliencePipeline.ExecuteAsync(Func`2 callback, CancellationToken cancellationToken)
   at Aspire.Hosting.Garnet.Tests.GarnetFunctionalTests.VerifyGarnetResource() in /_/tests/Aspire.Hosting.Garnet.Tests/GarnetFunctionalTests.cs:line 85
--- End of stack trace from previous location ---
----- Inner Stack Trace -----

Stdout

Details

| [2025-05-23T01:56:53] Aspire.Hosting.DistributedApplication Information: Aspire version: 9.4.0-ci
| [2025-05-23T01:56:53] Aspire.Hosting.DistributedApplication Information: Distributed application starting.
| [2025-05-23T01:56:53] Aspire.Hosting.DistributedApplication Information: Application host directory is: /home/runner/work/aspire/aspire/tests/Aspire.Hosting.Tests
| [2025-05-23T01:56:53] Aspire.Hosting.Cli.BackchannelService Debug: Backchannel socket path was not specified.
| [2025-05-23T01:56:53] Aspire.Hosting.Dcp.DcpHost Information: Starting DCP with arguments: start-apiserver --monitor 3803 --detach --kubeconfig "/tmp/aspire.3NSMM3/kubeconfig"
| [2025-05-23T01:56:53] Aspire.Hosting.ApplicationModel.ResourceNotificationService Trace: Version: 1 Resource garnet/garnet-aargznfx update published: ResourceType = GarnetResource, CreationTimeStamp = (null), State = { Text = (null), Style = (null) }, IsHidden = False, HeathStatus = (null), ResourceReady = False, ExitCode = (null), Urls = {  }, EnvironmentVariables = {  }, Properties = {  }, HealthReports = { garnet_check = (null) }, Commands = { resource-start (Start) = Disabled resource-stop (Stop) = Hidden resource-restart (Restart) = Disabled }
| [2025-05-23T01:56:53] Aspire.Hosting.Dcp.DcpExecutor Debug: Watching over DCP Endpoint resources.
| [2025-05-23T01:56:53] Aspire.Hosting.Dcp.DcpExecutor Debug: Watching over DCP Service resources.
| [2025-05-23T01:56:53] Aspire.Hosting.Dcp.DcpExecutor Debug: Watching over DCP Executable resources.
| [2025-05-23T01:56:53] Aspire.Hosting.Dcp.DcpExecutor Debug: Watching over DCP Container resources.
| [2025-05-23T01:56:55] Aspire.Hosting.Dcp.dcp.start-apiserver.api-server Information: Starting API server...
| [2025-05-23T01:56:55] Aspire.Hosting.Dcp.dcp.start-apiserver.api-server Information: API server started	{"Address": "::1", "Port": 36137}
| [2025-05-23T01:56:55] Aspire.Hosting.Dcp.dcp.start-apiserver.dcp-host Information: Starting DCP controller host
| [2025-05-23T01:56:55] Aspire.Hosting.Dcp.dcp.start-apiserver.dcp-host Information: Started all services	{"count": 1}
| [2025-05-23T01:56:55] Aspire.Hosting.Dcp.KubernetesService Debug: Successfully read Kubernetes configuration from '/tmp/aspire.3NSMM3/kubeconfig' after 1606 milliseconds.
| [2025-05-23T01:56:55] Aspire.Hosting.Dcp.dcpctrl.ContainerOrchestrator Information: runtime status	{"ContainerRuntime": "", "runtime": "docker", "status": {"Installed":true,"Running":true,"Error":""}}
| [2025-05-23T01:56:55] Aspire.Hosting.Dcp.dcpctrl.ContainerOrchestrator Information: runtime status	{"ContainerRuntime": "", "runtime": "podman", "status": {"Installed":true,"Running":true,"Error":""}}
| [2025-05-23T01:56:55] Aspire.Hosting.Dcp.dcpctrl.IdeExecutableRunner Information: Executables cannot be started via IDE: missing required environment variable 'DEBUG_SESSION_PORT'
| [2025-05-23T01:56:55] Aspire.Hosting.Dcp.dcpctrl Information: starting controller manager
| [2025-05-23T01:56:55] Aspire.Hosting.ApplicationModel.ResourceNotificationService Trace: Version: 2 Resource garnet/garnet-aargznfx update published: ResourceType = GarnetResource, CreationTimeStamp = (null), State = { Text = (null), Style = (null) }, IsHidden = False, HeathStatus = (null), ResourceReady = False, ExitCode = (null), Urls = { tcp = tcp://localhost:37049 }, EnvironmentVariables = {  }, Properties = {  }, HealthReports = { garnet_check = (null) }, Commands = { resource-start (Start) = Disabled resource-stop (Stop) = Hidden resource-restart (Restart) = Disabled }
| [2025-05-23T01:56:55] Aspire.Hosting.ApplicationModel.ResourceNotificationService Trace: Version: 3 Resource garnet/garnet-aargznfx update published: ResourceType = Container, CreationTimeStamp = (null), State = { Text = (null), Style = (null) }, IsHidden = False, HeathStatus = (null), ResourceReady = False, ExitCode = (null), Urls = { tcp = tcp://localhost:37049 }, EnvironmentVariables = {  }, Properties = { container.image = ghcr.io/microsoft/garnet:1.0 container.id = (null) container.command = (null) container.args =  container.ports =  container.lifetime = Session resource.appArgs = (null) resource.appArgsSensitivity = (null) }, HealthReports = { garnet_check = (null) }, Commands = { resource-start (Start) = Disabled resource-stop (Stop) = Hidden resource-restart (Restart) = Disabled }
| [2025-05-23T01:56:55] Aspire.Hosting.ApplicationModel.ResourceNotificationService Debug: Resource garnet/garnet-aargznfx changed state: Starting
| [2025-05-23T01:56:55] Aspire.Hosting.ApplicationModel.ResourceNotificationService Trace: Version: 4 Resource garnet/garnet-aargznfx update published: ResourceType = Container, CreationTimeStamp = (null), State = { Text = Starting, Style = (null) }, IsHidden = False, HeathStatus = (null), ResourceReady = False, ExitCode = (null), Urls = { tcp = tcp://localhost:37049 }, EnvironmentVariables = {  }, Properties = { container.image = ghcr.io/microsoft/garnet:1.0 container.id = (null) container.command = (null) container.args =  container.ports =  container.lifetime = Session resource.appArgs = (null) resource.appArgsSensitivity = (null) }, HealthReports = { garnet_check = (null) }, Commands = { resource-start (Start) = Disabled resource-stop (Stop) = Hidden resource-restart (Restart) = Disabled }
| [2025-05-23T01:56:55] Aspire.Hosting.Dcp.DcpExecutor Trace: Updating application model resource garnet with Container resource garnet-aargznfx
| [2025-05-23T01:56:55] Aspire.Hosting.ApplicationModel.ResourceNotificationService Trace: Version: 5 Resource garnet/garnet-aargznfx update published: ResourceType = Container, CreationTimeStamp = 2025-05-23T01:56:55, State = { Text = (null), Style = (null) }, IsHidden = False, HeathStatus = (null), ResourceReady = False, ExitCode = (null), Urls = { tcp = tcp://localhost:37049 }, EnvironmentVariables = { GARNET_PASSWORD = n37Wu1k1PJgcQPqBym1QtE }, Properties = { container.image = ghcr.io/microsoft/garnet:1.0 container.id = (null) container.command = /bin/sh container.args =  container.ports = 6379 container.lifetime = Session resource.appArgs = -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD resource.appArgsSensitivity = 0, 0 }, HealthReports = { garnet_check = (null) }, Commands = { resource-start (Start) = Disabled resource-stop (Stop) = Hidden resource-restart (Restart) = Disabled }
| [2025-05-23T01:56:55] Aspire.Hosting.Devcontainers.Codespaces.CodespacesResourceUrlRewriterService Trace: Not running in Codespaces, skipping URL rewriting.
| [2025-05-23T01:56:55] Aspire.Hosting.DistributedApplication Information: Distributed application started. Press Ctrl+C to shut down.
| [2025-05-23T01:56:55] Aspire.Hosting.Dcp.DcpExecutor Trace: Updating application model resource garnet with Container resource garnet-aargznfx
| [2025-05-23T01:56:55] Aspire.Hosting.ApplicationModel.ResourceNotificationService Trace: Version: 6 Resource garnet/garnet-aargznfx update published: ResourceType = Container, CreationTimeStamp = 2025-05-23T01:56:55, State = { Text = (null), Style = (null) }, IsHidden = False, HeathStatus = (null), ResourceReady = False, ExitCode = (null), Urls = { tcp = tcp://localhost:37049 }, EnvironmentVariables = { GARNET_PASSWORD = n37Wu1k1PJgcQPqBym1QtE }, Properties = { container.image = ghcr.io/microsoft/garnet:1.0 container.id = (null) container.command = /bin/sh container.args =  container.ports = 6379 container.lifetime = Session resource.appArgs = -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD resource.appArgsSensitivity = 0, 0 }, HealthReports = { garnet_check = (null) }, Commands = { resource-start (Start) = Disabled resource-stop (Stop) = Hidden resource-restart (Restart) = Disabled }
| [2025-05-23T01:56:55] Aspire.Hosting.Dcp.DcpExecutor Trace: Updating application model resource garnet with Container resource garnet-aargznfx
| [2025-05-23T01:56:55] Aspire.Hosting.ApplicationModel.ResourceNotificationService Debug: Resource garnet/garnet-aargznfx changed state: Starting
| [2025-05-23T01:56:55] Aspire.Hosting.ApplicationModel.ResourceNotificationService Trace: Version: 7 Resource garnet/garnet-aargznfx update published: ResourceType = Container, CreationTimeStamp = 2025-05-23T01:56:55, State = { Text = Starting, Style = (null) }, IsHidden = False, HeathStatus = (null), ResourceReady = False, ExitCode = (null), Urls = { tcp = tcp://localhost:37049 }, EnvironmentVariables = { GARNET_PASSWORD = n37Wu1k1PJgcQPqBym1QtE }, Properties = { container.image = ghcr.io/microsoft/garnet:1.0 container.id = (null) container.command = /bin/sh container.args =  container.ports = 6379 container.lifetime = Session resource.appArgs = -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD resource.appArgsSensitivity = 0, 0 }, HealthReports = { garnet_check = (null) }, Commands = { resource-start (Start) = Disabled resource-stop (Stop) = Hidden resource-restart (Restart) = Disabled }
| [2025-05-23T01:56:55] Aspire.Hosting.Dcp.DcpExecutor Debug: Starting log streaming for garnet-aargznfx.
| [2025-05-23T01:56:55] Aspire.Hosting.Dcp.DcpExecutor Trace: Updating application model resource garnet with Container resource garnet-aargznfx
| [2025-05-23T01:56:55] Aspire.Hosting.ApplicationModel.ResourceNotificationService Trace: Version: 8 Resource garnet/garnet-aargznfx update published: ResourceType = Container, CreationTimeStamp = 2025-05-23T01:56:55, State = { Text = Starting, Style = (null) }, IsHidden = False, HeathStatus = (null), ResourceReady = False, ExitCode = (null), Urls = { tcp = tcp://localhost:37049 }, EnvironmentVariables = { GARNET_PASSWORD = n37Wu1k1PJgcQPqBym1QtE }, Properties = { container.image = ghcr.io/microsoft/garnet:1.0 container.id = (null) container.command = /bin/sh container.args = -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD container.ports = 6379 container.lifetime = Session resource.appArgs = -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD resource.appArgsSensitivity = 0, 0 }, HealthReports = { garnet_check = (null) }, Commands = { resource-start (Start) = Disabled resource-stop (Stop) = Hidden resource-restart (Restart) = Disabled }
| [2025-05-23T01:56:55] Aspire.Hosting.Dcp.dcp.start-apiserver.api-server.container-logstreamer.ContainerOrchestrator Information: runtime status	{"ContainerRuntime": "", "runtime": "docker", "status": {"Installed":true,"Running":true,"Error":""}}
| [2025-05-23T01:56:55] Aspire.Hosting.Dcp.dcp.start-apiserver.api-server.container-logstreamer.ContainerOrchestrator Information: runtime status	{"ContainerRuntime": "", "runtime": "podman", "status": {"Installed":true,"Running":true,"Error":""}}
| [2025-05-23T01:56:55] Aspire.Hosting.Tests.Resources.garnet Information: 1: 2025-05-23T01:56:55.8520000Z 5b040373c9cda721c3c652a6a1f20205ee9f87f0a947805ae74f3d9bb4fb20c9
| [2025-05-23T01:56:56] Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler Information: network created	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 2, "Network": "default-aspire-network-a616datjno"}
| [2025-05-23T01:56:56] Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler Information: all expected containers are connected to the network	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 3, "Network": "default-aspire-network-a616datjno", "Expected": 0, "Found": 0}
| [2025-05-23T01:56:56] Aspire.Hosting.Dcp.dcpctrl.ContainerReconciler Information: Added new ContainerNetworkConnection	{"Container": {"name":"garnet-aargznfx"}, "Reconciliation": 5, "Container": "5b040373c9cda721c3c652a6a1f20205ee9f87f0a947805ae74f3d9bb4fb20c9", "ContainerNetworkConnection": "/garnet-aargznfx-default-aspire-network-resunmbo40"}
| [2025-05-23T01:56:56] Aspire.Hosting.Dcp.DcpExecutor Trace: Updating application model resource garnet with Container resource garnet-aargznfx
| [2025-05-23T01:56:56] Aspire.Hosting.ApplicationModel.ResourceNotificationService Trace: Version: 9 Resource garnet/garnet-aargznfx update published: ResourceType = Container, CreationTimeStamp = 2025-05-23T01:56:55, State = { Text = Starting, Style = (null) }, IsHidden = False, HeathStatus = (null), ResourceReady = False, ExitCode = (null), Urls = { tcp = tcp://localhost:37049 }, EnvironmentVariables = { GARNET_PASSWORD = n37Wu1k1PJgcQPqBym1QtE }, Properties = { container.image = ghcr.io/microsoft/garnet:1.0 container.id = 5b040373c9cda721c3c652a6a1f20205ee9f87f0a947805ae74f3d9bb4fb20c9 container.command = /bin/sh container.args = /bin/sh, -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD container.ports = 6379 container.lifetime = Session resource.appArgs = -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD resource.appArgsSensitivity = 0, 0 }, HealthReports = { garnet_check = (null) }, Commands = { resource-start (Start) = Disabled resource-stop (Stop) = Hidden resource-restart (Restart) = Disabled }
| [2025-05-23T01:56:56] Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler Information: not all expected containers are connected to the network, retrying...	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 4, "Network": "default-aspire-network-a616datjno", "Expected": 1, "Found": 0}
| [2025-05-23T01:56:58] Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler Information: all expected containers are connected to the network	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 5, "Network": "default-aspire-network-a616datjno", "Expected": 1, "Found": 1}
| [2025-05-23T01:56:59] Aspire.Hosting.Dcp.DcpExecutor Trace: Updating application model resource garnet with Container resource garnet-aargznfx
| [2025-05-23T01:56:59] Aspire.Hosting.ApplicationModel.ResourceNotificationService Debug: Resource garnet/garnet-aargznfx changed state: Starting -> Running
| [2025-05-23T01:56:59] Aspire.Hosting.ApplicationModel.ResourceNotificationService Trace: Version: 10 Resource garnet/garnet-aargznfx update published: ResourceType = Container, CreationTimeStamp = 2025-05-23T01:56:55, State = { Text = Running, Style = (null) }, IsHidden = False, HeathStatus = Unhealthy, ResourceReady = False, ExitCode = (null), Urls = { tcp = tcp://localhost:37049 }, EnvironmentVariables = { GARNET_PASSWORD = n37Wu1k1PJgcQPqBym1QtE }, Properties = { container.image = ghcr.io/microsoft/garnet:1.0 container.id = 5b040373c9cda721c3c652a6a1f20205ee9f87f0a947805ae74f3d9bb4fb20c9 container.command = /bin/sh container.args = /bin/sh, -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD container.ports = 6379 container.lifetime = Session resource.appArgs = -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD resource.appArgsSensitivity = 0, 0 }, HealthReports = { garnet_check = (null) }, Commands = { resource-start (Start) = Hidden resource-stop (Stop) = Enabled resource-restart (Restart) = Enabled }
| [2025-05-23T01:56:59] Aspire.Hosting.Health.ResourceHealthCheckService Debug: Starting health monitoring for resource 'garnet'.
| [2025-05-23T01:56:59] Aspire.Hosting.Health.ResourceHealthCheckService Debug: Resource 'garnet' health checks to monitor: garnet_check
| [2025-05-23T01:56:59] Aspire.Hosting.ApplicationModel.ResourceNotificationService Trace: Version: 11 Resource garnet/garnet-aargznfx update published: ResourceType = Container, CreationTimeStamp = 2025-05-23T01:56:55, State = { Text = Running, Style = (null) }, IsHidden = False, HeathStatus = Unhealthy, ResourceReady = False, ExitCode = (null), Urls = { tcp = tcp://localhost:37049 }, EnvironmentVariables = { GARNET_PASSWORD = n37Wu1k1PJgcQPqBym1QtE }, Properties = { container.image = ghcr.io/microsoft/garnet:1.0 container.id = 5b040373c9cda721c3c652a6a1f20205ee9f87f0a947805ae74f3d9bb4fb20c9 container.command = /bin/sh container.args = /bin/sh, -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD container.ports = 6379 container.lifetime = Session resource.appArgs = -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD resource.appArgsSensitivity = 0, 0 }, HealthReports = { garnet_check = (null) }, Commands = { resource-start (Start) = Hidden resource-stop (Stop) = Enabled resource-restart (Restart) = Enabled }
| [2025-05-23T01:56:59] Aspire.Hosting.ApplicationModel.ResourceNotificationService Trace: Version: 12 Resource garnet/garnet-aargznfx update published: ResourceType = Container, CreationTimeStamp = 2025-05-23T01:56:55, State = { Text = Running, Style = (null) }, IsHidden = False, HeathStatus = Unhealthy, ResourceReady = False, ExitCode = (null), Urls = { tcp = tcp://localhost:37049 }, EnvironmentVariables = { GARNET_PASSWORD = n37Wu1k1PJgcQPqBym1QtE }, Properties = { container.image = ghcr.io/microsoft/garnet:1.0 container.id = 5b040373c9cda721c3c652a6a1f20205ee9f87f0a947805ae74f3d9bb4fb20c9 container.command = /bin/sh container.args = /bin/sh, -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD container.ports = 6379 container.lifetime = Session resource.appArgs = -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD resource.appArgsSensitivity = 0, 0 }, HealthReports = { garnet_check = (null) }, Commands = { resource-start (Start) = Hidden resource-stop (Stop) = Enabled resource-restart (Restart) = Enabled }
| [2025-05-23T01:56:59] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Health report status for 'garnet' is Unhealthy.
| [2025-05-23T01:56:59] Aspire.Hosting.Dcp.DcpExecutor Trace: Updating application model resource garnet with Container resource garnet-aargznfx
| [2025-05-23T01:56:59] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Health reports for 'garnet' have changed. Publishing updated reports.
| [2025-05-23T01:56:59] Aspire.Hosting.ApplicationModel.ResourceNotificationService Trace: Version: 13 Resource garnet/garnet-aargznfx update published: ResourceType = Container, CreationTimeStamp = 2025-05-23T01:56:55, State = { Text = Running, Style = (null) }, IsHidden = False, HeathStatus = Unhealthy, ResourceReady = False, ExitCode = (null), Urls = { tcp = tcp://localhost:37049 }, EnvironmentVariables = { GARNET_PASSWORD = n37Wu1k1PJgcQPqBym1QtE }, Properties = { container.image = ghcr.io/microsoft/garnet:1.0 container.id = 5b040373c9cda721c3c652a6a1f20205ee9f87f0a947805ae74f3d9bb4fb20c9 container.command = /bin/sh container.args = /bin/sh, -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD container.ports = 6379 container.lifetime = Session resource.appArgs = -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD resource.appArgsSensitivity = 0, 0 }, HealthReports = { garnet_check = (null) }, Commands = { resource-start (Start) = Hidden resource-stop (Stop) = Enabled resource-restart (Restart) = Enabled }
| [2025-05-23T01:56:59] Aspire.Hosting.ApplicationModel.ResourceNotificationService Trace: Version: 14 Resource garnet/garnet-aargznfx update published: ResourceType = Container, CreationTimeStamp = 2025-05-23T01:56:55, State = { Text = Running, Style = (null) }, IsHidden = False, HeathStatus = Unhealthy, ResourceReady = False, ExitCode = (null), Urls = { tcp = tcp://localhost:37049 }, EnvironmentVariables = { GARNET_PASSWORD = n37Wu1k1PJgcQPqBym1QtE }, Properties = { container.image = ghcr.io/microsoft/garnet:1.0 container.id = 5b040373c9cda721c3c652a6a1f20205ee9f87f0a947805ae74f3d9bb4fb20c9 container.command = /bin/sh container.args = /bin/sh, -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD container.ports = 6379 container.lifetime = Session resource.appArgs = -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD resource.appArgsSensitivity = 0, 0 }, HealthReports = { garnet_check = Unhealthy }, Commands = { resource-start (Start) = Hidden resource-stop (Stop) = Enabled resource-restart (Restart) = Enabled }
| [2025-05-23T01:56:59] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Resource 'garnet' health check monitoring loop starting delay of 00:00:01.
| [2025-05-23T01:56:59] Aspire.Hosting.Tests.Resources.garnet Information: 2: 2025-05-23T01:56:59.3334667Z \u001b[31m    _________
| [2025-05-23T01:56:59] Aspire.Hosting.Tests.Resources.garnet Information: 3: 2025-05-23T01:56:59.3335244Z    /_||___||_\      \u001b[0mGarnet 1.0.66 64 bit; standalone mode\u001b[31m
| [2025-05-23T01:56:59] Aspire.Hosting.Tests.Resources.garnet Information: 4: 2025-05-23T01:56:59.3335318Z    '. \   / .'      \u001b[0mListening on: 127.0.0.1:6379 and 1 more\u001b[31m
| [2025-05-23T01:56:59] Aspire.Hosting.Tests.Resources.garnet Information: 5: 2025-05-23T01:56:59.3335358Z      '.\ /.'        \u001b[35mhttps://aka.ms/GetGarnet\u001b[31m
| [2025-05-23T01:56:59] Aspire.Hosting.Tests.Resources.garnet Information: 6: 2025-05-23T01:56:59.3335395Z        '.'
| [2025-05-23T01:56:59] Aspire.Hosting.Tests.Resources.garnet Information: 7: 2025-05-23T01:56:59.3335430Z \u001b[0m
| [2025-05-23T01:56:59] Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler Information: all expected containers are connected to the network	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 6, "Network": "default-aspire-network-a616datjno", "Expected": 1, "Found": 1}
| [2025-05-23T01:56:59] Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler Information: all expected containers are connected to the network	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 7, "Network": "default-aspire-network-a616datjno", "Expected": 1, "Found": 1}
| [2025-05-23T01:56:59] Aspire.Hosting.Tests.Resources.garnet Information: 8: 2025-05-23T01:56:59.4183652Z * Ready to accept connections
| [2025-05-23T01:57:00] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "9: [::1]:54602 -> [::1]:37049 (proxy) 127.0.0.1:36570 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"540","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:57:00.233","ReadError":"writeto tcp 127.0.0.1:36570->127.0.0.1:32770: read tcp 127.0.0.1:36570->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:57:00.233","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:36570->127.0.0.1:32770: read tcp 127.0.0.1:36570->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:57:00] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "8: [::1]:54612 -> [::1]:37049 (proxy) 127.0.0.1:36564 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"347","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:57:00.233","ReadError":"writeto tcp 127.0.0.1:36564->127.0.0.1:32770: read tcp 127.0.0.1:36564->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:57:00.233","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:36564->127.0.0.1:32770: read tcp 127.0.0.1:36564->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:57:00] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Health report status for 'garnet' is Unhealthy.
| [2025-05-23T01:57:00] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Resource 'garnet' health check monitoring loop starting delay of 00:00:02.
| [2025-05-23T01:57:02] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "20: [::1]:54716 -> [::1]:37049 (proxy) 127.0.0.1:36686 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"540","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:57:02.239","ReadError":"writeto tcp 127.0.0.1:36686->127.0.0.1:32770: read tcp 127.0.0.1:36686->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:57:02.239","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:36686->127.0.0.1:32770: read tcp 127.0.0.1:36686->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:57:02] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "21: [::1]:54722 -> [::1]:37049 (proxy) 127.0.0.1:36698 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"347","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:57:02.240","ReadError":"writeto tcp 127.0.0.1:36698->127.0.0.1:32770: read tcp 127.0.0.1:36698->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:57:02.240","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:36698->127.0.0.1:32770: read tcp 127.0.0.1:36698->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:57:02] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Health report status for 'garnet' is Unhealthy.
| [2025-05-23T01:57:02] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Resource 'garnet' health check monitoring loop starting delay of 00:00:03

... (snip) ...

... rror":"writeto tcp 127.0.0.1:41334->127.0.0.1:32770: read tcp 127.0.0.1:41334->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:57:54.276","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:41334->127.0.0.1:32770: read tcp 127.0.0.1:41334->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:57:54] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "91: [::1]:42198 -> [::1]:37049 (proxy) 127.0.0.1:41336 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"347","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:57:54.276","ReadError":"writeto tcp 127.0.0.1:41336->127.0.0.1:32770: read tcp 127.0.0.1:41336->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:57:54.276","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:41336->127.0.0.1:32770: read tcp 127.0.0.1:41336->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:57:54] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "93: [::1]:42214 -> [::1]:37049 (proxy) 127.0.0.1:41348 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"540","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:57:54.277","ReadError":"writeto tcp 127.0.0.1:41348->127.0.0.1:32770: read tcp 127.0.0.1:41348->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:57:54.277","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:41348->127.0.0.1:32770: read tcp 127.0.0.1:41348->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:57:54] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Health report status for 'garnet' is Unhealthy.
| [2025-05-23T01:57:54] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Resource 'garnet' health check monitoring loop starting delay of 00:00:05.
| [2025-05-23T01:57:56] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "97: [::1]:46116 -> [::1]:37049 (proxy) 127.0.0.1:52584 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"540","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:57:56.232","ReadError":"writeto tcp 127.0.0.1:52584->127.0.0.1:32770: read tcp 127.0.0.1:52584->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:57:56.232","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:52584->127.0.0.1:32770: read tcp 127.0.0.1:52584->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:57:57] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "98: [::1]:46120 -> [::1]:37049 (proxy) 127.0.0.1:52594 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"347","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:57:57.232","ReadError":"writeto tcp 127.0.0.1:52594->127.0.0.1:32770: read tcp 127.0.0.1:52594->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:57:57.232","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:52594->127.0.0.1:32770: read tcp 127.0.0.1:52594->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:57:59] Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler Information: all expected containers are connected to the network	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 16, "Network": "default-aspire-network-a616datjno", "Expected": 1, "Found": 1}
| [2025-05-23T01:57:59] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "100: [::1]:46130 -> [::1]:37049 (proxy) 127.0.0.1:52604 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"347","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:57:59.281","ReadError":"writeto tcp 127.0.0.1:52604->127.0.0.1:32770: read tcp 127.0.0.1:52604->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:57:59.281","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:52604->127.0.0.1:32770: read tcp 127.0.0.1:52604->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:57:59] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Health report status for 'garnet' is Unhealthy.
| [2025-05-23T01:57:59] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Resource 'garnet' health check monitoring loop starting delay of 00:00:05.
| [2025-05-23T01:58:02] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "105: [::1]:46176 -> [::1]:37049 (proxy) 127.0.0.1:52636 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"540","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:02.232","ReadError":"writeto tcp 127.0.0.1:52636->127.0.0.1:32770: read tcp 127.0.0.1:52636->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:02.232","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:52636->127.0.0.1:32770: read tcp 127.0.0.1:52636->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:02] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "106: [::1]:46180 -> [::1]:37049 (proxy) 127.0.0.1:52646 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"347","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:02.232","ReadError":"writeto tcp 127.0.0.1:52646->127.0.0.1:32770: read tcp 127.0.0.1:52646->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:02.232","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:52646->127.0.0.1:32770: read tcp 127.0.0.1:52646->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:04] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "107: [::1]:46184 -> [::1]:37049 (proxy) 127.0.0.1:52656 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"540","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:04.286","ReadError":"writeto tcp 127.0.0.1:52656->127.0.0.1:32770: read tcp 127.0.0.1:52656->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:04.286","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:52656->127.0.0.1:32770: read tcp 127.0.0.1:52656->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:04] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "108: [::1]:46196 -> [::1]:37049 (proxy) 127.0.0.1:52658 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"347","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:04.287","ReadError":"writeto tcp 127.0.0.1:52658->127.0.0.1:32770: read tcp 127.0.0.1:52658->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:04.287","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:52658->127.0.0.1:32770: read tcp 127.0.0.1:52658->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:04] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Health report status for 'garnet' is Unhealthy.
| [2025-05-23T01:58:04] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Resource 'garnet' health check monitoring loop starting delay of 00:00:05.
| [2025-05-23T01:58:06] Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler Information: all expected containers are connected to the network	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 17, "Network": "default-aspire-network-a616datjno", "Expected": 1, "Found": 1}
| [2025-05-23T01:58:08] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "111: [::1]:42130 -> [::1]:37049 (proxy) 127.0.0.1:52822 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"540","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:08.232","ReadError":"writeto tcp 127.0.0.1:52822->127.0.0.1:32770: read tcp 127.0.0.1:52822->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:08.232","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:52822->127.0.0.1:32770: read tcp 127.0.0.1:52822->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:09] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "112: [::1]:42136 -> [::1]:37049 (proxy) 127.0.0.1:52830 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"347","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:09.232","ReadError":"writeto tcp 127.0.0.1:52830->127.0.0.1:32770: read tcp 127.0.0.1:52830->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:09.232","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:52830->127.0.0.1:32770: read tcp 127.0.0.1:52830->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:09] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "113: [::1]:42146 -> [::1]:37049 (proxy) 127.0.0.1:52848 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"347","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:09.290","ReadError":"writeto tcp 127.0.0.1:52848->127.0.0.1:32770: read tcp 127.0.0.1:52848->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:09.290","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:52848->127.0.0.1:32770: read tcp 127.0.0.1:52848->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:09] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Health report status for 'garnet' is Unhealthy.
| [2025-05-23T01:58:09] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Resource 'garnet' health check monitoring loop starting delay of 00:00:05.
| [2025-05-23T01:58:14] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "121: [::1]:42236 -> [::1]:37049 (proxy) 127.0.0.1:52912 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"347","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:14.294","ReadError":"writeto tcp 127.0.0.1:52912->127.0.0.1:32770: read tcp 127.0.0.1:52912->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:14.294","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:52912->127.0.0.1:32770: read tcp 127.0.0.1:52912->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:14] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "122: [::1]:42224 -> [::1]:37049 (proxy) 127.0.0.1:52920 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"540","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:14.294","ReadError":"writeto tcp 127.0.0.1:52920->127.0.0.1:32770: read tcp 127.0.0.1:52920->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:14.294","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:52920->127.0.0.1:32770: read tcp 127.0.0.1:52920->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:14] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Health report status for 'garnet' is Unhealthy.
| [2025-05-23T01:58:14] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Resource 'garnet' health check monitoring loop starting delay of 00:00:05.
| [2025-05-23T01:58:15] Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler Information: all expected containers are connected to the network	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 18, "Network": "default-aspire-network-a616datjno", "Expected": 1, "Found": 1}
| [2025-05-23T01:58:19] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "128: [::1]:41456 -> [::1]:37049 (proxy) 127.0.0.1:57470 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"347","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:19.299","ReadError":"writeto tcp 127.0.0.1:57470->127.0.0.1:32770: read tcp 127.0.0.1:57470->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:19.299","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:57470->127.0.0.1:32770: read tcp 127.0.0.1:57470->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:19] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Health report status for 'garnet' is Unhealthy.
| [2025-05-23T01:58:19] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Resource 'garnet' health check monitoring loop starting delay of 00:00:05.
| [2025-05-23T01:58:20] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "134: [::1]:41502 -> [::1]:37049 (proxy) 127.0.0.1:57502 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"540","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:20.232","ReadError":"writeto tcp 127.0.0.1:57502->127.0.0.1:32770: read tcp 127.0.0.1:57502->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:20.232","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:57502->127.0.0.1:32770: read tcp 127.0.0.1:57502->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:20] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "133: [::1]:41512 -> [::1]:37049 (proxy) 127.0.0.1:57492 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"347","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:20.232","ReadError":"writeto tcp 127.0.0.1:57492->127.0.0.1:32770: read tcp 127.0.0.1:57492->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:20.232","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:57492->127.0.0.1:32770: read tcp 127.0.0.1:57492->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:22] Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler Information: all expected containers are connected to the network	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 19, "Network": "default-aspire-network-a616datjno", "Expected": 1, "Found": 1}
| [2025-05-23T01:58:24] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "135: [::1]:41526 -> [::1]:37049 (proxy) 127.0.0.1:57508 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"347","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:24.231","ReadError":"writeto tcp 127.0.0.1:57508->127.0.0.1:32770: read tcp 127.0.0.1:57508->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:24.231","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:57508->127.0.0.1:32770: read tcp 127.0.0.1:57508->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:24] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "138: [::1]:41554 -> [::1]:37049 (proxy) 127.0.0.1:57532 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"540","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:24.304","ReadError":"writeto tcp 127.0.0.1:57532->127.0.0.1:32770: read tcp 127.0.0.1:57532->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:24.304","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:57532->127.0.0.1:32770: read tcp 127.0.0.1:57532->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:24] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "140: [::1]:41580 -> [::1]:37049 (proxy) 127.0.0.1:57544 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"347","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:24.305","ReadError":"writeto tcp 127.0.0.1:57544->127.0.0.1:32770: read tcp 127.0.0.1:57544->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:24.305","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:57544->127.0.0.1:32770: read tcp 127.0.0.1:57544->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:24] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Health report status for 'garnet' is Unhealthy.
| [2025-05-23T01:58:24] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Resource 'garnet' health check monitoring loop starting delay of 00:00:05.
| [2025-05-23T01:58:28] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "141: [::1]:60396 -> [::1]:37049 (proxy) 127.0.0.1:40430 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"540","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:28.231","ReadError":"writeto tcp 127.0.0.1:40430->127.0.0.1:32770: read tcp 127.0.0.1:40430->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:28.231","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:40430->127.0.0.1:32770: read tcp 127.0.0.1:40430->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:29] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "142: [::1]:60398 -> [::1]:37049 (proxy) 127.0.0.1:40440 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"540","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:29.307","ReadError":"writeto tcp 127.0.0.1:40440->127.0.0.1:32770: read tcp 127.0.0.1:40440->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:29.306","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:40440->127.0.0.1:32770: read tcp 127.0.0.1:40440->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:29] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "143: [::1]:60402 -> [::1]:37049 (proxy) 127.0.0.1:40448 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"347","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:29.307","ReadError":"writeto tcp 127.0.0.1:40448->127.0.0.1:32770: read tcp 127.0.0.1:40448->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:29.307","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:40448->127.0.0.1:32770: read tcp 127.0.0.1:40448->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:29] Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy Error: The outgoing TCP connection encountered a read error	{"Service": {"name":"garnet"}, "Stream": "146: [::1]:60438 -> [::1]:37049 (proxy) 127.0.0.1:40494 -> 127.0.0.1:32770", "Stats": {"BytesRead":"0","BytesWritten":"540","LastSuccessfulRead":"(zero)","LastSuccessfulWrite":"May 23 01:58:29.309","ReadError":"writeto tcp 127.0.0.1:40494->127.0.0.1:32770: read tcp 127.0.0.1:40494->127.0.0.1:32770: read: connection reset by peer","ReadErrorTimestamp":"May 23 01:58:29.309","WriteError":"(none)","WriteErrorTimestamp":"(zero)"}, "error": "writeto tcp 127.0.0.1:40494->127.0.0.1:32770: read tcp 127.0.0.1:40494->127.0.0.1:32770: read: connection reset by peer"}
| [2025-05-23T01:58:29] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Health report status for 'garnet' is Unhealthy.
| [2025-05-23T01:58:29] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Resource 'garnet' health check monitoring loop starting delay of 00:00:05.
| [2025-05-23T01:58:30] Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler Information: all expected containers are connected to the network	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 20, "Network": "default-aspire-network-a616datjno", "Expected": 1, "Found": 1}
| [2025-05-23T01:58:32] Aspire.Hosting.Dcp.DcpExecutor Debug: Cancellation received while watching Container resources.
| [2025-05-23T01:58:32] Aspire.Hosting.Dcp.DcpExecutor Debug: Stopped watching Container resources.
| [2025-05-23T01:58:32] Aspire.Hosting.Dcp.DcpExecutor Debug: Cancellation received while watching Executable resources.
| [2025-05-23T01:58:32] Aspire.Hosting.Dcp.DcpExecutor Debug: Stopped watching Executable resources.
| [2025-05-23T01:58:32] Aspire.Hosting.Dcp.DcpExecutor Debug: Cancellation received while watching Service resources.
| [2025-05-23T01:58:32] Aspire.Hosting.Dcp.DcpExecutor Debug: Stopped watching Service resources.
| [2025-05-23T01:58:32] Aspire.Hosting.Dcp.DcpExecutor Debug: Cancellation received while watching Endpoint resources.
| [2025-05-23T01:58:32] Aspire.Hosting.Dcp.DcpExecutor Debug: Stopped watching Endpoint resources.
| [2025-05-23T01:58:32] Aspire.Hosting.Dcp.DcpExecutor Debug: Log streaming for garnet-aargznfx was cancelled.
| [2025-05-23T01:58:32] Aspire.Hosting.Dcp.dcp.start-apiserver.api-server.adminHttpHandler Information: API server changed status	{"OldStatus": "Running", "NewStatus": "CleaningResources"}
| [2025-05-23T01:58:34] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Health report status for 'garnet' is Unhealthy.
| [2025-05-23T01:58:34] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Health reports for 'garnet' have changed. Publishing updated reports.
| [2025-05-23T01:58:34] Aspire.Hosting.ApplicationModel.ResourceNotificationService Trace: Version: 15 Resource garnet/garnet-aargznfx update published: ResourceType = Container, CreationTimeStamp = 2025-05-23T01:56:55, State = { Text = Running, Style = (null) }, IsHidden = False, HeathStatus = Unhealthy, ResourceReady = False, ExitCode = (null), Urls = { tcp = tcp://localhost:37049 }, EnvironmentVariables = { GARNET_PASSWORD = n37Wu1k1PJgcQPqBym1QtE }, Properties = { container.image = ghcr.io/microsoft/garnet:1.0 container.id = 5b040373c9cda721c3c652a6a1f20205ee9f87f0a947805ae74f3d9bb4fb20c9 container.command = /bin/sh container.args = /bin/sh, -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD container.ports = 6379 container.lifetime = Session resource.appArgs = -c, /app/GarnetServer --auth Password --password $GARNET_PASSWORD resource.appArgsSensitivity = 0, 0 }, HealthReports = { garnet_check = Unhealthy }, Commands = { resource-start (Start) = Hidden resource-stop (Stop) = Enabled resource-restart (Restart) = Enabled }
| [2025-05-23T01:58:34] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Resource 'garnet' health check monitoring loop starting delay of 00:00:05.
| [2025-05-23T01:58:38] Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler Information: all expected containers are connected to the network	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 21, "Network": "default-aspire-network-a616datjno", "Expected": 1, "Found": 1}
| [2025-05-23T01:58:39] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Health report status for 'garnet' is Unhealthy.
| [2025-05-23T01:58:39] Aspire.Hosting.Health.ResourceHealthCheckService Trace: Resource 'garnet' health check monitoring loop starting delay of 00:00:05.

Report for PRs and Rolling builds

Last 10 failures:

Run date	Test Name
May 23 1 AM	[linux] Aspire.Hosting.Garnet.Tests.GarnetFunctionalTests.VerifyGarnetResource
May 22 11 PM	[linux] Aspire.Hosting.Garnet.Tests.GarnetFunctionalTests.VerifyGarnetResource
May 22 9 PM	[linux] Aspire.Hosting.Garnet.Tests.GarnetFunctionalTests.VerifyGarnetResource

Summary of the failures

Last 24 hrs	Last 7 days	Last 30 days
3	3	3

-- Updated on 5/23/2025 5:24:08 PM UTC

[danegsta]

This was fixed in #9477 and was a legitimate issue with the Garnet image.

[danegsta]

I opened an issue on the garnet repo related to these test failures: microsoft/garnet#1220

[danegsta]

Do we track success vs. failure rate for diagnosing flaky tests? It seems we may want to require a certain success rate before a test can be quarantined to avoid suppressing real regressions as noise. At least for any tests that involve containers since the risk of an external regression goes up.

[radical]

Do we track success vs. failure rate for diagnosing flaky tests?

There is on-going work for this. Right now I'm tracking the success vs failure rate of the quarantined tests in #8813 . And potential flaky tests in #8813 (comment) . I will be adding more here to track deterministic failures that start happening on main.

It seems we may want to require a certain success rate before a test can be quarantined to avoid suppressing real regressions as noise. At least for any tests that involve containers since the risk of an external regression goes up.

Quarantining helps to unblock main and PRs, meanwhile we still have the tests running. And we can watch the report to see if there is 100% failure rate to identify bugs vs flaky tests. This is the rough approach that I have been following. But none of it is set in stone, so I'm open to ideas/suggestions/discussion.

cc @joperezr @davidfowl @eerhardt @danmoseley

[radical]

Also, I'm improving the report to better track the failure rates.

[radical]

This was fixed in #9477 and was a legitimate issue with the Garnet image.

Thank you, closing the issue now.