grid: changing to defered deletion #34832

alyssawilk · 2024-06-20T16:50:25Z

changing the wrapper callbacks and async callbacks in the grid to do deferred deletion.

currently we infer immediate response / async newStream from return codes.
This gets more complicated when we add in happy eyeballs, so instead we want to determine if we should return a cancelable handle based on if the caller has been informed of success or failure. This requires the objects to not be deleted under the stack of newStream hence the migration to defered delete.

Also doing a follow-up from the prior PR and creating connection pools explicitly.

Risk Level: high (object lifetime changes)
Testing: existing tests
Docs Changes: n/a
Release Notes: n/a

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>

alyssawilk · 2024-06-20T18:27:53Z

This one in not a hurry as I'm still working on tests for the final PR :-)

RyanTheOptimist

Nice! I think the getOrCreate is a nice cleanup.

RyanTheOptimist · 2024-06-21T00:05:48Z

source/common/http/conn_pool_grid.h

-  // TODO(alyssawilk) replace this now we have explicit pools.
-  virtual ConnectionPool::Instance* createNextPool();
+  // Creates the requested pool
+  virtual ConnectionPool::Instance* getOrCreateHttp3Pool();


Looking at the tests, it seems that we really only want to override the "create" part of this method (not the "get" part or the "setupPool" parts, etc). So how 'bout we cook up createHttp3Pool() and createHttp2Pool() and make them virtual. Then they can be called from the non-virutal getOrCreate(). This would simplify the mocking. (Alternatively we could pass two absl::AnyInvokables in the constructor and call them without using any virtual functions.

RyanTheOptimist · 2024-06-21T00:08:17Z

source/common/http/conn_pool_grid.cc

@@ -324,8 +321,7 @@ ConnectionPool::Cancellable* ConnectivityGrid::newStream(Http::ResponseDecoder&
    }
  } else {
    // Make sure the HTTP/2 pool is created.


I think we can probably delete this comment now since:

pool = getOrCreateHttp2Pool()

is super readable. Woo hoo!

RyanTheOptimist · 2024-06-21T00:09:22Z

source/common/http/conn_pool_grid.cc

@@ -356,8 +351,7 @@ void ConnectivityGrid::addIdleCallback(IdleCb cb) {

 void ConnectivityGrid::drainConnections(Envoy::ConnectionPool::DrainBehavior drain_behavior) {
  if (drain_behavior == Envoy::ConnectionPool::DrainBehavior::DrainAndDelete) {
-    // Note that no new pools can be created from this point on
-    // as createNextPool fast-fails if `draining_` is true.
+    // Note that no new pools should be created from this point on.


nit: "should be" => "will be" since this invariant is enforced.

I don't think it is, there's just an assert?
we used to handle null pools, now we'll create on drain.

RyanTheOptimist · 2024-06-21T00:10:14Z

source/common/http/conn_pool_grid.h

-  virtual ConnectionPool::Instance* createNextPool();
+  // Creates the requested pool
+  virtual ConnectionPool::Instance* getOrCreateHttp3Pool();
+  virtual ConnectionPool::Instance* getOrCreateHttp2Pool();


OOC, can these methods return null? If not, maybe they should return refs? (But I kinda think they can return null?)

they shouldn't return null but the callers want pointers so I didn't think changing the types made sense?

RyanTheOptimist · 2024-06-21T00:10:59Z

source/common/http/conn_pool_grid.h

-  // Creates the next pool in the priority list, or nullptr if all pools have been created.
-  // TODO(alyssawilk) replace this now we have explicit pools.
-  virtual ConnectionPool::Instance* createNextPool();
+  // Creates the requested pool


nit: How about "Returns the specified pool, which will be created if necessary"

RyanTheOptimist · 2024-06-21T00:12:20Z

source/common/http/conn_pool_grid.h

  public:
    WrapperCallbacks(ConnectivityGrid& grid, Http::ResponseDecoder& decoder,
                     ConnectionPool::Callbacks& callbacks, const Instance::StreamOptions& options);

+    bool hasNotifiedCaller() { return inner_callbacks_ == nullptr; }
+    void deleteIsPending() override { next_attempt_timer_.reset(); }


What's the relationship between the next attempt timer and delete pending?

commented inline

Oh! I see. I missed that this was a virtual method as part of Event::DeferredDeletable. Mind adding:
``// Event::DeferredDeletable:`

RyanTheOptimist · 2024-06-21T00:16:48Z

source/common/http/conn_pool_grid.cc

-  return ret;
+
+  // Return a handle if the caller hasn't yet been notified of success/failure.
+  // Note there may still be connection attempts, if we'e waiting on a slow H3 connection.


nit: "if we'e waiting on" => "if the TCP attempt is delayed and the H3 attempt is slow" (or somesuch)

go/avoid-we

RyanTheOptimist · 2024-06-21T00:19:11Z

test/common/http/conn_pool_grid_test.cc

-      return nullptr;
+  ConnectionPool::Instance* getOrCreateHttp3Pool() override {
+    if (!http3_pool_) {
+      http3_pool_.reset(createMockPool("http3"));


nit: http3_pool_ = createMockPool("http3") (ditto below) or alternatively if we go with making createFooPool() methods, then this is even simpler.

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>

RyanTheOptimist

Looks great. Two minor comments.

RyanTheOptimist · 2024-06-24T13:54:21Z

test/common/http/conn_pool_grid_test.cc

+  static ConnectionPool::Instance* forceGetOrCreateHttp3Pool(ConnectivityGrid& grid) {
+    return grid.getOrCreateHttp3Pool();
+  }
+  static ConnectionPool::Instance* forceGetOrCreateHttp2Pool(ConnectivityGrid& grid) {


nit: Since we're now doing "using ConnectivityGrid::getOrCreateHttp2Pool" do we need this helper function, or can we call grid.getOrCreateHttp2Pool() directly?

by default the test grid creates mock pools. This allows forcing a real grid's real pools. commented inline

RyanTheOptimist · 2024-06-24T13:55:27Z

source/common/http/conn_pool_grid.h

  public:
    WrapperCallbacks(ConnectivityGrid& grid, Http::ResponseDecoder& decoder,
                     ConnectionPool::Callbacks& callbacks, const Instance::StreamOptions& options);

+    bool hasNotifiedCaller() { return inner_callbacks_ == nullptr; }
+    void deleteIsPending() override { next_attempt_timer_.reset(); }


Oh! I see. I missed that this was a virtual method as part of Event::DeferredDeletable. Mind adding:
``// Event::DeferredDeletable:`

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>

changing the wrapper callbacks and async callbacks in the grid to do deferred deletion. currently we infer immediate response / async newStream from return codes. This gets more complicated when we add in happy eyeballs, so instead we want to determine if we should return a cancelable handle based on if the caller has been informed of success or failure. This requires the objects to not be deleted under the stack of newStream hence the migration to defered delete. Also doing a follow-up from the prior PR and creating connection pools explicitly. Risk Level: high (object lifetime changes) Testing: existing tests Docs Changes: n/a Release Notes: n/a Signed-off-by: Alyssa Wilk <alyssar@chromium.org> Signed-off-by: antoniovleonti <leonti@google.com>

changing the wrapper callbacks and async callbacks in the grid to do deferred deletion. currently we infer immediate response / async newStream from return codes. This gets more complicated when we add in happy eyeballs, so instead we want to determine if we should return a cancelable handle based on if the caller has been informed of success or failure. This requires the objects to not be deleted under the stack of newStream hence the migration to defered delete. Also doing a follow-up from the prior PR and creating connection pools explicitly. Risk Level: high (object lifetime changes) Testing: existing tests Docs Changes: n/a Release Notes: n/a Signed-off-by: Alyssa Wilk <alyssar@chromium.org> Signed-off-by: Fernando Cainelli <fernando.cainelli-external@getyourguide.com>

alyssawilk added 6 commits June 18, 2024 11:03

WIP

91a3b9d

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>

grid: cleaning up pools

17d4920

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>

backing out some changes

1391262

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>

Merge branch 'main' into clean_grid

67e4ed7

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>

comments

72576eb

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>

Merge branch 'main' into clean_grid

e2139ce

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>

alyssawilk marked this pull request as draft June 20, 2024 16:50

alyssawilk added 2 commits June 20, 2024 12:56

grid: change to defered delete

35c26a1

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>

followups

9837246

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>

alyssawilk force-pushed the delete_grid branch from e256422 to 9837246 Compare June 20, 2024 16:57

alyssawilk marked this pull request as ready for review June 20, 2024 18:27

alyssawilk assigned RyanTheOptimist Jun 20, 2024

RyanTheOptimist reviewed Jun 21, 2024

View reviewed changes

comments

08d1aff

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>

RyanTheOptimist previously approved these changes Jun 24, 2024

View reviewed changes

comment

eed8634

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>

alyssawilk dismissed RyanTheOptimist’s stale review via eed8634 June 24, 2024 14:02

alyssawilk added 2 commits June 24, 2024 10:06

comment

ae42da9

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>

force comment

724db5b

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>

RyanTheOptimist approved these changes Jun 24, 2024

View reviewed changes

alyssawilk enabled auto-merge (squash) June 24, 2024 18:34

alyssawilk merged commit 243787f into envoyproxy:main Jun 24, 2024
51 of 52 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

grid: changing to defered deletion #34832

grid: changing to defered deletion #34832

alyssawilk commented Jun 20, 2024

alyssawilk commented Jun 20, 2024

RyanTheOptimist left a comment

RyanTheOptimist Jun 21, 2024

alyssawilk Jun 24, 2024

RyanTheOptimist Jun 21, 2024

alyssawilk Jun 24, 2024

RyanTheOptimist Jun 21, 2024

alyssawilk Jun 24, 2024

RyanTheOptimist Jun 21, 2024

alyssawilk Jun 24, 2024

RyanTheOptimist Jun 21, 2024

alyssawilk Jun 24, 2024

RyanTheOptimist Jun 21, 2024

alyssawilk Jun 24, 2024

RyanTheOptimist Jun 24, 2024

alyssawilk Jun 24, 2024

RyanTheOptimist Jun 21, 2024

alyssawilk Jun 24, 2024

RyanTheOptimist Jun 21, 2024

alyssawilk Jun 24, 2024

RyanTheOptimist left a comment

RyanTheOptimist Jun 24, 2024

alyssawilk Jun 24, 2024

RyanTheOptimist Jun 24, 2024

grid: changing to defered deletion #34832

grid: changing to defered deletion #34832

Conversation

alyssawilk commented Jun 20, 2024

alyssawilk commented Jun 20, 2024

RyanTheOptimist left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RyanTheOptimist left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment