[WIP] Finish isolating sessions to each client

[sipsma]

This is the last part of the saga around #6747 that fully isolates sessions to each client and thus enables support for passing host unix sockets to modules, plugging some existing holes in our secret isolation, etc.

This PR doesn't have all that yet, been in the process of rebuilding it cleanly after an initial messy prototype. Sending it out anyways because what I have here seems to fix the problem in #6914, so may want to merge a minimum safe changeset here early.

• The only extra thing strictly needed to safely merge is isolating the secret store to each client; in the current state of things nested execs end up with a shared secret store from their parent caller, which is probably a breaking change (though it's complicated, not 100% sure)
• Brief description of changes so far:
  • Consolidate module caller digest to just be the client ID. This is a pretty nice simplification from before since we less IDs to think about and pass around, but especially simplifies the rest of the pending changes
  • Share the server with nested execs, whereas previously they got their own. This is again just a generally nice simplification made possible by the fact that we can now isolate their session attachables.

@vito unfortunately these are the changes I was referring to that may cause rebase headaches with your OTEL PR (and there's more coming on top of the commits already here), so we'll have to sort that out if we merge this soon.

[sipsma]

Note to self: fix this doc string; the client ID is now what used to be ModuleCallerDigest, so it's actually a non-unique content hash for all clients other than "main callers" from the host.

[sipsma]

Just to update, I extracted a bit of this out to #7213 in order to get bug fixes in faster. Also been chipping away locally at a re-work of this PR locally as I got thing like 95% working but then hit a corner case that called for a different approach.

I'm trying to do it in such a way that it cleans up as many related things as possible since we accumulated quite a bit of cruft over the last 6ish months adding module support. The biggest things that I may end up incorporating here are:

1. Consolidate BuildkitController and DaggerServer into one thing
   • Even though the goal of this PR is to isolate client sessions more, we currently have DaggerServer, dagql.Server, Query, BuildkitClient and more things that all need to be isolated, which can easily result in profound convolution. So removing entities from that list by making them shared ends up simplifying quite a bit
2. Removing ftp_proxy hack and instead taking advantage of the new custom Executor added recently
   • This should help simplify the way sessions get setup and served to modules/nested-execs which is obviously tied into the changes here

Obviously don't want the scope of this to grow out of control either, so seeing what makes sense to do here still and what can be follow ups.

[github-actions]

This PR is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions]

This PR was closed because it has been stalled for 7 days with no activity.