feat: improve engine session loading logging

[TomChv]

Add few lines during engine loading to update user. This PR update Typescript and Go SDK.

Related to: #4820

Example on Typescript SDK

yarn start
yarn run v1.22.19
$ node --loader ts-node/esm ./index.mts
(node:97293) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
Starting Dagger session...
Dagger session started! Establishing connection with the SDK...
Connection established with Dagger session version 0.6.3

Example on Go SDK

go run ./test.go                                                    
Starting Dagger session...
Dagger session started! Establishing connection with the SDK...
Connection established with Dagger session version 0.6.3!
3: resolve image config for docker.io/library/alpine:latest

@gerhard Should I add a change in the release note with this PR? I think it's a really small change but it can be useful!

Note: Originally, I tried to do something with a progress bar but it became really complex for just a logging issue, I went back to something simpler but still effective!

@helderco I saw you already did some work on logging, should I also add these print in the Python SDK?

EDIT: NEW FORMAT

Creating new Engine session... OK!
Establishing connection to Engine... OK!

And in case of CLI downloading:

CLI not found, downloading it... OK!
Creating new Engine session... OK!
Establishing connection to Engine... OK!

[gerhard]

@TomChv: Should I add a change in the release note with this PR?

If it's a user-facing change, then yes. Remember to add it in as many places as needed (both sdk/go & sdk/nodejs in this case). FWIW: #5408

TIP: changie new --kind "Fixed" --body "Show when Engine is loading in the output" --custom "Author=TomChv" --custom "PR=5488"

[gerhard]

Picking this one up now.

[gerhard]

I pushed a wording change to the Node.js SDK. Does this read better to you @TomChv?

Before:

Starting Dagger session...
Dagger session started! Establishing connection with the SDK...
Connection established with Dagger session version 0.0.0

After:

Creating new Engine session... OK!
Establishing connection to Engine... OK!

---

I also removed the the SDK version from the output since it will be confusing if/when the Engine/CLI version differs from the SDK version.

WDYT?

[TomChv]

@gerhard LGTM, let's merge it after I also update the Go SDK with the same changes

[gerhard]

After the last few changes, this looks good to me! WDYT @TomChv?

I would be very keen to get @helderco & @vito's eyes on this. The changes in this PR feel just as much mine as they are yours @TomChv. As a result, I will abstain from approving.

[helderco]

You're missing the download step (i.e., when no CLI is found in the cache, it's downloaded).

[gerhard]

Is this one @helderco linked to #3806 by any chance? What about #3838?

[helderco]

It was #4186 as a follow up to #3806 but closed in favor of #4820.

[TomChv]

@gerhard @helderco I have applied your comments! 🚀

[helderco]

Almost there 🙂

[helderco]

It's very noisy when running multiple connects (e.g., in tests):

❯ go test
Downloading CLI... Downloading CLI... Downloading CLI... Downloading CLI... Downloading CLI... Downloading CLI... Downloading CLI... Downloading CLI... Downloading CLI... Downloading CLI... OK!
Creating new Engine session... OK!
Establishing connection to Engine... OK!
OK!
Creating new Engine session... OK!
Establishing connection to Engine... Creating new Engine session... OK!
Establishing connection to Engine... OK!
OK!
Creating new Engine session... OK!
Establishing connection to Engine... OK!
Creating new Engine session... OK!
OK!
Establishing connection to Engine... Creating new Engine session... OK!
Establishing connection to Engine... OK!
OK!
OK!
Creating new Engine session... OK!
Establishing connection to Engine... Creating new Engine session... Creating new Engine session... OK!
Establishing connection to Engine... Creating new Engine session... OK!
Establishing connection to Engine... OK!
Establishing connection to Engine... OK!
OK!
OK!
OK!
OK!
OK!
OK!
OK!
OK!
OK!
PASS
ok  	dagger.io/dagger	46.922s

In this case, the OK!s don't match.

[helderco]

It even shows in dagger run because of tests using LogOutput:

However, I'm using just go test here, not ./hack/make where the nested dev engine is. I expected the existing session to be detected.

EDIT: It's because of TestProvision.

[TomChv]

This is why I wanted to add a special flag so we can explicitely trigger the display of the loading

[helderco]

Python implementation:

• Python: add status updates during automatic provisioning #5522

[helderco]

We can do follow ups to improve on this, but lgtm for now. Rather have the fixed test, but non blocking.

[helderco]

One approach I took in Python was to only do this on TTY. The intent is to show that something's happening when it's run interactively, not when writing to a file for example.

I understand why you wanted the flag, but I'm more inclined for any smart detection we can make, up to a general escape hatch for the default of showing the output. Not a flag for explicitly enabling it. In Python there's an escape hatch, but not a specific setting for it either. And detecting for TTY probably avoids some of situations where you'd want to escape it anyway.

As for #5488 (comment), that test is testing concurrent provisioning for the automatic provisioning feature in the SDK, so it explicitly ignores an existing session. I say it's not a big deal in this case and we can remove LogOutput from the test. We'll be focusing more on the CLI experience going forward so hopefully less and less users will be running off the SDK directly in the future.

In the meantime this is the problematic situation that remains. It's if someone is doing multiple Connects when executing via the SDK, so that provisioning logging will be all over the place. I've seen it being done in a complex python SDK codebase, especially in a test suite that uses dagger, so I wanted to make sure I had that covered. Two things help there: 1) status update is replaced in line and disappears when run interactively; 2) No status updates are printed when running non-interactively (e.g., log_output to a file).

However, in this PR I think we can cross that bridge when we get there and improve then. This already feels like an improvement, and you can add the tty check as well.