UX improvement: godebug <run, test, build>

[jeremyschlatter]

Currently, godebug requires you to overwrite your files with the code-generated versions. It would be a big UX improvement to generate the files in a temporary directory and run them in a single step, like go test and go coverage.

[omeid]

The generated source can be perhaps kept under $GOPATH/godebug/$IMPORT/PATH? This will allow to just scan directory for changes against the counterparts under $GOPATH/godebug and only regenerate source if you must, of course, this means some changes to the import path of projects though.

What do you think?

[jeremyschlatter]

That's a nice idea. I had been working on a version that throws away the generated code every time, but I think something like this can work.

We can do it without rewriting import paths, too. $GOPATH can point to multiple directories. We can have a special godebug directory that contains generated source in godebug/src and compiled versions in godebug/pkg. Then when the user runs godebug <run | test | build>, we can set GOPATH=<godebug-dir>:$GOPATH in the environment for the go command and it will use the debug-instrumented packages instead of the normal ones. This avoids both regenerating and recompiling code for packages that haven't changed.

I want godebug to support instrumenting a subset of the packages in the program, though, and that subset can be different each time the user runs the command. So I want the go command to ignore any packages in the godebug directory that the user has not chosen to instrument for this run. My proposal for that is to recursively hide all of the subdirectories of the godebug directory, probably by prepending '.' to their names, and then temporarily unhiding the directories we want to instrument just before running the go command.

A downside of my proposal is that concurrent runs of the godebug tool will interfere with each other. One workaround for that is to have godebug check a lock file on startup, and if it finds another instance running it does all of its work in a new temporary directory.

[omeid]

I like the idea of multi GOPATH better.

The issue of concurrency can be mitigated with having a GOPATH per project under godebug with packages selected for instrumentation linked from the godebug/{src,pkg}

[glycerine]

How ever you do it, I would make it really easy to know, in fact it should be blindingly obvious, if a binary contains any debug information from godebug at all.

You don't want to accidentally release a build with debug info in it. Both for security and for performance.

[jeremyschlatter]

@glycerine good point. I thought about this a bit and have two proposals:

1. godebug build always outputs a binary file that ends in ".godebug"
2. The godebug library has an init() function that prints a prompt and waits for confirmation before continuing. This prompt is disabled for any godebug test and godebug run commands that do not produce binaries. Example prompt is something like:

Welcome to godebug!
For help, type "help". To run the program until the first breakpoint, type "run".
(godebug) _

If stdin is closed, the program is halted.

Thoughts?

[jeremyschlatter]

@omeid that's a good idea. A downside of GOPATH directory per package is that it results in a quadratic search for packages. For example, if a build transitively includes 100 packages and all of them are godebug-enabled, then the go tool will do 100 package lookups that each check on average 50 directories.

But even for 100 packages this isn't much work. It would take thousands of packages, all instrumented in the same build, for this to become a problem. That seems unrealistic for now.

[jeremyschlatter]

Another factor worth noting: the go tool supports packages outside of the GOPATH workspace. It would be nice for godebug to support them, too. I think we should use temporary directories for that case.

[omeid]

@jeremyschlatter I don't see how you would be looking up more than one directory, you keep the instrumented source in godebug/src and only symlink them to the required projects. This way there will be only one instrumented source of every project at a time.

[jeremyschlatter]

@omeid ah, I think I misunderstood your suggestion. Is this what you are envisioning?

my-real-gopath/
    src/
        pkg1/
        pkg2/
        pkg3/
    pkg/
        pkg1.a
        pkg2.a
        pkg3.a
godebug/
    src/
        pkg1/
        pkg2/
        pkg3/
    pkg/
        pkg1.a
        pkg2.a
        pkg3.a
tmp/
    synthesized-gopath-1/
        src/
            pkg2/  <- symlink to godebug/src/pkg2
            pkg3/  <- symlink to godebug/src/pkg3
        pkg/
            pkg2.a  <- symlink to godebug/pkg/pkg2.a
            pkg3.a  <- symlink to godebug/pkg/pkg3.a
    synthesized-gopath-2/
        src/
            pkg3/  <- symlink to godebug/src/pkg3
        pkg/
            pkg3.a  <- symlink to godebug/pkg/pkg3.a

Each time godebug runs it generates a temporary GOPATH directory that contains symlinks to only the packages that we want to instrument, then prepends that directory to the user's GOPATH. The go tool does at most one extra lookup for each package.

^ This is cleaner than what I was thinking. Is this the same thing you were imagining?

[omeid]

Yup, that is exactly what I meant by having a GOPATH per project under godebug with packages selected for instrumentation linked from the godebug/{src,pkg}.

[glycerine]

@jeremyschlatter Sounds good--together those two precautions should catch most unintended debug -compiles.

[jeremyschlatter]

godebug run and godebug test have now been implemented, using temporary directories for now.

run: 5e8843d
test: b1c7171