Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: cmd/compile: allow compile-time override of constants #63372

Closed
metux opened this issue Oct 4, 2023 · 11 comments
Closed

proposal: cmd/compile: allow compile-time override of constants #63372

metux opened this issue Oct 4, 2023 · 11 comments
Labels
Milestone

Comments

@metux
Copy link

metux commented Oct 4, 2023

Add compiler flags to override specific const values

Rationale: there are situations where certain aspects (eg. features, presets, etc) shall be customized at compile-time.
Common scenarios are eg. system/distro specific pathes, disable features. And for various reasons (eg. reduced footprint
on constrained/embedded systems, reduced possible attack surfaces, ...) the dead code of disabled features shall be eliminated.

Why existing methods aren't sufficient ?

  1. Linker option -X:

We already can tell the linker to override initial values for variables (but not constants).
Obviously, those can't help with dead code elimination, since the compiler can't know which actual value it will have.
OTOH, with constants, the compiler can do that, eg. that code

if (FeatureFooEnabled) {
    doSomething()
} else {
   doOther()
}

can be easily reduced either one of the branches (depending on whether FeatureFooEnabled is true or false).
Similarily on any const expression (eg. Foo > 12, or backend == "boo", ...).

Thus, we can just inject some preset values, forcing some possible code pathes to be practically dead, but those can't be eliminated yet. On large/complex projects, those practically dead code can make up large portion of the final binary
(e.g. measured in docker/containerd: systemd support and its long chain of dependency contributes about 20%)

  1. Conditional compile by tags

This is the currently only practical option for (compile-time) optional features. Sufficient in small projects, but can easily become insane to maintain in large projects w/ lots of dependencies, e.g.:

  • for each switch needs two extra files
  • only boolean -- choices of n > 2 options (usually) need to exploded to n * 2 files
  • can only work on values that can be transformed to bools (eg. not list of names/strings)

But there's more:

Build tags are global for the whole build, thus affect all dependencies as well - there's no scoping yet.
Therefore, tag naming must be done very carefully - one tag could have different meaning in different packages.
And when certain features depend on others, it can easily become tricky picking the right combinations.

  1. Code generation

As long as the affected code is only in main module itself, it's not hard to write a little generator for creating some little .go files defining some consts. But this won't help for within dependencies. Even if those also implement their own generators one by one, we'd still need a common protocol for naming / passing along the individual settings.

Proposed solution:

Add a compiler flag similar to the existing -X linker flag (also fully qualified names), but here affecting consts instead:

  1. Compiler has an list (actually map) of overrides (name & value pairs) - untyped literals
  2. Whenever an const is resolved to its value, it consults the overrides list and take value from there, if exists
  3. The original type is always retained - the compiler converts the literals to the original type
  4. If conversion impossible, emit an type mismatch error (build should certainly break here)
  5. If some values had been replaced, at least one code elimination phase is necessary (if it wouldn't come after, anyways)

Practical use examples:

package johndoe
const EnableFoo = true
...
func doWhatever() {
    ...
    if EnableFoo {
        doSomeComplicatedStuff()
        useMoreDeps()
    }
   ...

Compiling with: go build -Djohndoe.EnableFoo=false ...
will result in the const EnableFoo changed to false and the big branch in doWhatever() will be eliminated
entirely - as well as everything else that isn't called anymore.

Impact:

Everything (IMHO) can be implemented just inside the compiler (possibly in build tool, some bits for passing through the args), right at the point after the AST had been built and validated, right before dead code elimination starts. Some extra care needs to be taken for checking type compatibility, but that shouldn't be a big deal since const can only be scalars anyways.

No language change necessary, no risk for incompatibilities.

@metux metux added the Proposal label Oct 4, 2023
@gopherbot gopherbot added this to the Proposal milestone Oct 4, 2023
@renthraysk
Copy link

renthraysk commented Oct 4, 2023

Another option is use inlining.

func EnableFoo() bool { return false }

func doWhatever() {
    ...
    if EnableFoo() {
        doSomeComplicatedStuff()
        useMoreDeps()
    }
   ...

EnableFoo will get inlined, and the if body will be eliminated if EnableFoo returns false.

@ianlancetaylor ianlancetaylor changed the title proposal: compiler: allow compile-time override of constants proposal: cmd/compile: allow compile-time override of constants Oct 4, 2023
@metux
Copy link
Author

metux commented Oct 5, 2023

Another option is use inlining.

func EnableFoo() bool { return false }

func doWhatever() {
    ...
    if EnableFoo() {
        doSomeComplicatedStuff()
        useMoreDeps()
    }
   ...

EnableFoo will get inlined, and the if body will be eliminated if EnableFoo returns false.

Indeed it will be inlined, but I wonder how defining an function instead of const would be benefitial here.
It maybe could, if we already had a way to override individual functions in imported packages from within the main package - but I'm not aware of anything like that already existing.

@y1yang0
Copy link
Contributor

y1yang0 commented Oct 6, 2023

Just from a practical point of view, I think this is a good addition, as I often wish to add some assert codes(sanity check, either expensive or verbose, to make code more robust) but also want them to be zero-cost, enabled only in development and testing environments, e.g.

rewireEdges()
if Assert { // should be keep or remove during compile-time
    verifyDominance()
}

@zigo101
Copy link

zigo101 commented Oct 6, 2023

When I firstly used this feature, I did expected it to apply to constants instead of variables.

@erifan
Copy link

erifan commented Oct 8, 2023

It sounds a bit like the define directive of C/C++ to me.

@metux
Copy link
Author

metux commented Oct 12, 2023

It sounds a bit like the define directive of C/C++ to me.

Not quite, because "#define " defines macros that are processed before actual parsing (preprocessor) - it's pure text pattern replacement - they eg. know nothing about types.

Instead, I'm proposing not to use a preprocessor, but just let the compiler use different values (but still same types!) for some constants. No change to the language itself, neither to runtime library - just done inside the compiler.

@rsc
Copy link
Contributor

rsc commented Dec 4, 2023

This is already possible by using //go:build tags to selectively choose a file containing the constant you want. It does not seem worth the trouble to add more code to the compiler to provide more fine-grained capabilties, and I'm sure it would lead to all sorts of "interesting" build scripts, like go build -gcflags=-C=math.Pi=4.

@rsc
Copy link
Contributor

rsc commented Dec 4, 2023

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc
Copy link
Contributor

rsc commented Dec 9, 2023

Based on the discussion above, this proposal seems like a likely decline.
— rsc for the proposal review group

@rsc
Copy link
Contributor

rsc commented Dec 14, 2023

No change in consensus, so declined.
— rsc for the proposal review group

@metux
Copy link
Author

metux commented Jan 11, 2024

This is already possible by using //go:build tags to selectively choose a file containing the constant you want.

Not quite. It would at least require one extra file per choice value, and this quickly bloats up code size and hurting maintainability.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Declined
Development

No branches or pull requests

7 participants