expvar: make it possible to remove memstat and cmdline defaults

[as]

The expvar package contains an init function that registers the memstats and cmdline vars to the debug/vars http handler. The package does not provide a means to remove these defaults, forcing the consumer to deal with them. It would be nice if the user had a means to remove these defaults without resorting to unsafe or forking the package.

A benchmark I've added reveals a mild but unnecessary performance impact with the defaults (when the number of total expvar.Vars is expected to be small), but the real problem I think is aesthetic: My other vars have nothing to do with memory usage or the command line, so the memory profile is intrusive to the application.

We also can't use memstats and cmdline as named for our own purposes. This does not impact me personally.

goos: windows
goarch: amd64
pkg: expvar
BenchmarkExpvarHandler/cmdline.memstats-8               200000          7219 ns/op
BenchmarkExpvarHandler/none-8                            5000000           278 ns/op
BenchmarkExpvarHandler/user.10.vars-8                     300000          5130 ns/op
PASS
ok      expvar    4.822s

func BenchmarkExpvarHandler(b *testing.B) {
    for _, tc := range []struct {
        Name string
        Init func()
    }{
        {"cmdline.memstats", func() {}},
        {"none", func() { RemoveAll() }},
        {"user.10.vars", func() { for i := 0; i < 10; i++ { NewInt(fmt.Sprintf("i%d", i)) } }},
    } {
        tc.Init()
        b.Run(tc.Name, func(b *testing.B) {
            for n := 0; n < b.N; n++ {
                expvarHandler(writer{}, nil)
            }
        })
    }
}

type writer http.Header
func (w writer) WriteHeader(_ int)           {}
func (w writer) Write(p []byte) (int, error) { return len(p), nil }
func (w writer) Header() http.Header         { return http.Header(w) }

Interestingly, the package tests contain an unexported RemoveAll function. The recently-closed issue #27555 discusses it in detail, but does not mention the potential use for removing the default vars.

[agnivade]

/cc @bradfitz

[gopherbot]

Change https://golang.org/cl/200319 mentions this issue: expvar: make possible to delete all exported variables

[fgm]

While this issue is still open, changelog 200319 is marked abandoned and I cannot see any explanation either here or on https://golang.org/cl/200319 and PR #34832 was closed.

Was it just abandoned as old, or actually rejected ? It was even part of the 1.13, then 1.14, milestones before @rsc removed it from there.

That tiny patch moving RemoveAll from test to main code solves the automatic publishing of cmdline and memstats and provides a more general solution to reset the expvar configuration. Is there anything I can do to make this issue/CL live again ?

[ianlancetaylor]

The change was closed because it was old. That was a mistake that should not have happened. Once the change was closed, the GitHub pull request was closed, and the GitHub repo was deleted. So you, or somebody, would have to recreate the change from the data that is still in Gerrit. Sorry about that.

That said, the change introduces new API, and our usual process for that is a proposal, as outlined at https://go.dev/s/proposal-process. We can open a new issue, or change this issue into a proposal for the change, as you prefer.

[MuhamedUsman]

I have found this, calling this handler instead of expvar.Handler()

func (app *application) customVarHandler(w http.ResponseWriter, _ *http.Request) {
	w.Header().Set("Content-Type", "application/json")

	w.Write([]byte("{"))
	appendComma := false

	expvar.Do(func(kv expvar.KeyValue) {
		if kv.Key == "memstats" || kv.Key == "cmdline" {
			return // Skip memstats and cmdline
		}
		if appendComma {
			w.Write([]byte(","))
		} else {
			appendComma = true
		}

		b, _ := json.Marshal(kv.Key)
		w.Write(b)
                w.Write([]byte(":"))
	        w.Write([]byte(kv.Value.String())) // here .String() returns json compatible string so, no marshal

	})
	w.Write([]byte("}}"))
}

[mohanson]

Another way: create a new http router to hijack the default one.

serveMux := http.NewServeMux()
serveMux.HandleFunc("/debug/vars", func(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "application/json; charset=utf-8")
	vars := new(expvar.Map).Init()
	expvar.Do(func(kv expvar.KeyValue) {
		vars.Set(kv.Key, kv.Value)
	})
	vars.Delete("cmdline")  // Delete cmdline
	vars.Delete("memstats") // Delete memstats
	w.Write([]byte(vars.String()))
})
// For all http requests that are not /debug/vars, forward them to the default route.
serveMux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
	http.DefaultServeMux.ServeHTTP(w, r)
})


http.ListenAndServe("127.0.0.1:8080", serveMux)