-
Notifications
You must be signed in to change notification settings - Fork 18.8k
Description
What version of Go are you using (go version)?
go version go1.9beta2 darwin/amd64
What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/bill/code/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/_v/4r515ktx08g5yr6dgkxhfyfr0000gn/T/go-build016620058=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
Code to reproduce the bug?
https://github.com/ardanlabs/gotraining/tree/master/topics/go/profiling/memcpu
What did you do?
$ go test -run none -bench . -benchtime 3s -memprofile mem.out
$ go tool pprof -alloc_space memcpu.test mem.out
(pprof) list algOne
File: memcpu.test
Type: alloc_space
Time: Jul 15, 2017 at 11:40am (MDT)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) list algOne
Total: 354.04MB
ROUTINE ======================== github.com/ardanlabs/gotraining/topics/go/profiling/memcpu.algOne in /Users/bill/code/go/src/github.com/ardanlabs/gotraining/topics/go/profiling/memcpu/stream.go
354.04MB 354.04MB (flat, cum) 100% of Total
. . 8:// The solution cannot meaningfully buffer to the end of the stream and
. . 9:// then process the replacement.
. . 10:package main
. . 11:
. . 12:import (
338.04MB 338.04MB 13: "bytes"
. . 14: "fmt"
. . 15: "io"
. . 16:)
. . 17:
. . 18:// data represents a table of input and expected output.
. . 19:var data = []struct {
. . 20: input []byte
. . 21: output []byte
. . 22:}{
. . 23: {[]byte("abc"), []byte("abc")},
. . 24: {[]byte("elvis"), []byte("Elvis")},
. . 25: {[]byte("aElvis"), []byte("aElvis")},
. . 26: {[]byte("abcelvis"), []byte("abcElvis")},
. . 27: {[]byte("eelvis"), []byte("eElvis")},
. . 28: {[]byte("aelvis"), []byte("aElvis")},
. . 29: {[]byte("aabeeeelvis"), []byte("aabeeeElvis")},
. . 30: {[]byte("e l v i s"), []byte("e l v i s")},
. . 31: {[]byte("aa bb e l v i saa"), []byte("aa bb e l v i saa")},
. . 32: {[]byte(" elvi s"), []byte(" elvi s")},
. . 33: {[]byte("elvielvis"), []byte("elviElvis")},
. . 34: {[]byte("elvielvielviselvi1"), []byte("elvielviElviselvi1")},
. . 35: {[]byte("elvielviselvis"), []byte("elviElvisElvis")},
. . 36:}
. . 37:
. . 38:// assembleInputStream combines all the input into a
. . 39:// single stream for processing.
. . 40:func assembleInputStream() []byte {
. . 41: var in []byte
. . 42: for _, d := range data {
. . 43: in = append(in, d.input...)
. . 44: }
. . 45: return in
. . 46:}
. . 47:
. . 48:// assembleOutputStream combines all the output into a
. . 49:// single stream for comparing.
. . 50:func assembleOutputStream() []byte {
. . 51: var out []byte
. . 52: for _, d := range data {
. . 53: out = append(out, d.output...)
. . 54: }
. . 55: return out
. . 56:}
. . 57:
. . 58:func main() {
. . 59: var output bytes.Buffer
. . 60: in := assembleInputStream()
. . 61: out := assembleOutputStream()
. . 62:
. . 63: find := []byte("elvis")
. . 64: repl := []byte("Elvis")
. . 65:
. . 66: fmt.Println("=======================================\nRunning Algorithm One")
. . 67: output.Reset()
. . 68: algOne(in, find, repl, &output)
. . 69: matched := bytes.Compare(out, output.Bytes())
. . 70: fmt.Printf("Matched: %v\nInp: [%s]\nExp: [%s]\nGot: [%s]\n", matched == 0, in, out, output.Bytes())
. . 71:
. . 72: fmt.Println("=======================================\nRunning Algorithm Two")
. . 73: output.Reset()
. . 74: algTwo(in, find, repl, &output)
. . 75: matched = bytes.Compare(out, output.Bytes())
. . 76: fmt.Printf("Matched: %v\nInp: [%s]\nExp: [%s]\nGot: [%s]\n", matched == 0, in, out, output.Bytes())
. . 77:}
. . 78:
. . 79:// algOne is one way to solve the problem.
. . 80:func algOne(data []byte, find []byte, repl []byte, output *bytes.Buffer) {
. . 81:
. . 82: // Use a bytes Buffer to provide a stream to process.
. . 83: input := bytes.NewBuffer(data)
. . 84:
. . 85: // The number of bytes we are looking for.
. . 86: size := len(find)
. . 87:
. . 88: // Declare the buffers we need to process the stream.
16MB 16MB 89: buf := make([]byte, size)
. . 90: end := size - 1
. . 91:
. . 92: // Read in an initial number of bytes we need to get started.
. . 93: if n, err := io.ReadFull(input, buf[:end]); err != nil {
. . 94: output.Write(buf[:n])
What did I expect?
Total: 339.03MB
ROUTINE ======================== github.com/ardanlabs/gotraining/topics/go/profiling/memcpu.algOne in /Users/bill/code/go/src/github.com/ardanlabs/gotraining/topics/go/profiling/memcpu/stream.go
339.03MB 339.03MB (flat, cum) 100% of Total
. . 78:
. . 79:// algOne is one way to solve the problem.
. . 80:func algOne(data []byte, find []byte, repl []byte, output *bytes.Buffer) {
. . 81:
. . 82: // Use a bytes Buffer to provide a stream to process.
325.03MB 325.03MB 83: input := bytes.NewBuffer(data)
. . 84:
. . 85: // The number of bytes we are looking for.
. . 86: size := len(find)
. . 87:
. . 88: // Declare the buffers we need to process the stream.
14MB 14MB 89: buf := make([]byte, size)
. . 90: end := size - 1
. . 91:
. . 92: // Read in an initial number of bytes we need to get started.
. . 93: if n, err := io.ReadFull(input, buf[:end]); err != nil {
. . 94: output.Write(buf[:n])
The new list command is not isolating itself to just the function that matches the regular expression. Showing all the code is a lot of noise.
Also
ROUTINE ========================
github.com/ardanlabs/gotraining/topics/go/profiling/memcpu.algOne in
/Users/bill/code/go/src/github.com/ardanlabs/gotraining/topics/go/profiling/memcpu/stream.go
Showing the source code file is nice but the entire path being listed twice is noise. This would have been better.
ROUTINE ========================
memcpu.algOne in github.com/ardanlabs/gotraining/topics/go/profiling/memcpu/stream.go