New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hot compile is very slow #290

Closed
cswank opened this Issue Aug 30, 2013 · 26 comments

Comments

Projects
None yet
9 participants
@cswank

cswank commented Aug 30, 2013

I've been working on a new revel app and have been switching back and forth between 2 computers. On one computer the app builds in about 1 second, on the other it takes more than 30 seconds. By "the app builds" I mean after typing revel run /path/to/my/app and then refreshing the browser and waiting for the page to load.

They are both running ubuntu 13.04, are both new computers with decent processors, both have go 1.1.1 installed. I can get more info later (one of them is at work), but just wanted to see if anyone else has had problems with a slow build. It makes development a challenge.

@robfig

This comment has been minimized.

Contributor

robfig commented Sep 2, 2013

It should take 1-3 seconds. Perhaps you somehow installed Go without installing the standard library, and it's rebuilding the whole thing every time? (Which is about 30 seconds)

On the slow machine, I would suggest removing your existing Go installation and following the instructions to install from source:
http://golang.org/doc/install/source

Additionally, you can see what packages are being built if you make this change to revel source code, github.com/robfig/revel/harness/build.go:

Existing:

        buildCmd := exec.Command(goPath, "build",
            "-tags", buildTags,
            "-o", binName, path.Join(revel.ImportPath, "app", "tmp"))

Add a -v:

        buildCmd := exec.Command(goPath, "build",
            "-tags", buildTags,
            "-v", 
            "-o", binName, path.Join(revel.ImportPath, "app", "tmp"))

That will cause all the packages built to be printed to the TRACE logger.

@cswank

This comment has been minimized.

cswank commented Sep 2, 2013

I followed both suggestions and am still getting the same results. I built go from source and changed my PATH and GOROOT to point to the new install. Here is what the build command output looks like after I updated it (I don't see evidence that the standard lib is being built):

INFO 2013/09/02 05:43:27 harness.go:157: Listening on :5000
TRACE 2013/09/02 05:43:30 harness.go:124: Rebuild
TRACE 2013/09/02 05:43:31 build.go:73: Exec: [/home/craig/go/bin/go build -tags gorp -v -o /home/craig/Projects/registry/container/bin/docker-registry bitbucket.com/cswank/docker-registry/app/tmp]
TRACE 2013/09/02 05:43:56 app.go:56: Exec app: /home/craig/Projects/registry/container/bin/docker-registry [/home/craig/Projects/registry/container/bin/docker-registry -port=34423 -importPath=bitbucket.com/cswank/docker-registry -runMode=dev]
INFO 2013/09/02 05:43:56 revel.go:286: Loaded module static
INFO 2013/09/02 05:43:56 revel.go:286: Loaded module testrunner
INFO 2013/09/02 05:43:56 revel.go:286: Loaded module jobs
INFO 2013/09/02 05:43:56 main.go:32: Running revel server
TRACE 2013/09/02 05:43:56 controller.go:363: Registered controller: Application
TRACE 2013/09/02 05:43:56 controller.go:363: Registered controller: Jobs
TRACE 2013/09/02 05:43:56 controller.go:363: Registered controller: Static
TRACE 2013/09/02 05:43:56 controller.go:363: Registered controller: TestRunner
TRACE 2013/09/02 05:43:56 controller.go:363: Registered controller: StorageApplication
TRACE 2013/09/02 05:43:56 controller.go:363: Registered controller: Index
TRACE 2013/09/02 05:43:56 controller.go:363: Registered controller: Tags
TRACE 2013/09/02 05:43:56 controller.go:363: Registered controller: Images
TRACE 2013/09/02 05:43:56 watcher.go:94: Watching: /home/craig/Projects/registry/container/src/bitbucket.com/cswank/docker-registry/app/views
TRACE 2013/09/02 05:43:56 watcher.go:94: Watching: /home/craig/Projects/registry/container/src/bitbucket.com/cswank/docker-registry/app/views/Application
TRACE 2013/09/02 05:43:56 watcher.go:94: Watching: /home/craig/Projects/registry/container/src/github.com/robfig/revel/templates
TRACE 2013/09/02 05:43:56 watcher.go:94: Watching: /home/craig/Projects/registry/container/src/github.com/robfig/revel/templates/errors
TRACE 2013/09/02 05:43:56 watcher.go:94: Watching: /home/craig/Projects/registry/container/src/github.com/robfig/revel/modules/testrunner/app/views
TRACE 2013/09/02 05:43:56 watcher.go:94: Watching: /home/craig/Projects/registry/container/src/github.com/robfig/revel/modules/testrunner/app/views/TestRunner
TRACE 2013/09/02 05:43:56 watcher.go:94: Watching: /home/craig/Projects/registry/container/src/github.com/robfig/revel/modules/jobs/app/views
TRACE 2013/09/02 05:43:56 watcher.go:94: Watching: /home/craig/Projects/registry/container/src/github.com/robfig/revel/modules/jobs/app/views/Jobs
TRACE 2013/09/02 05:43:56 watcher.go:72: Watching: /home/craig/Projects/registry/container/src/bitbucket.com/cswank/docker-registry/conf/routes
Go to /@tests to run the tests.
Go to /@jobs to see job status.
Listening on port 34423...
TRACE 2013/09/02 05:43:57 template.go:170: Refreshing templates from [/home/craig/Projects/registry/container/src/bitbucket.com/cswank/docker-registry/app/views /home/craig/Projects/registry/container/src/github.com/robfig/revel/templates /home/craig/Projects/registry/container/src/github.com/robfig/revel/modules/testrunner/app/views /home/craig/Projects/registry/container/src/github.com/robfig/revel/modules/jobs/app/views]
TRACE 2013/09/02 05:43:57 i18n.go:184: Unable to read locale cookie with name 'REVEL_LANG': http: named cookie not present
TRACE 2013/09/02 05:43:57 i18n.go:150: Found Accept-Language header value: en-us
TRACE 2013/09/02 05:43:57 i18n.go:184: Unable to read locale cookie with name 'REVEL_LANG': http: named cookie not present
TRACE 2013/09/02 05:43:57 i18n.go:150: Found Accept-Language header value: en-us
TRACE 2013/09/02 05:43:57 invoker.go:26: Binding: moduleName as string
TRACE 2013/09/02 05:43:57 invoker.go:26: Binding: prefix as string
TRACE 2013/09/02 05:43:57 invoker.go:26: Binding: filepath as string
TRACE 2013/09/02 05:43:57 i18n.go:184: Unable to read locale cookie with name 'REVEL_LANG': http: named cookie not present
TRACE 2013/09/02 05:43:57 i18n.go:150: Found Accept-Language header value: en-us
TRACE 2013/09/02 05:43:57 invoker.go:26: Binding: moduleName as string
TRACE 2013/09/02 05:43:57 invoker.go:26: Binding: prefix as string
TRACE 2013/09/02 05:43:57 invoker.go:26: Binding: filepath as string
^CTRACE 2013/09/02 05:44:01 app.go:86: Killing revel server pid 7788

@robfig

This comment has been minimized.

Contributor

robfig commented Sep 2, 2013

Ah, it doesn't print the output for a successful build. Please add a print to see it:

        buildCmd := exec.Command(goPath, "build",
            "-tags", buildTags,
            "-v",
            "-o", binName, path.Join(revel.ImportPath, "app", "tmp"))
        revel.TRACE.Println("Exec:", buildCmd.Args)

        output, err := buildCmd.CombinedOutput()
        revel.TRACE.Println(string(output))  // <---- 
@cswank

This comment has been minimized.

cswank commented Sep 2, 2013

The trace didn't seem to print anything different, so I broke my code with an unneeded import. Here is the output. I wonder if it is the sqlite3 stuff that is taking a long time? I used the booking app as a starting template.

ERROR 2013/09/02 12:40:21 build.go:80: code.google.com/p/go.net/websocket
github.com/howeyc/fsnotify
github.com/robfig/config
github.com/robfig/pathtree
github.com/streadway/simpleuuid
bitbucket.com/cswank/docker-registry/app/checksums
bitbucket.com/cswank/docker-registry/app/storage
github.com/mattn/go-sqlite3
github.com/robfig/cron
github.com/robfig/revel
bitbucket.com/cswank/docker-registry/app
bitbucket.com/cswank/docker-registry/app/controllers
bitbucket.com/cswank/docker-registry/tests
github.com/robfig/revel/modules/jobs/app/jobs
github.com/robfig/revel/modules/static/app/controllers

bitbucket.com/cswank/docker-registry/app/controllers

bitbucket.com/cswank/docker-registry/app/controllers/images.go:6: imported and not used: "os"
github.com/robfig/revel/modules/testrunner/app
github.com/robfig/revel/modules/jobs/app/controllers
github.com/robfig/revel/modules/testrunner/app/controllers

github.com/mattn/go-sqlite3

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.7/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.7.3-1ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-4.7/README.Bugs --enable-languages=c,c++,go,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.7 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.7 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --enable-plugin --with-system-zlib --enable-objc-gc --with-cloog --enable-cloog-backend=ppl --disable-cloog-version-check --disable-ppl-version-check --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.7.3 (Ubuntu/Linaro 4.7.3-1ubuntu1)
COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.7/:/usr/lib/gcc/x86_64-linux-gnu/4.7/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.7/:/usr/lib/gcc/x86_64-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.7/:/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib/:/lib/x86_64-linux-gnu/:/lib/../lib/:/usr/lib/x86_64-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-I' '/home/craig/Projects/registry/container/src/github.com/mattn/go-sqlite3' '-g' '-O2' '-fPIC' '-m64' '-pthread' '-o' '/tmp/go-build267370817/github.com/mattn/go-sqlite3/_obj/cgo.o' '-v' '-mtune=generic' '-march=x86-64'
/usr/lib/gcc/x86_64-linux-gnu/4.7/collect2 --sysroot=/ --build-id --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -z relro -o $WORK/github.com/mattn/go-sqlite3/_obj/cgo.o /usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu/crt1.o /usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/4.7/crtbegin.o -L/usr/lib/gcc/x86_64-linux-gnu/4.7 -L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../.. $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_main.o $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_export.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3.cgo2.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3_other.cgo2.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3.o -ldl -lgcc --as-needed -lgcc_s --no-as-needed -lpthread -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/4.7/crtend.o /usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu/crtn.o

github.com/mattn/go-sqlite3

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.7/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.7.3-1ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-4.7/README.Bugs --enable-languages=c,c++,go,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.7 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.7 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --enable-plugin --with-system-zlib --enable-objc-gc --with-cloog --enable-cloog-backend=ppl --disable-cloog-version-check --disable-ppl-version-check --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.7.3 (Ubuntu/Linaro 4.7.3-1ubuntu1)
COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.7/:/usr/lib/gcc/x86_64-linux-gnu/4.7/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.7/:/usr/lib/gcc/x86_64-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.7/:/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib/:/lib/x86_64-linux-gnu/:/lib/../lib/:/usr/lib/x86_64-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-I' '/home/craig/Projects/registry/container/src/github.com/mattn/go-sqlite3' '-g' '-O2' '-fPIC' '-m64' '-pthread' '-o' '/tmp/go-build267370817/github.com/mattn/go-sqlite3/_obj/_all.o' '-v' '-nostdlib' '-mtune=generic' '-march=x86-64'
/usr/lib/gcc/x86_64-linux-gnu/4.7/collect2 --sysroot=/ --build-id --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -z relro -o $WORK/github.com/mattn/go-sqlite3/_obj/_all.o -L/usr/lib/gcc/x86_64-linux-gnu/4.7 -L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../.. $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_export.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3.cgo2.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3_other.cgo2.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3.o -r /usr/lib/gcc/x86_64-linux-gnu/4.7/libgcc.a

@cswank

This comment has been minimized.

cswank commented Sep 2, 2013

Oh, didn't notice that first line on the output:

ERROR 2013/09/02 12:40:21 build.go:80: code.google.com/p/go.net/websocket

@cswank

This comment has been minimized.

cswank commented Sep 4, 2013

Since I'm not using the sqlite3 stuff, I removed these lines from the app.conf file:

db.import = github.com/mattn/go-sqlite3
db.driver = sqlite3
db.spec = :memory:

And now it compiles quickly. I'd still like to know why that takes so long when I have those lines in app.conf, and why it is only slow on one of my two computers.

@robfig

This comment has been minimized.

Contributor

robfig commented Sep 4, 2013

Very weird, I haven't seen that before.

If you run go build -x github.com/mattn/go-sqlite from the command line, does it still take that long? I assume so -- you would be able to see exactly which command takes all the time from that output. Maybe follow up on golang-nuts, if it isn't obvious what's wrong/different from that?

There is a revel thing here, though. Revel should not be re-building all of the dependent libraries every time, unless they change. I guess the default go build doesn't save the intermediaries? I'll investigate that situation -- perhaps it has to install things explicitly.

@cswank

This comment has been minimized.

cswank commented Sep 4, 2013

Building sqlite3 does indeed take a pretty long time (24 seconds). I've marked the actual gcc call that takes almost all of the time below by typing "#### long pause here###"

go build -x github.com/mattn/go-sqlite3
WORK=/tmp/go-build394621480
mkdir -p $WORK/github.com/mattn/go-sqlite3/_obj/
mkdir -p $WORK/github.com/mattn/
cd /home/craig/Projects/registry/container/src/github.com/mattn/go-sqlite3
/usr/local/go/pkg/tool/linux_amd64/cgo -objdir $WORK/github.com/mattn/go-sqlite3/_obj/ -- -I $WORK/github.com/mattn/go-sqlite3/_obj/ sqlite3.go sqlite3_other.go
/usr/local/go/pkg/tool/linux_amd64/6c -F -V -w -I $WORK/github.com/mattn/go-sqlite3/_obj/ -I /usr/local/go/pkg/linux_amd64 -o $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_defun.6 -D GOOS_linux -D GOARCH_amd64 $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_defun.c
gcc -I . -g -O2 -fPIC -m64 -pthread -print-libgcc-file-name
gcc -I . -g -O2 -fPIC -m64 -pthread -I $WORK/github.com/mattn/go-sqlite3/_obj/ -o $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_main.o -c $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_main.c
gcc -I . -g -O2 -fPIC -m64 -pthread -I $WORK/github.com/mattn/go-sqlite3/_obj/ -o $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_export.o -c $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_export.c
gcc -I . -g -O2 -fPIC -m64 -pthread -I $WORK/github.com/mattn/go-sqlite3/_obj/ -o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3.cgo2.o -c $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3.cgo2.c
gcc -I . -g -O2 -fPIC -m64 -pthread -I $WORK/github.com/mattn/go-sqlite3/_obj/ -o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3_other.cgo2.o -c $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3_other.cgo2.c
gcc -I . -g -O2 -fPIC -m64 -pthread -I $WORK/github.com/mattn/go-sqlite3/_obj/ -o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3.o -c ./sqlite3.c

long pause here

gcc -I . -g -O2 -fPIC -m64 -pthread -o $WORK/github.com/mattn/go-sqlite3/_obj/cgo.o $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_main.o $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_export.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3.cgo2.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3_other.cgo2.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3.o -v -ldl

github.com/mattn/go-sqlite3

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.7/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.7.3-1ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-4.7/README.Bugs --enable-languages=c,c++,go,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.7 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.7 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --enable-plugin --with-system-zlib --enable-objc-gc --with-cloog --enable-cloog-backend=ppl --disable-cloog-version-check --disable-ppl-version-check --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.7.3 (Ubuntu/Linaro 4.7.3-1ubuntu1)
COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.7/:/usr/lib/gcc/x86_64-linux-gnu/4.7/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.7/:/usr/lib/gcc/x86_64-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.7/:/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib/:/lib/x86_64-linux-gnu/:/lib/../lib/:/usr/lib/x86_64-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-I' '/home/craig/Projects/registry/container/src/github.com/mattn/go-sqlite3' '-g' '-O2' '-fPIC' '-m64' '-pthread' '-o' '/tmp/go-build394621480/github.com/mattn/go-sqlite3/_obj/cgo.o' '-v' '-mtune=generic' '-march=x86-64'
/usr/lib/gcc/x86_64-linux-gnu/4.7/collect2 --sysroot=/ --build-id --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -z relro -o $WORK/github.com/mattn/go-sqlite3/_obj/cgo.o /usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu/crt1.o /usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/4.7/crtbegin.o -L/usr/lib/gcc/x86_64-linux-gnu/4.7 -L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../.. $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_main.o $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_export.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3.cgo2.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3_other.cgo2.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3.o -ldl -lgcc --as-needed -lgcc_s --no-as-needed -lpthread -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/4.7/crtend.o /usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu/crtn.o
/usr/local/go/pkg/tool/linux_amd64/cgo -objdir $WORK/github.com/mattn/go-sqlite3/_obj/ -dynimport $WORK/github.com/mattn/go-sqlite3/_obj/cgo.o -dynout $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_import.c
/usr/local/go/pkg/tool/linux_amd64/6c -F -V -w -I $WORK/github.com/mattn/go-sqlite3/_obj/ -I /usr/local/go/pkg/linux_amd64 -o $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_import.6 -D GOOS_linux -D GOARCH_amd64 $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_import.c
gcc -I . -g -O2 -fPIC -m64 -pthread -o $WORK/github.com/mattn/go-sqlite3/_obj/_all.o $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_export.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3.cgo2.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3_other.cgo2.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3.o -v -Wl,-r -nostdlib /usr/lib/gcc/x86_64-linux-gnu/4.7/libgcc.a

github.com/mattn/go-sqlite3

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.7/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.7.3-1ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-4.7/README.Bugs --enable-languages=c,c++,go,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.7 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.7 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --enable-plugin --with-system-zlib --enable-objc-gc --with-cloog --enable-cloog-backend=ppl --disable-cloog-version-check --disable-ppl-version-check --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.7.3 (Ubuntu/Linaro 4.7.3-1ubuntu1)
COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.7/:/usr/lib/gcc/x86_64-linux-gnu/4.7/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.7/:/usr/lib/gcc/x86_64-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.7/:/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib/:/lib/x86_64-linux-gnu/:/lib/../lib/:/usr/lib/x86_64-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-I' '/home/craig/Projects/registry/container/src/github.com/mattn/go-sqlite3' '-g' '-O2' '-fPIC' '-m64' '-pthread' '-o' '/tmp/go-build394621480/github.com/mattn/go-sqlite3/_obj/_all.o' '-v' '-nostdlib' '-mtune=generic' '-march=x86-64'
/usr/lib/gcc/x86_64-linux-gnu/4.7/collect2 --sysroot=/ --build-id --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -z relro -o $WORK/github.com/mattn/go-sqlite3/_obj/_all.o -L/usr/lib/gcc/x86_64-linux-gnu/4.7 -L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.7/../../.. $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_export.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3.cgo2.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3_other.cgo2.o $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3.o -r /usr/lib/gcc/x86_64-linux-gnu/4.7/libgcc.a
/usr/local/go/pkg/tool/linux_amd64/6g -o $WORK/github.com/mattn/go-sqlite3/_obj/go.6 -p github.com/mattn/go-sqlite3 -D _/home/craig/Projects/registry/container/src/github.com/mattn/go-sqlite3 -I $WORK $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_gotypes.go $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3.cgo1.go $WORK/github.com/mattn/go-sqlite3/_obj/sqlite3_other.cgo1.go
/usr/local/go/pkg/tool/linux_amd64/pack grcP $WORK $WORK/github.com/mattn/go-sqlite3.a $WORK/github.com/mattn/go-sqlite3/_obj/go.6 $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_import.6 $WORK/github.com/mattn/go-sqlite3/_obj/_cgo_defun.6 $WORK/github.com/mattn/go-sqlite3/_obj/_all.o

@robfig

This comment has been minimized.

Contributor

robfig commented Sep 6, 2013

I'm afraid I can't help you -- perhaps if you sent an email to golang-nuts saying that sqlite3 took 30 seconds to build and hung on that one gcc command, they would know what the problem is. (At least one of the gcc authors/maintainers is a core Go contributor..)

Sorry about that!

@robfig robfig closed this Sep 11, 2013

@roblillack

This comment has been minimized.

roblillack commented Aug 15, 2014

I saw the same issue and I'm pretty sure it's somehow revel's fault.

If I run the buildCmd manually, it will take ~30s every time I run it—even without changes to the code. If I just change the “build” verb to “install”, any further “build” (and revel's hot reload) will be done in less than a second.

@ghost

This comment has been minimized.

ghost commented Aug 16, 2014

To elaborate, what's happening here is that:

  • the mattn/go-sqlite3 package has a 5 megabyte .c file in it
  • when building the package, this .c file is compiled
  • building a 5mb .c takes a while
  • unless you go install it, the package is built every time you build a package when depends on it.

https://groups.google.com/forum/#!topic/Golang-nuts/KexEyiy6PvA

So, no. It's not a revel related issue.

@robfig

This comment has been minimized.

Contributor

robfig commented Aug 16, 2014

Actually, I agree that revel should be using "go install" instead of "go build". Although this appears to be an unusual pathological case for slow builds, it would make all builds faster, even with just native Go code.

I believe it would be a 1 line change

@pushrax

This comment has been minimized.

Contributor

pushrax commented Aug 16, 2014

There is no -o flag for go install, always uses GOBIN :(

Edit: though, it could set GOBIN to app/tmp actually.

@robfig

This comment has been minimized.

Contributor

robfig commented Aug 16, 2014

Oh, even easier. "go build" now has a "-i" flag:

   The -i flag installs the packages that are dependencies of the target.

@brendensoares brendensoares added this to the v0.11 milestone Aug 18, 2014

@pushrax pushrax reopened this Aug 21, 2014

@pushrax pushrax self-assigned this Aug 21, 2014

@roblillack

This comment has been minimized.

roblillack commented Aug 27, 2014

Alas, even -i does not help in my case. If I completely clean out external deps and run the following command multiple times:

$ time go build -v -i MYAPP/app/tmp

… the build will take ~40s every time—even without changing any files. If I run the build for MYAPP/app/controllers instead, any succeeding build (being it for …/controllers or …/tmp) will finish in 0.5–2s.

Any idea, why this could make a difference?

@roblillack

This comment has been minimized.

roblillack commented Aug 27, 2014

Digging deeper, this is what is installed for building …/tmp with -i (or during a hot build):

pkg/darwin_amd64/code.google.com/p/go.net/websocket.a
pkg/darwin_amd64/github.com/agtorre/gocolorize.a
pkg/darwin_amd64/github.com/howeyc/fsnotify.a
pkg/darwin_amd64/github.com/revel/revel/harness.a
pkg/darwin_amd64/github.com/revel/revel/modules/testrunner/app/controllers.a
pkg/darwin_amd64/github.com/revel/revel.a
pkg/darwin_amd64/github.com/robfig/config.a
pkg/darwin_amd64/github.com/robfig/pathtree.a

When building …/controllers, in addition to the packages above, these get installed:

pkg/darwin_amd64/code.google.com/p/go.crypto/bcrypt.a
pkg/darwin_amd64/code.google.com/p/go.crypto/blowfish.a
pkg/darwin_amd64/code.google.com/p/gofpdf.a
pkg/darwin_amd64/github.com/coopernurse/gorp.a
pkg/darwin_amd64/github.com/mattn/go-sqlite3.a
pkg/darwin_amd64/github.com/revel/revel/modules/db/app.a

As you can see, none of the real dependencies of my app will be installed by the go build -i call in harness/build.go—and I don't have the slightest idea, why. :)

@brendensoares brendensoares added support and removed enhancement labels Sep 10, 2014

@brendensoares

This comment has been minimized.

Member

brendensoares commented Sep 10, 2014

@roblillack can you try a different machine/VM to see if the issue continues? Also, a different version of Go?

Can we agree that there are no Revel-based action items here? Seems to be a go command issue.

@brendensoares

This comment has been minimized.

Member

brendensoares commented Sep 14, 2014

Closing since there are no action items and no additional details. Reopen if needed. Thanks.

@kokizzu

This comment has been minimized.

kokizzu commented Nov 4, 2014

this also happened to me, it's painfully slow when I develop using Revel, on 3 different machine (OSes: Archlinux AMD, Manjaro Intel and Manjaro AMD)

here's some log of the action when starting up:

INFO  2014/11/04 10:42:35 build.go:92: Exec: [/usr/bin/go build -ldflags -X puis/app.APP_VERSION "git-c5f70d1-dirty" -tags  -v -o /home/kyz/Dropbox/go/bin/puis puis/app/tmp]                                                 
INFO  2014/11/04 10:42:44 build.go:94

when hot reload triggered:

INFO  2014/11/04 10:43:59 build.go:92: Exec: [/usr/bin/go build -ldflags -X puis/app.APP_VERSION "git-c5f70d1-dirty" -tags  -v -o /home/kyz/Dropbox/go/bin/puis puis/app/tmp]
INFO  2014/11/04 10:44:38 build.go:94

machine: i3-4150, 4GB RAM, Non-SSD hdd

I'm not using sqlite, just PostgreSQL with GORM, step by step tutorial:
http://kokizzu.blogspot.com/2014/11/how-to-start-revelgormpostgresql.html

@acrisci

This comment has been minimized.

Contributor

acrisci commented Nov 4, 2014

I can confirm installing sqlite with go install github.com/mattn/go-sqlite3 significantly reduced build time for my app.

@kokizzu

This comment has been minimized.

kokizzu commented Nov 5, 2014

Confirmed '___')

go get -u -v github.com/mattn/go-sqlite3

from 40+ to 1-2 seconds hot-reload..

@nabati

This comment has been minimized.

nabati commented Jan 3, 2015

+1 on "go install github.com/mattn/go-sqlite3".

Significantly reduced build time.

@pedromorgan

This comment has been minimized.

Member

pedromorgan commented Jan 3, 2015

Can someone explain why ?

So I can stick on the FAQ

@ghost

This comment has been minimized.

ghost commented Jan 3, 2015

@pedromorgan

This comment has been minimized.

Member

pedromorgan commented Jan 3, 2015

Yes apologies, I noticed that later..

Check the bottom of http://revel-docs.daffodil.uk.com/manual/faq.html

@roblillack

This comment has been minimized.

roblillack commented Jan 3, 2015

I still think that somehow the ‘dependency detection’ is broken. If you look at my comment above, you'll notice that all the missing packages are dependencies not used by revel, but my custom controllers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment