New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: downloads follow plain-HTTP redirects even when the -insecure flag is not set #29591

Open
xftroxgpx opened this Issue Jan 6, 2019 · 19 comments

Comments

Projects
None yet
7 participants
@xftroxgpx
Copy link

xftroxgpx commented Jan 6, 2019

What version of Go are you using (go version)?

$ go version
go version devel +35f4ec152b Sat Jan 5 00:45:14 2019 +0000 linux/amd64

or
ArchLinux's community/go 2:1.11.4-1

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/xftroxgpx/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build832694890=/tmp/go-build -gno-record-gcc-switches"

What did you do?

EDIT: First, make sure outgoing tcp connections to port 80 are blocked / notallowed, in firewall! Otherwise you won't get the connection refused error, and thus won't realize the download was done over http.

$ cd $GOPATH
#note: NOT $GOPATH/src  !!!
$ git clone https://github.com/perkeep/perkeep.git perkeep.org
...
$ cd perkeep.org
$ time go run make.go
...
(error here)
$ time go run make.go
...
(same error here)
$ go clean
...
(same error here)

What did you expect to see?

no http fetches, ie. all fetches should've been done via https (because all sites support https)

What did you see instead?

go: bazil.org/fuse@v0.0.0-20160811212531-371fbbdaa898: unrecognized import path "bazil.org/fuse" (https fetch: Get http://bazil.org/fuse/?go-get=1: dial tcp 104.28.6.28:80: connect: connection refused)
go: error loading module requirements

That bazil.org is the first entry in $GOPATH/perkeep.org/go.mod (in go.sum too but it doesn't matter here), so if I change it to something else, it's reflected in the error message.

Everything works and no error is emited when I move the cloned dir perkeep.org from $GOPATH into $GOPATH/src

Related:
#29590 (comment)
perkeep/perkeep#1242 (comment)

@xftroxgpx xftroxgpx changed the title packages listed in go.mod are fetched via http port 80 even when -insecure arg isn't used packages listed in go.mod are fetched via http port 80 even when -insecure arg isn't used only when repo lives in $GOPATH instead of $GOPATH/src Jan 6, 2019

@agnivade agnivade changed the title packages listed in go.mod are fetched via http port 80 even when -insecure arg isn't used only when repo lives in $GOPATH instead of $GOPATH/src cmd/go: packages listed in go.mod are fetched via http port 80 even when -insecure arg isn't used only when repo lives in $GOPATH instead of $GOPATH/src Jan 7, 2019

@agnivade

This comment has been minimized.

Copy link
Member

agnivade commented Jan 7, 2019

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Jan 7, 2019

Everything works and no error is emited when I move the cloned dir perkeep.org from $GOPATH into $GOPATH/src

The behavior of GO111MODULE=auto changes when you are in $GOPATH/src, and the GOMOD="" line from your go env output seems to imply that GO111MODULE is still in auto mode (the default).

Please attach the output of go env from both of the working directories in which you executed go run.

CC @bradfitz

@bcmills bcmills added this to the Go1.13 milestone Jan 7, 2019

@xftroxgpx

This comment has been minimized.

Copy link

xftroxgpx commented Jan 7, 2019

Since I erased everything, I redid it now with go version go1.11.4 linux/amd64 from ArchLinux's go package 2:1.11.4-1.

Please attach the output of go env from both of the working directories in which you executed go run.

diff between them:

--- a	2019-01-07 16:12:54.158893618 +0100
+++ b	2019-01-07 16:12:43.825023532 +0100
@@ -1,4 +1,4 @@
-/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/perkeep.org 
+/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/src/perkeep.org 
 $ go env
 GOARCH="amd64"
 GOBIN=""
@@ -18,11 +18,12 @@ GCCGO="gccgo"
 CC="gcc"
 CXX="g++"
 CGO_ENABLED="1"
-GOMOD="/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/perkeep.org/go.mod"
+GOMOD=""
 CGO_CFLAGS="-g -O2"
 CGO_CPPFLAGS=""
 CGO_CXXFLAGS="-g -O2"
 CGO_FFLAGS="-g -O2"
 CGO_LDFLAGS="-g -O2"
 PKG_CONFIG="pkg-config"
-GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build450389250=/tmp/go-build -gno-record-gcc-switches"
+GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build836258175=/tmp/go-build -gno-record-gcc-switches"
+
them:
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/perkeep.org 
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/xftroxgpx/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/perkeep.org/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build450389250=/tmp/go-build -gno-record-gcc-switches"
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/src/perkeep.org 
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/xftroxgpx/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build836258175=/tmp/go-build -gno-record-gcc-switches"
full compilation output
xftroxgpx@z5 2019/01/07 16:04:41 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24115
!77043 1 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx 
$ cd $GOPATH
-----------
xftroxgpx@z5 2019/01/07 16:04:44 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24118
!77044 2 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath 
$ git clone https://github.com/perkeep/perkeep.git perkeep.org
Cloning into 'perkeep.org'...
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: git-remote-https[19447](full:'/usr/lib/git-core/git-remote-https') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
github.com
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: !! REENTRANT_NAME == 'gethostbyname2_r' hostname='github.com'
git-remote-https: git-remote-https[19447](full:'/usr/lib/git-core/git-remote-https') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('github.com') which was not transformed('github.com') as follows:
140.82.118.3 github.com
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 63355 (delta 0), reused 0 (delta 0), pack-reused 63354
Receiving objects: 100% (63355/63355), 43.97 MiB | 9.94 MiB/s, done.
Resolving deltas: 100% (37854/37854), done.
-----------
xftroxgpx@z5 2019/01/07 16:05:13 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24147
!77045 3 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath 
$ cd perkeep.org/
-----------
xftroxgpx@z5 2019/01/07 16:05:50 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24184
!77046 4 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/perkeep.org 
$ cd^C
-----------
xftroxgpx@z5 2019/01/07 16:06:28 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24221
!77046 4 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/perkeep.org 
$ time go run make.go
-bash: go: command not found

real	0m0.002s
user	0m0.002s
sys	0m0.000s
-----------
xftroxgpx@z5 2019/01/07 16:06:34 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24228
!77047 5 127  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/perkeep.org 
$ time go run make.go
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
golang.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('golang.org') which was not transformed('golang.org') as follows:
216.58.214.241 golang.org
go: finding github.com/cznic/zappy v0.0.0-20160723133515-2533cb5b45cc
go: finding github.com/cznic/kv v0.0.0-20170515202733-892ccf731fb7
go: finding github.com/go-sql-driver/mysql v1.4.1-0.20180719071942-99ff426eb706
go: finding github.com/inconshreveable/mousetrap v1.0.0
go: finding github.com/go-ini/ini v1.38.1
go: finding github.com/mattn/go-sqlite3 v1.6.0
go: finding github.com/garyburd/go-oauth v0.0.0-20180319155456-bca2e7f09a17
go: finding github.com/golang/snappy v0.0.0-20170215233205-553a64147049
go: finding github.com/miekg/dns v0.0.0-20161003181808-3f1f7c8ec9ea
go: finding golang.org/x/image v0.0.0-20171214225156-12117c17ca67
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
cloud.google.com
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('cloud.google.com') which was not transformed('cloud.google.com') as follows:
172.217.20.14 cloud.google.com
go: finding github.com/cznic/sortutil v0.0.0-20150617083342-4c7342852e65
go: finding github.com/gopherjs/gopherjs v0.0.0-20181004052249-0fee806ad497
go: finding github.com/cznic/lldb v1.1.0
go: finding github.com/syndtr/goleveldb v0.0.0-20180608030153-db3ee9ee8931
go: finding cloud.google.com/go v0.0.0-20161006024015-b70ccc799b9d
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
bazil.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('bazil.org') which was not transformed('bazil.org') as follows:
104.28.6.28 bazil.org
go: finding github.com/cznic/mathutil v0.0.0-20180214153908-5455a562bccb
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
bazil.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('bazil.org') which was not transformed('bazil.org') as follows:
104.28.6.28 bazil.org
go: bazil.org/fuse@v0.0.0-20160811212531-371fbbdaa898: unrecognized import path "bazil.org/fuse" (https fetch: Get http://bazil.org/fuse/?go-get=1: dial tcp 104.28.6.28:80: connect: connection refused)
go: finding github.com/plaid/plaid-go v0.0.0-20161222051224-02b6af68061b
go: finding github.com/kr/fs v0.1.0
go: finding github.com/shurcooL/sanitized_anchor_name v0.0.0-20150515002706-11a20b799bf2
go: finding golang.org/x/net v0.0.0-20171212005608-d866cfc389ce
go: finding github.com/bradfitz/grpc-go v0.0.0-20170203184515-188a132adcfba339f1f2d5da52498451341f9ee8
go: finding github.com/edsrzf/mmap-go v0.0.0-20170320065105-0bce6a688712
go: finding github.com/kardianos/osext v0.0.0-20151222153229-29ae4ffbc9a6
go: finding github.com/neelance/sourcemap v0.0.0-20151028013722-8c68805598ab
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
google.golang.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('google.golang.org') which was not transformed('google.golang.org') as follows:
216.58.214.241 google.golang.org
go: finding golang.org/x/text v0.0.0-20171102192421-88f656faf3f3
go: finding google.golang.org/genproto v0.0.0-20161115011420-08f135d1a31b
go: finding github.com/lib/pq v0.0.0-20130607063955-9afcd9aa7931
go: finding github.com/gorilla/websocket v1.2.0
go: finding github.com/gopherjs/jquery v0.0.0-20151020083714-fbbfc4bbe29a
go: finding google.golang.org/api v0.0.0-20170327174102-48e49d1645e2
go: finding github.com/pkg/errors v0.8.0
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
go4.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('go4.org') which was not transformed('go4.org') as follows:
212.47.227.56 go4.org
go: finding github.com/tomnomnom/linkheader v0.0.0-20160328204959-6953a30d4443
go: finding github.com/cznic/internal v0.0.0-20170905175358-4747030f7cf2
go: finding go4.org v0.0.0-20180413184151-a2a47940e6bc
go: finding github.com/rwcarlsen/goexif v0.0.0-20180518182100-8d986c03457a
go: finding github.com/kisielk/gotool v0.0.0-20161130080628-0de1eaf82fa3
go: finding github.com/mattn/go-mastodon v0.0.0-20180129050910-2ccbcfe14d7a
go: finding golang.org/x/time v0.0.0-20160202183820-a4bde1265759
go: finding github.com/fsnotify/fsnotify v1.4.2
go: finding github.com/FiloSottile/b2 v0.0.0-20170207175032-b197f7a2c317
go: finding github.com/russross/blackfriday v2.0.0+incompatible
go: finding google.golang.org/appengine v1.0.0
go: finding github.com/cznic/fileutil v0.0.0-20180108211300-6a051e75936f
go: finding github.com/gopherjs/jsbuiltin v0.0.0-20170427220125-67703bfb044e
go: finding github.com/hjfreyer/taglib-go v0.0.0-20151027170453-0ef8bba9c41b
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
rsc.io
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('rsc.io') which was not transformed('rsc.io') as follows:
216.239.32.21 rsc.io
go: finding rsc.io/qr v0.1.0
go: finding github.com/mailgun/mailgun-go v0.0.0-20171127222028-17e8bd11e87c
go: finding github.com/shurcooL/httpfs v0.0.0-20171119174359-809beceb2371
go: finding github.com/golang/protobuf v1.0.0
go: finding github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
gopkg.in
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('gopkg.in') which was not transformed('gopkg.in') as follows:
35.196.143.184 gopkg.in
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
honnef.co
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('honnef.co') which was not transformed('honnef.co') as follows:
138.197.72.227 honnef.co
go: finding gopkg.in/mgo.v2 v2.0.0-20160818020120-3f83fa500528
go: finding honnef.co/go/js/dom v0.0.0-20160310112645-24aa052bc5c6
go: finding golang.org/x/sync v0.0.0-20171101214715-fd80eb99c8f6
go: finding github.com/spf13/pflag v0.0.0-20151218134703-7f60f83a2c81
go: finding github.com/pkg/sftp v0.0.0-20180419200840-5bf2a174b604
go: finding golang.org/x/tools v0.0.0-20170915040203-e531a2a1c15f
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
myitcv.io
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('myitcv.io') which was not transformed('myitcv.io') as follows:
216.239.32.21 myitcv.io
go: finding github.com/tgulacsi/picago v0.0.0-20171229130838-9e1ac2306c70
go: finding myitcv.io v0.0.0-20180717150903-bf2c84553c78
go: finding rsc.io/pdf v0.0.0-20170302045715-1d34785eb915
go: finding golang.org/x/crypto v0.0.0-20180723164146-c126467f60eb
go: finding golang.org/x/oauth2 v0.0.0-20171226133531-197281d4e0ec
go: finding github.com/aws/aws-sdk-go v1.14.31
go: finding github.com/bradfitz/latlong v0.0.0-20140711231157-b74550508561
go: finding github.com/nf/cr2 v0.0.0-20140528043846-05d46fef4f2f
go: finding github.com/googleapis/gax-go v0.0.0-20161107002406-da06d194a00e
go: finding golang.org/x/sys v0.0.0-20180115085844-fff93fa7cd27
go: finding github.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86
go: finding github.com/spf13/cobra v0.0.0-20160322171042-c678ff029ee2
go: error loading module requirements

real	0m58.175s
user	1m15.052s
sys	0m11.878s
-----------
xftroxgpx@z5 2019/01/07 16:08:06 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24319
!77048 6 1  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/perkeep.org 
$ time go run make.go^C
-----------
xftroxgpx@z5 2019/01/07 16:09:46 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24420
!77048 6 1  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/perkeep.org 
$ cd ..
-----------
xftroxgpx@z5 2019/01/07 16:09:48 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24421
!77049 7 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath 
$ mkdir src
-----------
xftroxgpx@z5 2019/01/07 16:09:51 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24425
!77050 8 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath 
$ mv perkeep.org/ src/
-----------
xftroxgpx@z5 2019/01/07 16:09:55 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24428
!77051 9 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath 
$ cd src/perkeep.org/
-----------
xftroxgpx@z5 2019/01/07 16:09:57 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24430
!77052 10 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/src/perkeep.org 
$ time go run make.go
perkeep.org/vendor/myitcv.io/gogenerate
perkeep.org/vendor/myitcv.io/react/cmd/reactGen
gopherjs rebuilt at /home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/bin/gopherjs
generated /home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/src/perkeep.org/app/publisher/js/zsearch.go
perkeep.org/internal/rollsum
perkeep.org/pkg/fileembed/genfileembed
perkeep.org/pkg/constants
perkeep.org/vendor/go4.org/legal
perkeep.org/vendor/golang.org/x/crypto/openpgp/errors
perkeep.org/vendor/golang.org/x/crypto/cast5
perkeep.org/internal/netutil
perkeep.org/vendor/go4.org/errorutil
perkeep.org/vendor/go4.org/wkfs
perkeep.org/pkg/buildinfo
perkeep.org/internal/testhooks
perkeep.org/vendor/go4.org/readerutil
perkeep.org/vendor/go4.org/jsonconfig
perkeep.org/pkg/blob
perkeep.org/vendor/go4.org/syncutil
perkeep.org/vendor/go4.org/types
perkeep.org/vendor/golang.org/x/net/http2/hpack
perkeep.org/vendor/golang.org/x/text/transform
perkeep.org/internal/osutil
perkeep.org/vendor/golang.org/x/text/unicode/bidi
perkeep.org/vendor/golang.org/x/text/unicode/norm
perkeep.org/internal/hashutil
perkeep.org/vendor/cloud.google.com/go/internal
perkeep.org/vendor/golang.org/x/net/context
perkeep.org/pkg/blobserver
perkeep.org/vendor/golang.org/x/net/context/ctxhttp
perkeep.org/vendor/cloud.google.com/go/compute/metadata
perkeep.org/vendor/golang.org/x/text/secure/bidirule
perkeep.org/pkg/blobserver/protocol
perkeep.org/pkg/env
perkeep.org/vendor/golang.org/x/crypto/openpgp/armor
perkeep.org/vendor/golang.org/x/crypto/openpgp/elgamal
perkeep.org/vendor/golang.org/x/crypto/openpgp/s2k
perkeep.org/internal/lru
perkeep.org/internal/gpgagent
perkeep.org/vendor/golang.org/x/crypto/openpgp/packet
perkeep.org/pkg/blobserver/memory
perkeep.org/internal/pinentry
perkeep.org/pkg/camerrors
perkeep.org/vendor/github.com/bradfitz/latlong
perkeep.org/vendor/github.com/rwcarlsen/goexif/tiff
perkeep.org/vendor/go4.org/strutil
perkeep.org/vendor/github.com/rwcarlsen/goexif/exif
perkeep.org/vendor/go4.org/syncutil/singleflight
perkeep.org/vendor/golang.org/x/net/idna
perkeep.org/internal/magic
perkeep.org/pkg/types/camtypes
perkeep.org/vendor/github.com/gorilla/websocket
perkeep.org/vendor/golang.org/x/oauth2/internal
perkeep.org/vendor/golang.org/x/crypto/openpgp
perkeep.org/vendor/golang.org/x/oauth2
perkeep.org/vendor/golang.org/x/net/lex/httplex
perkeep.org/vendor/golang.org/x/net/http2
perkeep.org/vendor/go4.org/ctxutil
perkeep.org/pkg/jsonsign
perkeep.org/internal/geocode
perkeep.org/vendor/github.com/hjfreyer/taglib-go/taglib/id3
perkeep.org/vendor/go4.org/media/heif/bmff
perkeep.org/vendor/github.com/hjfreyer/taglib-go/taglib
perkeep.org/vendor/github.com/nf/cr2
perkeep.org/pkg/schema
perkeep.org/vendor/golang.org/x/image/tiff/lzw
perkeep.org/vendor/golang.org/x/image/tiff
perkeep.org/vendor/go4.org/media/heif
perkeep.org/internal/images/fastjpeg
perkeep.org/vendor/golang.org/x/image/math/f64
perkeep.org/internal/media
perkeep.org/vendor/golang.org/x/image/draw
perkeep.org/pkg/schema/nodeattr
perkeep.org/vendor/github.com/syndtr/goleveldb/leveldb/comparer
perkeep.org/vendor/github.com/syndtr/goleveldb/leveldb/storage
perkeep.org/pkg/client/android
perkeep.org/vendor/github.com/syndtr/goleveldb/leveldb/util
perkeep.org/pkg/types/serverconfig
perkeep.org/vendor/github.com/syndtr/goleveldb/leveldb/errors
perkeep.org/pkg/cmdmain
perkeep.org/vendor/github.com/syndtr/goleveldb/leveldb/iterator
perkeep.org/vendor/github.com/garyburd/go-oauth/oauth
perkeep.org/internal/httputil
perkeep.org/vendor/github.com/syndtr/goleveldb/leveldb/memdb
perkeep.org/pkg/sorted
perkeep.org/vendor/golang.org/x/net/xsrftoken
perkeep.org/pkg/fileembed
perkeep.org/pkg/auth
perkeep.org/pkg/blobserver/gethandler
perkeep.org/pkg/types/clientconfig
perkeep.org/pkg/jsonsign/signhandler
perkeep.org/clients/web/embed/keepy
perkeep.org/clients/web/embed/fontawesome
perkeep.org/pkg/blobserver/handlers
perkeep.org/clients/web/embed/leaflet
perkeep.org/internal/images/resize
perkeep.org/clients/web/embed/less
perkeep.org/clients/web/embed/opensans
perkeep.org/clients/web/embed/react
perkeep.org/internal/images
perkeep.org/internal/closure
perkeep.org/pkg/blobserver/files
perkeep.org/pkg/blobserver/local
perkeep.org/server/perkeepd/ui/closure
perkeep.org/server/perkeepd/ui
perkeep.org/pkg/index
perkeep.org/pkg/blobserver/localdisk
perkeep.org/pkg/cacher
perkeep.org/vendor/rsc.io/qr/gf256
perkeep.org/vendor/rsc.io/qr/coding
perkeep.org/vendor/golang.org/x/net/html/atom
perkeep.org/vendor/golang.org/x/net/html
perkeep.org/vendor/rsc.io/qr
perkeep.org/vendor/golang.org/x/text/encoding/internal/identifier
perkeep.org/vendor/golang.org/x/text/encoding
perkeep.org/vendor/golang.org/x/text/encoding/internal
perkeep.org/vendor/golang.org/x/text/encoding/charmap
perkeep.org/vendor/golang.org/x/text/encoding/japanese
perkeep.org/pkg/search
perkeep.org/vendor/golang.org/x/text/encoding/korean
perkeep.org/vendor/golang.org/x/text/encoding/simplifiedchinese
perkeep.org/vendor/golang.org/x/text/encoding/traditionalchinese
perkeep.org/vendor/golang.org/x/text/internal/utf8internal
perkeep.org/vendor/golang.org/x/text/runes
perkeep.org/vendor/golang.org/x/text/encoding/unicode
perkeep.org/vendor/golang.org/x/text/internal/tag
perkeep.org/vendor/golang.org/x/text/language
perkeep.org/pkg/client
perkeep.org/pkg/server/app
perkeep.org/pkg/importer/feed/atom
perkeep.org/pkg/importer/feed/rdf
perkeep.org/pkg/importer/feed/rss
perkeep.org/vendor/golang.org/x/oauth2/jws
perkeep.org/vendor/golang.org/x/sync/errgroup
perkeep.org/vendor/golang.org/x/time/rate
perkeep.org/vendor/golang.org/x/oauth2/jwt
perkeep.org/vendor/google.golang.org/api/googleapi/internal/uritemplates
perkeep.org/vendor/golang.org/x/oauth2/google
perkeep.org/vendor/google.golang.org/api/googleapi
perkeep.org/vendor/github.com/tgulacsi/picago
perkeep.org/pkg/server
perkeep.org/vendor/golang.org/x/text/encoding/htmlindex
perkeep.org/vendor/google.golang.org/api/gensupport
perkeep.org/vendor/golang.org/x/net/html/charset
perkeep.org/vendor/github.com/tomnomnom/linkheader
perkeep.org/vendor/github.com/plaid/plaid-go/plaid
perkeep.org/vendor/github.com/mattn/go-mastodon
perkeep.org/vendor/google.golang.org/api/drive/v3
perkeep.org/cmd/pk-get
perkeep.org/vendor/github.com/syndtr/goleveldb/leveldb/cache
perkeep.org/vendor/github.com/syndtr/goleveldb/leveldb/filter
perkeep.org/vendor/github.com/syndtr/goleveldb/leveldb/journal
perkeep.org/vendor/github.com/syndtr/goleveldb/leveldb/opt
perkeep.org/pkg/importer
perkeep.org/vendor/github.com/golang/snappy
perkeep.org/vendor/github.com/syndtr/goleveldb/leveldb/table
perkeep.org/internal/chanworker
perkeep.org/vendor/go4.org/lock
perkeep.org/vendor/github.com/cznic/fileutil
perkeep.org/pkg/importer/dummy
perkeep.org/pkg/importer/feed
perkeep.org/pkg/importer/flickr
perkeep.org/pkg/importer/picasa
perkeep.org/pkg/importer/instapaper
perkeep.org/pkg/importer/mastodon
perkeep.org/pkg/importer/gphotos
perkeep.org/pkg/importer/pinboard
perkeep.org/pkg/importer/plaid
perkeep.org/pkg/importer/swarm
perkeep.org/pkg/importer/twitter
perkeep.org/vendor/github.com/syndtr/goleveldb/leveldb
perkeep.org/vendor/github.com/cznic/mathutil
perkeep.org/pkg/importer/allimporters
perkeep.org/dev/devcam
perkeep.org/vendor/github.com/cznic/internal/slice
perkeep.org/vendor/github.com/cznic/internal/buffer
perkeep.org/vendor/github.com/edsrzf/mmap-go
perkeep.org/vendor/github.com/cznic/internal/file
perkeep.org/vendor/github.com/cznic/sortutil
perkeep.org/vendor/github.com/cznic/zappy
perkeep.org/pkg/sorted/leveldb
perkeep.org/vendor/github.com/cznic/lldb
perkeep.org/internal/leak
perkeep.org/pkg/blobserver/stats
perkeep.org/pkg/sorted/sqlkv
perkeep.org/vendor/google.golang.org/api/iterator
perkeep.org/pkg/sorted/sqlite
perkeep.org/vendor/github.com/golang/protobuf/proto
perkeep.org/vendor/golang.org/x/net/internal/timeseries
perkeep.org/vendor/golang.org/x/net/trace
perkeep.org/vendor/google.golang.org/grpc/codes
perkeep.org/vendor/google.golang.org/grpc/credentials
perkeep.org/vendor/google.golang.org/grpc/grpclog
perkeep.org/vendor/google.golang.org/grpc/internal
perkeep.org/vendor/google.golang.org/grpc/metadata
perkeep.org/vendor/github.com/cznic/kv
perkeep.org/vendor/google.golang.org/grpc/naming
perkeep.org/vendor/google.golang.org/grpc/stats
perkeep.org/vendor/google.golang.org/grpc/tap
perkeep.org/vendor/google.golang.org/grpc/peer
perkeep.org/vendor/google.golang.org/grpc/transport
perkeep.org/pkg/kvutil
perkeep.org/pkg/sorted/kvfile
perkeep.org/pkg/blobserver/diskpacked
perkeep.org/vendor/google.golang.org/api/storage/v1
perkeep.org/vendor/google.golang.org/grpc
perkeep.org/pkg/blobserver/dir
perkeep.org/cmd/pk-put
perkeep.org/vendor/google.golang.org/api/internal
perkeep.org/vendor/google.golang.org/api/option
perkeep.org/vendor/google.golang.org/api/googleapi/transport
perkeep.org/vendor/google.golang.org/grpc/credentials/oauth
perkeep.org/vendor/google.golang.org/api/transport
perkeep.org/vendor/github.com/go-sql-driver/mysql
perkeep.org/vendor/cloud.google.com/go/storage
perkeep.org/vendor/github.com/lib/pq/oid
perkeep.org/vendor/github.com/lib/pq
perkeep.org/vendor/go4.org/oauthutil
perkeep.org/vendor/gopkg.in/mgo.v2/internal/json
perkeep.org/vendor/gopkg.in/mgo.v2/internal/scram
perkeep.org/vendor/cloud.google.com/go/internal/bundler
perkeep.org/vendor/github.com/googleapis/gax-go
perkeep.org/vendor/google.golang.org/genproto/googleapis/api/label
perkeep.org/vendor/google.golang.org/genproto/googleapis/api/monitoredres
perkeep.org/vendor/github.com/golang/protobuf/ptypes/any
perkeep.org/vendor/github.com/golang/protobuf/ptypes/duration
perkeep.org/vendor/github.com/golang/protobuf/ptypes/empty
perkeep.org/vendor/github.com/golang/protobuf/ptypes/struct
perkeep.org/vendor/github.com/golang/protobuf/ptypes/timestamp
perkeep.org/vendor/github.com/golang/protobuf/ptypes/wrappers
perkeep.org/vendor/google.golang.org/genproto/googleapis/api/metric
perkeep.org/vendor/google.golang.org/genproto/protobuf
perkeep.org/vendor/gopkg.in/mgo.v2/bson
perkeep.org/vendor/google.golang.org/genproto/googleapis/rpc/status
perkeep.org/vendor/cloud.google.com/go/logging/internal
perkeep.org/vendor/github.com/golang/protobuf/ptypes
perkeep.org/vendor/go4.org/wkfs/gcs
perkeep.org/vendor/google.golang.org/api/compute/v1
perkeep.org/vendor/google.golang.org/genproto/googleapis/api/serviceconfig
perkeep.org/vendor/google.golang.org/genproto/googleapis/logging/type
perkeep.org/vendor/google.golang.org/genproto/googleapis/logging/v2
perkeep.org/vendor/gopkg.in/mgo.v2
perkeep.org/vendor/cloud.google.com/go/logging/apiv2
perkeep.org/vendor/cloud.google.com/go/logging
perkeep.org/vendor/google.golang.org/api/drive/v2
perkeep.org/pkg/sorted/mongo
perkeep.org/pkg/serverinit
perkeep.org/vendor/google.golang.org/api/sqladmin/v1beta3
perkeep.org/pkg/sorted/mysql
perkeep.org/pkg/sorted/postgres
perkeep.org/pkg/blobserver/google/drive/service
perkeep.org/vendor/google.golang.org/api/servicemanagement/v1
perkeep.org/vendor/google.golang.org/api/cloudresourcemanager/v1
perkeep.org/pkg/blobserver/google/drive
perkeep.org/vendor/golang.org/x/crypto/acme
perkeep.org/vendor/golang.org/x/crypto/acme/autocert
perkeep.org/internal/azure/storage
perkeep.org/pkg/blobserver/azure
perkeep.org/vendor/github.com/FiloSottile/b2
perkeep.org/internal/pools
perkeep.org/pkg/conv
perkeep.org/pkg/blobserver/cond
perkeep.org/pkg/blobserver/blobpacked
perkeep.org/vendor/golang.org/x/crypto/internal/subtle
perkeep.org/pkg/blobserver/b2
perkeep.org/vendor/golang.org/x/crypto/poly1305
perkeep.org/vendor/golang.org/x/crypto/salsa20/salsa
perkeep.org/vendor/golang.org/x/crypto/pbkdf2
perkeep.org/vendor/golang.org/x/crypto/scrypt
perkeep.org/vendor/go4.org/cloud/google/gcsutil
perkeep.org/vendor/golang.org/x/crypto/nacl/secretbox
perkeep.org/pkg/blobserver/encrypt
perkeep.org/pkg/blobserver/google/cloudstorage
perkeep.org/pkg/blobserver/mongo
perkeep.org/pkg/blobserver/overlay
perkeep.org/pkg/blobserver/proxycache
perkeep.org/pkg/blobserver/remote
perkeep.org/pkg/blobserver/replica
perkeep.org/vendor/github.com/aws/aws-sdk-go/aws/awserr
perkeep.org/vendor/github.com/aws/aws-sdk-go/internal/shareddefaults
perkeep.org/vendor/github.com/go-ini/ini
perkeep.org/vendor/github.com/aws/aws-sdk-go/aws/endpoints
perkeep.org/vendor/github.com/aws/aws-sdk-go/internal/sdkio
perkeep.org/vendor/github.com/aws/aws-sdk-go/aws/client/metadata
perkeep.org/vendor/github.com/jmespath/go-jmespath
perkeep.org/vendor/github.com/aws/aws-sdk-go/aws/credentials
perkeep.org/vendor/github.com/aws/aws-sdk-go/internal/sdkrand
perkeep.org/vendor/github.com/aws/aws-sdk-go/internal/sdkuri
perkeep.org/vendor/github.com/aws/aws-sdk-go/aws/awsutil
perkeep.org/vendor/go4.org/fault
perkeep.org/pkg/blobserver/shard
perkeep.org/pkg/blobserver/union
perkeep.org/pkg/camlegal
perkeep.org/pkg/gpgchallenge
perkeep.org/vendor/go4.org/net/throttle
perkeep.org/pkg/webserver/listen
perkeep.org/pkg/webserver
perkeep.org/pkg/app
perkeep.org/pkg/publish
perkeep.org/app/hello
perkeep.org/internal/osutil/gce
perkeep.org/vendor/go4.org/cloud/google/gceutil
perkeep.org/pkg/deploy/gce
perkeep.org/cmd/pk
perkeep.org/cmd/pk-deploy
perkeep.org/vendor/github.com/aws/aws-sdk-go/aws
perkeep.org/vendor/github.com/aws/aws-sdk-go/aws/request
perkeep.org/vendor/github.com/aws/aws-sdk-go/aws/client
perkeep.org/vendor/github.com/aws/aws-sdk-go/aws/corehandlers
perkeep.org/vendor/github.com/aws/aws-sdk-go/private/protocol
perkeep.org/vendor/github.com/aws/aws-sdk-go/private/protocol/rest
perkeep.org/vendor/github.com/aws/aws-sdk-go/aws/signer/v4
perkeep.org/vendor/github.com/aws/aws-sdk-go/private/protocol/query/queryutil
perkeep.org/vendor/github.com/aws/aws-sdk-go/private/protocol/xml/xmlutil
perkeep.org/vendor/github.com/aws/aws-sdk-go/private/protocol/query
perkeep.org/vendor/github.com/aws/aws-sdk-go/service/sts
perkeep.org/vendor/github.com/aws/aws-sdk-go/aws/csm
perkeep.org/vendor/github.com/aws/aws-sdk-go/aws/ec2metadata
perkeep.org/vendor/github.com/aws/aws-sdk-go/aws/credentials/stscreds
perkeep.org/vendor/github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds
perkeep.org/vendor/github.com/aws/aws-sdk-go/aws/credentials/endpointcreds
perkeep.org/vendor/github.com/aws/aws-sdk-go/private/protocol/eventstream
perkeep.org/vendor/github.com/aws/aws-sdk-go/aws/defaults
perkeep.org/vendor/github.com/aws/aws-sdk-go/aws/session
perkeep.org/vendor/github.com/aws/aws-sdk-go/private/protocol/eventstream/eventstreamapi
perkeep.org/vendor/github.com/aws/aws-sdk-go/private/protocol/restxml
perkeep.org/vendor/github.com/aws/aws-sdk-go/service/s3
perkeep.org/app/publisher
perkeep.org/app/scanningcabinet/ui
perkeep.org/app/scanningcabinet
perkeep.org/vendor/rsc.io/pdf
perkeep.org/vendor/github.com/aws/aws-sdk-go/service/s3/s3iface
perkeep.org/app/scanningcabinet/scancab
perkeep.org/vendor/github.com/aws/aws-sdk-go/service/s3/s3manager
perkeep.org/pkg/blobserver/s3
perkeep.org/server/perkeepd
perkeep.org/vendor/bazil.org/fuse
perkeep.org/vendor/bazil.org/fuse/fuseutil
perkeep.org/vendor/bazil.org/fuse/fs
perkeep.org/pkg/fs
perkeep.org/cmd/pk-mount
Success. Binaries are in /home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/bin

real	2m2.889s
user	2m52.036s
sys	0m22.663s
-----------
xftroxgpx@z5 2019/01/07 16:12:03 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24557
!77053 11 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/src/perkeep.org 
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/xftroxgpx/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build836258175=/tmp/go-build -gno-record-gcc-switches"
-----------
xftroxgpx@z5 2019/01/07 16:12:21 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24574
!77054 12 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/src/perkeep.org 
$ 
-----------
xftroxgpx@z5 2019/01/07 16:14:51 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24724
!77054 12 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/src/perkeep.org 
$ 

the go env output from within /home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/perkeep.org dir is missing from the above because I did it in another terminal window, but here's me moving it back to bad location, in the same above terminal:

xftroxgpx@z5 2019/01/07 16:14:51 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24724
!77054 12 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/src/perkeep.org 
$ cd ..
-----------
xftroxgpx@z5 2019/01/07 16:16:33 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24827
!77055 13 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/src 
$ mv perkeep.org/ ../
-----------
xftroxgpx@z5 2019/01/07 16:16:37 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24830
!77056 14 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/src 
$ l
total 0
drwxr-xr-x 1 xftroxgpx xftroxgpx 40 07.01.2019 16:16 ../
drwxr-xr-x 1 xftroxgpx xftroxgpx  0 07.01.2019 16:16 ./
-----------
xftroxgpx@z5 2019/01/07 16:16:38 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24832
!77057 15 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/src 
$ cd ..
-----------
xftroxgpx@z5 2019/01/07 16:16:40 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24834
!77058 16 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath 
$ cd perkeep.org/
-----------
xftroxgpx@z5 2019/01/07 16:16:42 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24836
!77059 17 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/perkeep.org 
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/xftroxgpx/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/perkeep.org/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build256747807=/tmp/go-build -gno-record-gcc-switches"
-----------
xftroxgpx@z5 2019/01/07 16:16:44 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24838
!77060 18 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/perkeep.org 
$ time go run make.go
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[6568](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
bazil.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[6568](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('bazil.org') which was not transformed('bazil.org') as follows:
104.28.6.28 bazil.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[6568](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
bazil.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[6568](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('bazil.org') which was not transformed('bazil.org') as follows:
104.28.6.28 bazil.org
go: bazil.org/fuse@v0.0.0-20160811212531-371fbbdaa898: unrecognized import path "bazil.org/fuse" (https fetch: Get http://bazil.org/fuse/?go-get=1: dial tcp 104.28.6.28:80: connect: connection refused)
go: error loading module requirements

real	0m0.427s
user	0m0.158s
sys	0m0.035s
-----------
xftroxgpx@z5 2019/01/07 16:17:00 -bash5.0.0 t:4 j:0 d:3 pp:936 p:18168 ut24854
!77061 19 1  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/perkeep.org 
$ 

the diff is the same (I checked)

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Jan 7, 2019

Ok, so the difference here is definitely due to module mode vs. GOPATH mode: when you're in GOPATH/src, you're in GOPATH mode.

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Jan 7, 2019

make.go explicitly requires itself to be run from $GOPATH/src/perkeep.org:
https://github.com/perkeep/perkeep/blob/eb547718cb285cd53c77ae500c68b29ec362f098/make.go#L734

I don't know how you're getting to the error that you're getting, but I can't reproduce it.

@xftroxgpx

This comment has been minimized.

Copy link

xftroxgpx commented Jan 7, 2019

I think you have to deny outgoing http tcp port 80 in firewall, if you want to reproduce it

@xftroxgpx

This comment has been minimized.

Copy link

xftroxgpx commented Jan 7, 2019

Is make.go considered when I run go clean because the error happens in this case also.

$ cd $GOPATH
#note: NOT $GOPATH/src  !!!
$ git clone https://github.com/perkeep/perkeep.git perkeep.org
...
$ cd perkeep.org
$ go clean

xftroxgpx@z5 2019/01/07 22:26:01 -bash5.0.0 t:3 j:0 d:3 pp:937 p:23122 ut1824
!77206 4 0  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/perkeep.org 
$ go clean
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
golang.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('golang.org') which was not transformed('golang.org') as follows:
216.58.214.241 golang.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
golang.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('golang.org') which was not transformed('golang.org') as follows:
216.58.214.241 golang.org
go: finding github.com/spf13/cobra v0.0.0-20160322171042-c678ff029ee2
go: finding github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8
go: finding github.com/shurcooL/sanitized_anchor_name v0.0.0-20150515002706-11a20b799bf2
go: finding github.com/fsnotify/fsnotify v1.4.2
go: finding github.com/FiloSottile/b2 v0.0.0-20170207175032-b197f7a2c317
go: finding github.com/garyburd/go-oauth v0.0.0-20180319155456-bca2e7f09a17
go: finding github.com/aws/aws-sdk-go v1.14.31
go: finding golang.org/x/text v0.0.0-20171102192421-88f656faf3f3
go: finding golang.org/x/sync v0.0.0-20171101214715-fd80eb99c8f6
go: finding golang.org/x/time v0.0.0-20160202183820-a4bde1265759
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
cloud.google.com
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('cloud.google.com') which was not transformed('cloud.google.com') as follows:
172.217.20.14 cloud.google.com
go: finding github.com/cznic/sortutil v0.0.0-20150617083342-4c7342852e65
go: finding github.com/gopherjs/gopherjs v0.0.0-20181004052249-0fee806ad497
go: finding github.com/cznic/lldb v1.1.0
go: finding github.com/pkg/sftp v0.0.0-20180419200840-5bf2a174b604
go: finding github.com/cznic/mathutil v0.0.0-20180214153908-5455a562bccb
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
bazil.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('bazil.org') which was not transformed('bazil.org') as follows:
104.28.6.28 bazil.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
bazil.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('bazil.org') which was not transformed('bazil.org') as follows:
104.28.6.28 bazil.org
go: bazil.org/fuse@v0.0.0-20160811212531-371fbbdaa898: unrecognized import path "bazil.org/fuse" (https fetch: Get http://bazil.org/fuse/?go-get=1: dial tcp 104.28.6.28:80: connect: connection refused)
go: finding github.com/mattn/go-sqlite3 v1.6.0
go: finding github.com/kr/fs v0.1.0
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
rsc.io
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('rsc.io') which was not transformed('rsc.io') as follows:
216.239.32.21 rsc.io
go: finding rsc.io/qr v0.1.0
go: finding github.com/nf/cr2 v0.0.0-20140528043846-05d46fef4f2f
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
myitcv.io
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('myitcv.io') which was not transformed('myitcv.io') as follows:
216.239.32.21 myitcv.io
go: finding github.com/edsrzf/mmap-go v0.0.0-20170320065105-0bce6a688712
go: finding github.com/kardianos/osext v0.0.0-20151222153229-29ae4ffbc9a6
go: finding myitcv.io v0.0.0-20180717150903-bf2c84553c78
go: finding github.com/cznic/kv v0.0.0-20170515202733-892ccf731fb7
go: finding github.com/tomnomnom/linkheader v0.0.0-20160328204959-6953a30d4443
go: finding golang.org/x/oauth2 v0.0.0-20171226133531-197281d4e0ec
go: finding github.com/golang/snappy v0.0.0-20170215233205-553a64147049
go: finding github.com/gorilla/websocket v1.2.0
go: finding github.com/gopherjs/jquery v0.0.0-20151020083714-fbbfc4bbe29a
go: finding golang.org/x/image v0.0.0-20171214225156-12117c17ca67
go: finding github.com/mailgun/mailgun-go v0.0.0-20171127222028-17e8bd11e87c
go: finding github.com/russross/blackfriday v2.0.0+incompatible
go: finding github.com/rwcarlsen/goexif v0.0.0-20180518182100-8d986c03457a
go: finding github.com/cznic/internal v0.0.0-20170905175358-4747030f7cf2
go: finding github.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86
go: finding github.com/kisielk/gotool v0.0.0-20161130080628-0de1eaf82fa3
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
go4.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('go4.org') which was not transformed('go4.org') as follows:
212.47.227.56 go4.org
go: finding github.com/go-ini/ini v1.38.1
go: finding go4.org v0.0.0-20180413184151-a2a47940e6bc
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
honnef.co
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('honnef.co') which was not transformed('honnef.co') as follows:
138.197.72.227 honnef.co
go: finding cloud.google.com/go v0.0.0-20161006024015-b70ccc799b9d
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
google.golang.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('google.golang.org') which was not transformed('google.golang.org') as follows:
216.58.214.241 google.golang.org
go: finding honnef.co/go/js/dom v0.0.0-20160310112645-24aa052bc5c6
go: finding google.golang.org/appengine v1.0.0
go: finding github.com/miekg/dns v0.0.0-20161003181808-3f1f7c8ec9ea
go: finding golang.org/x/net v0.0.0-20171212005608-d866cfc389ce
go: finding github.com/cznic/fileutil v0.0.0-20180108211300-6a051e75936f
go: finding github.com/gopherjs/jsbuiltin v0.0.0-20170427220125-67703bfb044e
go: finding github.com/hjfreyer/taglib-go v0.0.0-20151027170453-0ef8bba9c41b
go: finding google.golang.org/api v0.0.0-20170327174102-48e49d1645e2
go: finding github.com/inconshreveable/mousetrap v1.0.0
go: finding github.com/neelance/sourcemap v0.0.0-20151028013722-8c68805598ab
go: finding github.com/golang/protobuf v1.0.0
go: finding google.golang.org/genproto v0.0.0-20161115011420-08f135d1a31b
go: finding golang.org/x/sys v0.0.0-20180115085844-fff93fa7cd27
go: finding github.com/bradfitz/grpc-go v0.0.0-20170203184515-188a132adcfba339f1f2d5da52498451341f9ee8
go: finding github.com/syndtr/goleveldb v0.0.0-20180608030153-db3ee9ee8931
go: finding github.com/go-sql-driver/mysql v1.4.1-0.20180719071942-99ff426eb706
go: finding github.com/mattn/go-mastodon v0.0.0-20180129050910-2ccbcfe14d7a
go: finding golang.org/x/crypto v0.0.0-20180723164146-c126467f60eb
go: finding github.com/plaid/plaid-go v0.0.0-20161222051224-02b6af68061b
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
gopkg.in
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[27351](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('gopkg.in') which was not transformed('gopkg.in') as follows:
35.196.143.184 gopkg.in
go: finding gopkg.in/mgo.v2 v2.0.0-20160818020120-3f83fa500528
go: finding golang.org/x/tools v0.0.0-20170915040203-e531a2a1c15f
go: finding github.com/shurcooL/httpfs v0.0.0-20171119174359-809beceb2371
go: finding github.com/spf13/pflag v0.0.0-20151218134703-7f60f83a2c81
go: finding rsc.io/pdf v0.0.0-20170302045715-1d34785eb915
go: finding github.com/bradfitz/latlong v0.0.0-20140711231157-b74550508561
go: finding github.com/lib/pq v0.0.0-20130607063955-9afcd9aa7931
go: finding github.com/googleapis/gax-go v0.0.0-20161107002406-da06d194a00e
go: finding github.com/tgulacsi/picago v0.0.0-20171229130838-9e1ac2306c70
go: finding github.com/cznic/zappy v0.0.0-20160723133515-2533cb5b45cc
go: finding github.com/pkg/errors v0.8.0
go: error loading module requirements
-----------
xftroxgpx@z5 2019/01/07 22:26:51 -bash5.0.0 t:3 j:0 d:3 pp:937 p:23122 ut1873
!77207 5 1  4.20.0-g8fe28cb58bcb #30 SMP PREEMPT Thu Jan 3 20:53:51 CET 2019
/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/perkeep.org 
$ go clean
go: finding github.com/go-ini/ini v1.25.4
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[2743](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
bazil.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[2743](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('bazil.org') which was not transformed('bazil.org') as follows:
104.28.6.28 bazil.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[2743](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
bazil.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[2743](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('bazil.org') which was not transformed('bazil.org') as follows:
104.28.6.28 bazil.org
go: bazil.org/fuse@v0.0.0-20160811212531-371fbbdaa898: unrecognized import path "bazil.org/fuse" (https fetch: Get http://bazil.org/fuse/?go-get=1: dial tcp 104.28.6.28:80: connect: connection refused)
go: error loading module requirements
@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Jan 7, 2019

$ go mod init golang.org/issue/scratch
go: creating new go.mod: module golang.org/issue/scratch
$ cd $GOPATH
_gopath$ git clone https://github.com/perkeep/perkeep.git perkeep.org
Cloning into 'perkeep.org'...
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 63355 (delta 0), reused 0 (delta 0), pack-reused 63354
Receiving objects: 100% (63355/63355), 43.97 MiB | 37.18 MiB/s, done.
Resolving deltas: 100% (37854/37854), done.
_gopath$ cd perkeep.org/
_gopath/perkeep.org$ go1.11.4 list -m all
go: finding github.com/plaid/plaid-go v0.0.0-20161222051224-02b6af68061b
go: finding github.com/spf13/cobra v0.0.0-20160322171042-c678ff029ee2
go: finding github.com/fsnotify/fsnotify v1.4.2
go: finding github.com/cznic/lldb v1.1.0
go: finding github.com/aws/aws-sdk-go v1.14.31
go: finding github.com/rwcarlsen/goexif v0.0.0-20180518182100-8d986c03457a
go: finding github.com/go-ini/ini v1.38.1
go: finding github.com/tgulacsi/picago v0.0.0-20171229130838-9e1ac2306c70
go: finding golang.org/x/sync v0.0.0-20171101214715-fd80eb99c8f6
go: finding cloud.google.com/go v0.0.0-20161006024015-b70ccc799b9d
go: finding github.com/pkg/sftp v0.0.0-20180419200840-5bf2a174b604
go: finding github.com/nf/cr2 v0.0.0-20140528043846-05d46fef4f2f
go: finding github.com/gopherjs/jquery v0.0.0-20151020083714-fbbfc4bbe29a
go: finding github.com/go-sql-driver/mysql v1.4.1-0.20180719071942-99ff426eb706
go: finding golang.org/x/net v0.0.0-20171212005608-d866cfc389ce
go: finding github.com/kr/fs v0.1.0
go: finding github.com/tomnomnom/linkheader v0.0.0-20160328204959-6953a30d4443
go: finding github.com/gopherjs/gopherjs v0.0.0-20181004052249-0fee806ad497
go: finding github.com/mailgun/mailgun-go v0.0.0-20171127222028-17e8bd11e87c
go: finding github.com/syndtr/goleveldb v0.0.0-20180608030153-db3ee9ee8931
go: finding github.com/cznic/internal v0.0.0-20170905175358-4747030f7cf2
go: finding github.com/shurcooL/sanitized_anchor_name v0.0.0-20150515002706-11a20b799bf2
go: finding github.com/russross/blackfriday v2.0.0+incompatible
go: finding rsc.io/qr v0.1.0
go: finding github.com/mattn/go-sqlite3 v1.6.0
go: finding github.com/kardianos/osext v0.0.0-20151222153229-29ae4ffbc9a6
go: finding github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8
go: finding github.com/FiloSottile/b2 v0.0.0-20170207175032-b197f7a2c317
go: finding github.com/golang/protobuf v1.0.0
go: finding rsc.io/pdf v0.0.0-20170302045715-1d34785eb915
go: finding github.com/shurcooL/httpfs v0.0.0-20171119174359-809beceb2371
go: finding go4.org v0.0.0-20180413184151-a2a47940e6bc
go: finding github.com/miekg/dns v0.0.0-20161003181808-3f1f7c8ec9ea
go: finding github.com/cznic/sortutil v0.0.0-20150617083342-4c7342852e65
go: finding github.com/kisielk/gotool v0.0.0-20161130080628-0de1eaf82fa3
go: finding github.com/cznic/mathutil v0.0.0-20180214153908-5455a562bccb
go: finding github.com/pkg/errors v0.8.0
go: finding google.golang.org/genproto v0.0.0-20161115011420-08f135d1a31b
go: finding google.golang.org/api v0.0.0-20170327174102-48e49d1645e2
go: finding github.com/golang/snappy v0.0.0-20170215233205-553a64147049
go: finding golang.org/x/text v0.0.0-20171102192421-88f656faf3f3
go: finding github.com/cznic/fileutil v0.0.0-20180108211300-6a051e75936f
go: finding github.com/mattn/go-mastodon v0.0.0-20180129050910-2ccbcfe14d7a
go: finding github.com/garyburd/go-oauth v0.0.0-20180319155456-bca2e7f09a17
go: finding golang.org/x/sys v0.0.0-20180115085844-fff93fa7cd27
go: finding github.com/bradfitz/grpc-go v0.0.0-20170203184515-188a132adcfba339f1f2d5da52498451341f9ee8
go: finding golang.org/x/tools v0.0.0-20170915040203-e531a2a1c15f
go: finding github.com/cznic/zappy v0.0.0-20160723133515-2533cb5b45cc
go: finding golang.org/x/crypto v0.0.0-20180723164146-c126467f60eb
go: finding github.com/lib/pq v0.0.0-20130607063955-9afcd9aa7931
go: finding myitcv.io v0.0.0-20180717150903-bf2c84553c78
go: finding github.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86
go: finding golang.org/x/image v0.0.0-20171214225156-12117c17ca67
go: finding github.com/spf13/pflag v0.0.0-20151218134703-7f60f83a2c81
go: finding gopkg.in/mgo.v2 v2.0.0-20160818020120-3f83fa500528
go: finding github.com/neelance/sourcemap v0.0.0-20151028013722-8c68805598ab
go: finding github.com/bradfitz/latlong v0.0.0-20140711231157-b74550508561
go: finding github.com/inconshreveable/mousetrap v1.0.0
go: finding github.com/edsrzf/mmap-go v0.0.0-20170320065105-0bce6a688712
go: finding github.com/cznic/kv v0.0.0-20170515202733-892ccf731fb7
go: finding github.com/gopherjs/jsbuiltin v0.0.0-20170427220125-67703bfb044e
go: finding bazil.org/fuse v0.0.0-20160811212531-371fbbdaa898
go: finding honnef.co/go/js/dom v0.0.0-20160310112645-24aa052bc5c6
go: finding github.com/hjfreyer/taglib-go v0.0.0-20151027170453-0ef8bba9c41b
go: finding golang.org/x/time v0.0.0-20160202183820-a4bde1265759
go: finding github.com/googleapis/gax-go v0.0.0-20161107002406-da06d194a00e
go: finding golang.org/x/oauth2 v0.0.0-20171226133531-197281d4e0ec
go: finding github.com/gorilla/websocket v1.2.0
go: finding google.golang.org/appengine v1.0.0
go: finding github.com/go-ini/ini v1.25.4
go: finding google.golang.org/grpc v1.14.0
perkeep.org
bazil.org/fuse v0.0.0-20160811212531-371fbbdaa898
cloud.google.com/go v0.0.0-20161006024015-b70ccc799b9d
github.com/FiloSottile/b2 v0.0.0-20170207175032-b197f7a2c317
github.com/aws/aws-sdk-go v1.14.31
github.com/bradfitz/latlong v0.0.0-20140711231157-b74550508561
github.com/cznic/fileutil v0.0.0-20180108211300-6a051e75936f
github.com/cznic/internal v0.0.0-20170905175358-4747030f7cf2
github.com/cznic/kv v0.0.0-20170515202733-892ccf731fb7
github.com/cznic/lldb v1.1.0
github.com/cznic/mathutil v0.0.0-20180214153908-5455a562bccb
github.com/cznic/sortutil v0.0.0-20150617083342-4c7342852e65
github.com/cznic/zappy v0.0.0-20160723133515-2533cb5b45cc
github.com/edsrzf/mmap-go v0.0.0-20170320065105-0bce6a688712
github.com/fsnotify/fsnotify v1.4.2
github.com/garyburd/go-oauth v0.0.0-20180319155456-bca2e7f09a17
github.com/go-ini/ini v1.38.1
github.com/go-sql-driver/mysql v1.4.1-0.20180719071942-99ff426eb706
github.com/golang/protobuf v1.0.0
github.com/golang/snappy v0.0.0-20170215233205-553a64147049
github.com/googleapis/gax-go v0.0.0-20161107002406-da06d194a00e
github.com/gopherjs/gopherjs v0.0.0-20181004052249-0fee806ad497
github.com/gopherjs/jquery v0.0.0-20151020083714-fbbfc4bbe29a
github.com/gopherjs/jsbuiltin v0.0.0-20170427220125-67703bfb044e
github.com/gorilla/websocket v1.2.0
github.com/hjfreyer/taglib-go v0.0.0-20151027170453-0ef8bba9c41b
github.com/inconshreveable/mousetrap v1.0.0
github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8
github.com/kardianos/osext v0.0.0-20151222153229-29ae4ffbc9a6
github.com/kisielk/gotool v0.0.0-20161130080628-0de1eaf82fa3
github.com/kr/fs v0.1.0
github.com/lib/pq v0.0.0-20130607063955-9afcd9aa7931
github.com/mailgun/mailgun-go v0.0.0-20171127222028-17e8bd11e87c
github.com/mattn/go-mastodon v0.0.0-20180129050910-2ccbcfe14d7a
github.com/mattn/go-sqlite3 v1.6.0
github.com/miekg/dns v0.0.0-20161003181808-3f1f7c8ec9ea
github.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86
github.com/neelance/sourcemap v0.0.0-20151028013722-8c68805598ab
github.com/nf/cr2 v0.0.0-20140528043846-05d46fef4f2f
github.com/pkg/errors v0.8.0
github.com/pkg/sftp v0.0.0-20180419200840-5bf2a174b604
github.com/plaid/plaid-go v0.0.0-20161222051224-02b6af68061b
github.com/russross/blackfriday v2.0.0+incompatible
github.com/rwcarlsen/goexif v0.0.0-20180518182100-8d986c03457a
github.com/shurcooL/httpfs v0.0.0-20171119174359-809beceb2371
github.com/shurcooL/sanitized_anchor_name v0.0.0-20150515002706-11a20b799bf2
github.com/spf13/cobra v0.0.0-20160322171042-c678ff029ee2
github.com/spf13/pflag v0.0.0-20151218134703-7f60f83a2c81
github.com/syndtr/goleveldb v0.0.0-20180608030153-db3ee9ee8931
github.com/tgulacsi/picago v0.0.0-20171229130838-9e1ac2306c70
github.com/tomnomnom/linkheader v0.0.0-20160328204959-6953a30d4443
go4.org v0.0.0-20180413184151-a2a47940e6bc
golang.org/x/crypto v0.0.0-20180723164146-c126467f60eb
golang.org/x/image v0.0.0-20171214225156-12117c17ca67
golang.org/x/net v0.0.0-20171212005608-d866cfc389ce
golang.org/x/oauth2 v0.0.0-20171226133531-197281d4e0ec
golang.org/x/sync v0.0.0-20171101214715-fd80eb99c8f6
golang.org/x/sys v0.0.0-20180115085844-fff93fa7cd27
golang.org/x/text v0.0.0-20171102192421-88f656faf3f3
golang.org/x/time v0.0.0-20160202183820-a4bde1265759
golang.org/x/tools v0.0.0-20170915040203-e531a2a1c15f
google.golang.org/api v0.0.0-20170327174102-48e49d1645e2
google.golang.org/appengine v1.0.0
google.golang.org/genproto v0.0.0-20161115011420-08f135d1a31b
google.golang.org/grpc v1.14.0 => github.com/bradfitz/grpc-go v0.0.0-20170203184515-188a132adcfba339f1f2d5da52498451341f9ee8
gopkg.in/mgo.v2 v2.0.0-20160818020120-3f83fa500528
honnef.co/go/js/dom v0.0.0-20160310112645-24aa052bc5c6
myitcv.io v0.0.0-20180717150903-bf2c84553c78
rsc.io/pdf v0.0.0-20170302045715-1d34785eb915
rsc.io/qr v0.1.0
_gopath/perkeep.org$ go1.11.4 mod download -json bazil.org/fuse
{
        "Path": "bazil.org/fuse",
        "Version": "v0.0.0-20160811212531-371fbbdaa898",
        "Info": "/tmp/tmp.YLhhiOqP3k/_gopath/pkg/mod/cache/download/bazil.org/fuse/@v/v0.0.0-20160811212531-371fbbdaa898.info",
        "GoMod": "/tmp/tmp.YLhhiOqP3k/_gopath/pkg/mod/cache/download/bazil.org/fuse/@v/v0.0.0-20160811212531-371fbbdaa898.mod",
        "Zip": "/tmp/tmp.YLhhiOqP3k/_gopath/pkg/mod/cache/download/bazil.org/fuse/@v/v0.0.0-20160811212531-371fbbdaa898.zip",
        "Dir": "/tmp/tmp.YLhhiOqP3k/_gopath/pkg/mod/bazil.org/fuse@v0.0.0-20160811212531-371fbbdaa898",
        "Sum": "h1:SC+c6A1qTFstO9qmB86mPV2IpYme/2ZoEQ0hrP+wo+Q=",
        "GoModSum": "h1:Xbm+BRKSBEpa4q4hTSxohYNQpsxXPbPry4JJWOB3LB8="
}
@xftroxgpx

This comment has been minimized.

Copy link

xftroxgpx commented Jan 7, 2019

You have to block http port 80? as I do. Sorry, I should've been more explicit, that by blocking outgoing tcp port 80 I am able to detect when transfers are attempted via http(instead of https) and that's why the connection refused error happens.

/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/perkeep.org 
$ go list -m all
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[9234](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
bazil.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[9234](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('bazil.org') which was not transformed('bazil.org') as follows:
104.28.6.28 bazil.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[9234](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 1of2 attempting to resolve (requested)hostname:
bazil.org
glibc64:../sysdeps/posix/getaddrinfo.c:2212/getaddrinfo: go[9234](full:'go') for user xftroxgpx(1000(eff:xftroxgpx(1000))) 2of2 successfully resolved requested hostname('bazil.org') which was not transformed('bazil.org') as follows:
104.28.6.28 bazil.org
go: bazil.org/fuse@v0.0.0-20160811212531-371fbbdaa898: unrecognized import path "bazil.org/fuse" (https fetch: Get http://bazil.org/fuse/?go-get=1: dial tcp 104.28.6.28:80: connect: connection refused)
go: error loading module requirements
@xftroxgpx

This comment has been minimized.

Copy link

xftroxgpx commented Jan 7, 2019

In other words, in module mode, go tries http port 80, but in GOPATH mode, it only tries https port 443

So this $ go mod download -json bazil.org/fuse tries to download it over http instead of https. I don't know why, but it doesn't seem safe :D (I jokingly assume that some people are running state-sponsored compromised go modules because of this =) but there's probably other checks that prevent this, right?)

@bcmills bcmills self-assigned this Jan 7, 2019

@bcmills bcmills modified the milestones: Go1.13, Go1.12 Jan 7, 2019

@bcmills bcmills removed the WaitingForInfo label Jan 7, 2019

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Jan 8, 2019

The problem is with bazil.org itself: its HTTPS handler issues a 301 redirect to the HTTP endpoint, which the go command follows. I do not know why it issues that redirect; perhaps @tv42 could explain.

$ curl -D - https://bazil.org/fuse?go-get=1
HTTP/2 301
date: Tue, 08 Jan 2019 14:34:20 GMT
content-type: text/html
set-cookie: __cfduid=d583f4772e2e45126235a034efc3ff2c11546958060; expires=Wed, 08-Jan-20 14:34:20 GMT; path=/; domain=.bazil.org; HttpOnly
location: http://bazil.org/fuse/?go-get=1
access-control-allow-origin: *
expires: Tue, 08 Jan 2019 14:44:20 GMT
cache-control: max-age=600
x-github-request-id: 919A:1E62:2C690B6:3C4A57A:5C34B4EC
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 495f6266a813c18b-IAD

<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx</center>
</body>
</html>

I suspect that the only reason you are not seeing an HTTPS fetch in GOPATH mode is because you already have a copy of bazil.org/fuse in your GOPATH, and whatever go command you end up running does not try to update it. Using strace, I can reproduce the fetch on port 80 even in GOPATH mode:

$ GO111MODULE=off strace -e trace=network -f go get -u bazil.org/fuse 2>&1 | grep 'port=htons(80)'
[pid 231299] connect(5, {sa_family=AF_INET6, sin6_port=htons(80), inet_pton(AF_INET6, "2606:4700:30::681c:61c", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = -1 EINPROGRESS (Operation now in progress)
[pid 231310] getpeername(5, {sa_family=AF_INET6, sin6_port=htons(80), inet_pton(AF_INET6, "2606:4700:30::681c:61c", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, [112->28]) = 0

@bcmills bcmills changed the title cmd/go: packages listed in go.mod are fetched via http port 80 even when -insecure arg isn't used only when repo lives in $GOPATH instead of $GOPATH/src cmd/go: downloads follow plain-HTTP redirects even when the -insecure flag is not set Jan 8, 2019

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Jan 8, 2019

The remaining question is, if the HTTPS handler explicitly indicates an insecure endpoint, should we follow it when the -insecure flag is not set?

I suspect that we should not, which probably means that the Go command should use a CheckRedirect handler to prohibit such redirects.

CC @FiloSottile, @bradfitz, @rsc

@bradfitz

This comment has been minimized.

Copy link
Member

bradfitz commented Jan 8, 2019

I suspect that we should not, which probably means that the Go command should use a CheckRedirect handler to prohibit such redirects.

Agreed.

@gopherbot

This comment has been minimized.

Copy link

gopherbot commented Jan 8, 2019

Change https://golang.org/cl/156837 mentions this issue: cmd/go/internal/modcmd: add the -v flag to 'go mod download'

@gopherbot

This comment has been minimized.

Copy link

gopherbot commented Jan 8, 2019

Change https://golang.org/cl/156838 mentions this issue: cmd/go/internal/web: reject insecure redirects from secure origins

@FiloSottile

This comment has been minimized.

Copy link
Member

FiloSottile commented Jan 8, 2019

Agreed, we should not follow redirects to HTTP. Depending on how many packages the patch will break, we should consider backporting it, too. It's true that it requires a server misconfiguration, but it breaks user expectations about not having to trust their network.

@gopherbot please open two backport issues.

@gopherbot

This comment has been minimized.

Copy link

gopherbot commented Jan 8, 2019

Backport issue(s) opened: #29617 (for 1.10), #29618 (for 1.11).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

@nightlyone

This comment has been minimized.

Copy link
Contributor

nightlyone commented Jan 8, 2019

@bradfitz / @FiloSottile does it make sense to add a tracking bug to expose such a redirect handler in the http package? Or even make it the default?

I don't believe clients want such unsafe redirects per default.

@xftroxgpx

This comment has been minimized.

Copy link

xftroxgpx commented Jan 8, 2019

from your comment above

I suspect that the only reason you are not seeing an HTTPS fetch in GOPATH mode is because you already have a copy of bazil.org/fuse in your GOPATH, and whatever go command you end up running does not try to update it.

Looks like in non-mod mode(ie. go.mod is ignored - i deduced), so in GOPATH mode, perkeep doesn't need to fetch bazil.org/fuse, seems to me that it instead just uses the one it has in its repo under the vendor dir.
That indeed explains why no http(or any) fetch for bazil.org/fuse happened in GOPATH mode (whereas I previously assumed it was happening but as https only).

The problem is with bazil.org itself: its HTTPS handler issues a 301 redirect to the HTTP endpoint, which the go command follows.

Thank you for your time/work/effort on this! (not tagging you because it's not important)

here's something that I wanted to say before/instead, but feel free to ignore it as it's irrelevant given the above

Where would this already-existing bazil.org/fuse be exactly? I've found only one in $GOPATH/src/perkeep.org/vendor/bazil.org/fuse (but that's not it, because it's part of perkeep's repo, says git status) which if I remove like mv $GOPATH/src/perkeep.org/vendor/bazil.org /tmp then go run make.go doesn't try to get it updated but instead it just says:

/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/src/perkeep.org 
$ time go run make.go
cmd/pk-mount/pkmount.go:45:2: cannot find package "bazil.org/fuse" in any of:
	/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/src/perkeep.org/vendor/bazil.org/fuse (vendor tree)
	/usr/lib/go/src/bazil.org/fuse (from $GOROOT)
	/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/src/bazil.org/fuse (from $GOPATH)
cmd/pk-mount/pkmount.go:46:2: cannot find package "bazil.org/fuse/fs" in any of:
	/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/src/perkeep.org/vendor/bazil.org/fuse/fs (vendor tree)
	/usr/lib/go/src/bazil.org/fuse/fs (from $GOROOT)
	/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/src/bazil.org/fuse/fs (from $GOPATH)
Error building main binaries: exit status 1
exit status 1

real	0m6.728s
user	0m6.672s
sys	0m1.795s

there's no bazil in GOPATH anymore:

/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath 
$ find . -name \*bazil\*

But if I put it back:

/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath 
$ find . -name \*bazil\*
./src/perkeep.org/vendor/bazil.org

/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath 
$ cd src/perkeep.org/

/home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/src/perkeep.org 
$ time go run make.go
perkeep.org/server/perkeepd/ui
perkeep.org/pkg/server
Success. Binaries are in /home/xftroxgpx/build/2nonpkgs/go.stuff/gopath/bin

real	0m13.459s
user	0m24.560s
sys	0m4.068s

Not entirely sure how to use go clean (i tried it, doesn't seem to do anything other than cause the above go run make.go redo only those two (ui and server). I guess I'd have to remove GOPATH/pkg/ dir.

I'm thinking either bazil doesn't redirect to http when in GOPATH mode OR bazil.org/fuse isn't needed/used when in GOPATH ... Why do I say this? because deleting everything in GOPATH (is there somewhere else? ~/.cache/ ? well, .cache -> /tmp/ for me, so cleaned that with the last boot) does still lead to successfull perkeep build which means it either did get bazil.org/fuse via https (since http is blocked) OR it didn't need it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment