Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: support self-hosted fossil in 'go get' #25811

Closed
schollz opened this issue Jun 10, 2018 · 15 comments

Comments

Projects
None yet
4 participants
@schollz
Copy link

commented Jun 10, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.10.3 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/zns/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/zns/go"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build451328689=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I am self-hosting fossils using a standalone server. I have a very simple "hello, world" example here: https://fossil.schollz.com/hello-world which I am trying to go get. This is a functional fossil, as you can clone and run it:

> fossil clone https://fossil.schollz.com/hello-world hello-world.fossil
> fossil open hello-world.fossil
> go run main.go
hello, world

I then tried to go get this fossil.

What did you expect to see?

I expected to be able to run

> go get fossil.schollz.com/hello-world.fossil

which would automatically download hello-world as a fossil into my $GOPATH.

What did you see instead?

Despite it being run using the canonical fossil server command, I am unable to go get it. Here are the attempts:

>  go get fossil.schollz.com/hello-world
package fossil.schollz.com/hello-world: unrecognized import path "fossil.schollz.com/hello-world" 
(parse https://fossil.schollz.com/hello-world?go-get=1: no go-import meta tags ())
> go get fossil.schollz.com/hello-world.fossil
package fossil.schollz.com/hello-world.fossil: cannot download, fossil.schollz.com/hello-world 
uses insecure protocol

It seems that the current fossil support only supports chiselapp.com - I refer to this change that added support.

Is there not yet support for self-hosted fossils?

@AlexRouSg

This comment has been minimized.

Copy link
Contributor

commented Jun 11, 2018

The correct command is go get -insecure fossil.schollz.com/hello-world.fossil
Your host is defaulting to http and go get doesn't like that so to force it to use http you need to pass the -insecure flag

@schollz

This comment has been minimized.

Copy link
Author

commented Jun 11, 2018

Thanks for your help. It actually still doesn't work for me though.

> go get -insecure fossil.schollz.com/hello-world.fossil
can't load package: package fossil.schollz.com/hello-world.fossil: 
no Go files in /home/zns/go/src/fossil.schollz.com/hello-world.fossil

I don't think it's processing the Go files correctly, as there is a go file in the main branch at the root folder.

But it does seem to create a directory, although there is also nothing in it (including the fossil repo):

> tree $GOPATH/src/fossil.schollz.com
/home/zns/go/src/fossil.schollz.com
└── hello-world.fossil

1 directory, 0 files
@AlexRouSg

This comment has been minimized.

Copy link
Contributor

commented Jun 11, 2018

Using raw fossil commands results in errors, is the repo setup correctly and can you clone manually with fossil?

@AlexRouSg

This comment has been minimized.

Copy link
Contributor

commented Jun 11, 2018

Sorry, I haven't used fossil before and was using the wrong url. Looks like go get is too.

cc @ksshannon @ianlancetaylor

go get is calling fossil clone fossil.schollz.com/hello-world however the correct usage would be fossil clone http://fossil.schollz.com/hello-world or fossil clone https://fossil.schollz.com/hello-world

@bcmills bcmills added this to the Go1.12 milestone Jun 11, 2018

@ksshannon

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2018

@schollz, there could be a few thinks going on here. If you have a public URL I can test against, it'd probably be easier. You should set the tag to something like:

<meta name="go-import" content="fossil.schollz.com/hello-world fossil https://fossil.schollz.com">

or something similar.

This is done a little differently depending on if your fossil is before or after the drafts enhancement was implemented.

I'd be glad to dig into it, but I need to be able to replicate. For posterity, please report your fossil version as well. We'll get it figured out.

@AlexRouSg

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2018

@ksshannon The URL is public btw, ohh and I tried 2.6 direct download from https://www.fossil-scm.org/xfer/uv/download.html

@ksshannon

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2018

@AlexRouSg, I believe it's the head metadata. It works the same with git hosting.

@AlexRouSg

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2018

@ksshannon The metadata error would explain the error with go get -insecure fossil.schollz.com/hello-world, however go get -insecure fossil.schollz.com/hello-world.fossil doesn't complain about metadata and progress until fossil errors out.

@AlexRouSg

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2018

go get -insecure -v fossil.schollz.com/hello-world.fossil
# cd .; fossil 
Usage: fossil COMMAND ...
   or: fossil help           -- for a list of common commands
   or: fossil help COMMAND   -- for help with the named command

Commands and filenames may be passed on to fossil from a file
by using:

    fossil --args FILENAME ...

Each line of the file is assumed to be a filename unless it starts
with '-' and contains a space, in which case it is assumed to be
another flag and is treated as such. --args FILENAME may be used
in conjunction with any other flags.
# cd .; fossil 
Usage: fossil COMMAND ...
   or: fossil help           -- for a list of common commands
   or: fossil help COMMAND   -- for help with the named command

Commands and filenames may be passed on to fossil from a file
by using:

    fossil --args FILENAME ...

Each line of the file is assumed to be a filename unless it starts
with '-' and contains a space, in which case it is assumed to be
another flag and is treated as such. --args FILENAME may be used
in conjunction with any other flags.
fossil.schollz.com/hello-world.fossil (download)
# cd .; fossil clone fossil.schollz.com/hello-world /home/asdf/go/src/fossil.schollz.com/hello-world.fossil/.fossil
unknown repository: fossil.schollz.com/hello-world
package fossil.schollz.com/hello-world.fossil: exit status 1

@ksshannon

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2018

@AlexRouSg go get -insecure fossil.schollz.com/hello-world.fossil is a 404, same as wget http://fossil.sholiz.com/hello-world.fossil so I may need to clean up some of the std out dump, but I think that is a different issue. I'll look further if the metadata tag doesn't solve this issue, and then we'll move on.

@AlexRouSg

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2018

@ksshannon Yes it's a 404, but looking at the -v output, it calls fossil clone fossil.schollz.com/hello-world so go get is stripping the .fossil.

This is going into undocumented territory ... someone should clarify whether go get myurl.com/repo.vcs needs to actually exist or if it's just a way to denote vcs and the url used is myurl.com/repo

If it's the latter then the issue is fossil clone myurl.com/repo is not a valid fossil command, it requires the protocol part as well and should instead run fossil clone http://myurl.com/repo or fossil clone https://myurl.com/repo

@ksshannon

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2018

@AlexRouSg No offense, but I'd like to solve @schollz 's issue first. We can file more issues after if we need to fix fossil support. I'm sure there are issues, but I'm not sure they should fall under this one.

@schollz

This comment has been minimized.

Copy link
Author

commented Jun 12, 2018

@ksshannon Thanks for helping too.

> fossil version
This is fossil version 2.6 [9718f3b078] 2018-05-04 12:56:42 UTC

The URL (https://fossil.schollz.com/hello-world) is indeed live and a working fossil. You can try it yourself with:

> fossil clone https://fossil.schollz.com/hello-world hello-world.fossil
> fossil open hello-world.fossil
> go run main.go
hello, world

Let me know if I can provide any other information, or if you can point me in the right direction I can try digging too. I'm not so familiar with the Go source yet though.

@ksshannon

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2018

@schollz read the docs at:

https://golang.org/cmd/go/#hdr-Remote_import_paths

And set the meta tag it mentions for you repository. You should be able to add it at:

https://fossil.schollz.com/hello-world/setup_skinedit

@schollz

This comment has been minimized.

Copy link
Author

commented Jun 13, 2018

@ksshannon Awesome. It works.

Here's the header I added:

<meta name="go-import" content="fossil.schollz.com/hello-world fossil https://fossil.schollz.com/hello-world">

It is now go get-able:

> go get -u -v fossil.schollz.com/hello-world
Fetching https://fossil.schollz.com/hello-world?go-get=1
Parsing meta tags from https://fossil.schollz.com/hello-world?go-get=1 (status code 200)
get "fossil.schollz.com/hello-world": found meta tag get.metaImport{Prefix:"fossil.schollz.com/hello-world", VCS:"fossil", RepoRoot:"https://fossil.schollz.com/hello-world"} at https://fossil.schollz.com/hello-world?go-get=1
fossil.schollz.com/hello-world (download)
> tree $GOPATH/src/fossil.schollz.com
/home/zns/go/src/fossil.schollz.com
hello-world
  main.go
  README.md

1 directory, 2 files

Thanks for your help @ksshannon and @AlexRouSg

@schollz schollz closed this Jun 13, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.