404-page "go get" magic isn't recognized

[kylelemons]

This works:

go get camlistore.org/pkg/fileembed

but it doesn't show up in pkgdoc (because the magic is on a 404 page, I think)

I suspect the following line:
https://github.com/garyburd/gopkgdoc/blob/master/doc/doc.go#L57

[simonklee]

Out of curiosity (as I'm not maintainer of gopkgdoc);

a) What kind of behavior do you expect gopkgdoc perform when faced with the 404. It seems go get's answer is to try the root URL and fetch everything available under https://camlistore.org?go-get=1 Is this what you want gopkgdoc to do?

b) Fetching the root URL would also fetch every other pkg, correct? It seems that you potentially can end up with a very big file download. Like fetching camliststore.org adds about 20MB of data, even though we just requested that single package which is 16K.

[garyburd]

The service used by GoPkgDoc to fetch arbitrary Git repositories is down. I notified the owner of the service.

The line at doc.go#L57 duplicates the logic in go get. See http://code.google.com/p/go/source/browse/src/cmd/go/http.go#66. I asked Brad about this line when I wrote the code. He said that it's intentional. The search path is: https with status == 200, http with any status.

[garyburd]

re a) GoPkgDoc attempts to duplicate the logic in the go get command at http://code.google.com/p/go/source/browse/src/cmd/go/vcs.go#422.

The logic is as follows:

1. Look for meta data on https page.
2. If https page does not return with status 200, then look for meta data on http page with any status including 404.
3. If meta data is found and it is not at the root of the repository, then repeat the search at the root of the repository. Confirm that the meta data at the root agrees with the meta data for the import path.

re b) The code is searching for meta data on HTML pages. It does not download any data from a VCS. Once the meta data is found, GoPkgDoc fetches the code from the VCS specified by the meta data. With the exception of launchpad.net repos, GoPkgDoc gets the code for exactly that import path and not any child import paths.

[garyburd]

The backend service is up and running again.

[simonklee]

Thanks for the explanation, Gary.

[kylelemons]

Sorry for the noise!