x/pkgsite: API for pkg.go.dev #36785
Comments
@rhcarvalho - it would be helpful to get a sense of what your current use cases are for api.godoc.org, and feature requests are for an API for pkg.go.dev. It sounds like getting the importers for a package is one of them. With pkg.go.dev being module aware, what specific information about importers would be useful to surface via an API? For example:
Additionally, what other information would be useful to you to surface via an API? /cc @tbpg who has also mentioned wanting an API for pkg.go.dev |
Without much thought, having an API to answer the more specific question "what are the importers of a specific version of a package" would make plausible to derive the answer to the other items in your list. At the moment I consume the godoc.org API and scrape data from pkg.go.dev to answer the question "who uses my package". As far as I can tell the data in the "Imported By" go.dev tab is unrelated to the version of the package I'm currently browsing. Here are the other endpoints in the GoDoc API: https://github.com/golang/gddo/blob/7365cb292b8bfd13dfe514a18b207b9cc70e6ceb/gddo-server/main.go#L901-L904
So if we need a more specific request, here it is: api.go.dev MVP
|
Another endpoint idea (standalone or as part of a set of info returned for a module): available versions of a given module. |
What information beyond |
There are some cases where
None. That works. :) |
I'd be wary of encouraging tool authors to query pkg.go.dev/a proxy directly. Because it could well introduce skew compared to the answers from |
In my use-case I don't want to execute |
In #37952 I raised the question of whether module/package license file information could be surfaced in the output of @ianthehat instead suggested exposing the information via a pkg.go.dev API, leveraging the fact that the content and presentation on pkg.go.dev has already jumped through the relevant legal hoops. This comment is therefore to explicitly request that we include license file information in the API. Thank you! |
Currently to get the publish times for all versions, you have to make N+1 requests: one for
Anyone know if there are plans to fix this? |
No immediate plans. We currently gather that information from import statements in the code, so there is no version information attached. The So we understand it's an approximation and we want to fix it, but it's going to take some time. |
@julieqiu my use for an API would be to:
|
@julieqiu oh and please do not retire http://api.godoc.org/packages unless there is an alternative! |
As a downstream package maintainer for Fedora, I'm also interested in an API. We have our own tool, Anitya, to track package releases, but it was not designed to track GIT commits. And many Go packages still don't publish version. So any information about latest published commit, with info like date of the commit, new dependencies, would be very helpful. I would gather data from the API in Python and compare it to the latest version we have for a given package. I'm also interested in getting the License info, so we could find recursively all the licenses used in a static binary. |
and BTW my general use case is for https://github.com/nexB/scancode-toolkit and related projects to provide my users with license, origin and dependencies details. And for https://github.com/nexb/vulnerablecode where we provide vulnerabilities details. |
My present use case is discovering major version upgrades. Currently, we have to scrape pkg.go.dev. An alternative to pkg.go.dev API, would be if there were an "official" way (via |
@StevenACoffman I've updated gomajor to find newer versions using the module proxy. The performance is acceptable. |
My use case is importing modules/packages to another package manager (Guix) where builds are completely reproducible and monolithic vendoring is strongly discouraged. Having a JSON API would simplify the obtention of metadata like licenses, brief package descriptions, dependencies and hashes. |
I didnt see it mentioned yet, I found this link to be helpful: https://github.com/golang/gddo/wiki/API I hope similar API will be available for new site, or at least maintain the existing API. |
We'll be keeping |
@jba with all due respect, that comment is pretty vague. I am actively using that API: https://github.com/89z/sienna/blob/master/go-repos/download.go so if it goes down, Im just going to end up cURLing this: https://pkg.go.dev/search?q=Query and scraping the HTML, as I suspect others will be too. So any further detail would be appreciated. |
Our current plan is to keep |
Prior to pkg.go.dev, godoc.org has had a JSON API that can be used to, among other things, discover importers of a given package.
Example: https://api.godoc.org/importers/golang.org/x/net/html
Given that pkg.go.dev does a much better job at tracking importers thanks to Go Modules and the Module Proxy, it would be nice if the community could get access to a public API similar to that of godoc.org.
The text was updated successfully, but these errors were encountered: