Valid version of a go package throws unrecognized import path error and breaks dependabot

[jeffwidman]

Saw this in our dependabot logs and from a quick glance it appears to be a bug in dependabot logic, not go tooling... but I could be wrong:

go: github.com/hashicorp/vault@v1.9.0 requires
	layeh.com/radius@v0.0.0-20190322222518-890bc1058917: unrecognized import path

Looking at the logs, I see a little more info:

updater | INFO <job_234135973> Handled error whilst updating github.com/deepmap/oapi-codegen: 
dependency_file_not_resolvable {:message=>"go: github.com/hashicorp/vault@v1.9.0 
requires\n\tlayeh.com/radius@v0.0.0-20190322222518-890bc1058917: unrecognized import path"}

I poked through old PR's and it looks like typically this error happens when something 404's... but this appears to be a valid/reachable import path:

$  go list -m -versions layeh.com/radius
layeh.com/radius

Any ideas on what the problem might be? Happy to help fix, but would appreciate a pointer on where to go...

[jurre]

Thanks for reporting this @jeffwidman. Do you happen to have a repo that we can reproduce this against?

The unrecognized import path error itself does come from the go tooling natively, but might be something about how we run the tooling or handle the error?

We catch the error in these classes:

dependabot-core/go_modules/lib/dependabot/go_modules/update_checker/latest_version_finder.rb

Line 22 in 5db0e3f

/unrecognized import path/,

https://github.com/dependabot/dependabot-core/blob/dependabot/pip/python/helpers/pipenv-2021.11.15/go_modules/lib/dependabot/go_modules/file_updater/go_mod_updater.rb#L40

[jeffwidman]

Thanks @jurre, my employer probably prefers to keep repo paths/names semi-private, so I emailed you an example of a repo where we are seeing this error.

Based on the code you pointed to above, it looks like somewhere in the chain is 404'ing... and sure enough:

curl -I layeh.com/radius
HTTP/1.1 301 Moved Permanently
Date: Fri, 19 Nov 2021 17:54:59 GMT
Connection: keep-alive
Cache-Control: max-age=3600
Expires: Fri, 19 Nov 2021 18:54:59 GMT
Location: https://layeh.com/radius

curl -I https://layeh.com/radius
HTTP/2 404
date: Fri, 19 Nov 2021 17:55:15 GMT
content-type: text/plain;charset=UTF-8
content-length: 13

What's weird is that go get and go list continue to work just fine despite the 404:

$ go get layeh.com/radius
go: downloading layeh.com/radius v0.0.0-20210819152912-ad72663a72ab
$ go list -m -versions layeh.com/radius
layeh.com/radius

I agree this error is being emitted by native go tooling, but given the above, perhaps Dependabot is invoking the command incorrectly... that or go has fixed some tooling to handle this case, but not all the tooling.

I won't have time for a few weeks to get re-spun-up with dependabot locally in order track down exactly which command invokation is throwing the error. Feel free to look into it if you get time. Once we know the exact go command is throwing the error, then it should be simple to repro outside of dependabot and dig into exactly what the problem is.

[mctofu]

I think this is because layeh.com/radius has been previously cached by the public proxy. If you disable the proxy then you'll see the same issue:

$ GOPRIVATE=* go list -m -versions layeh.com/radius
go list -m: unrecognized import path "layeh.com/radius": parse https://layeh.com/radius?go-get=1: no go-import meta tags (meta tag https://layeh.com/radius did not match import path layeh.com/radius)

[jeffwidman]

Oh interesting, nice find @mctofu. I always forget about the permanence of that cache.

Let me dig on this a little.

[jeffwidman]

Fixed upstream in layeh/radius#98.