Very similar to fsnotify/fsnotify

[niemeyer]

Hello there,

The very recent creation of this repository was brought to my attention, with some concern about the extreme similarity with fsnotify/fsnotify. It sounds somewhat unfortunate to have a project that is so similar to a very well known project, from known contributors to the community, and with the exact same project name.

I would suggest donating this repository to Nathan Youngman in respect for his many contributions, and creating the similar project in a different repository that would more clearly identify itself in a distinct way.

Thanks for your understanding.

[jodok]

it's even worse. if i try to download fsnotify as dependency via gopkg.in the default location for the fsnotify package is this repo. i realized this when trying to build oauth2_proxy:

➜   curl -i gopkg.in/fsnotify.v1
HTTP/1.1 404 Not Found
Date: Tue, 20 Mar 2018 22:53:19 GMT
Content-Length: 106
Content-Type: text/plain; charset=utf-8

GitHub repository at https://github.com/go-fsnotify/fsnotify has no branch or tag "v1", "v1.N" or "v1.N.M"%

[niemeyer]

@jodok Right, gopkg.in always resolves gopkg.in/foo as go-foo/foo, unambiguously.

In other words, it's not just the default.. it's the only place it resolves to.

[niemeyer]

Turns out the issue is even more problematic, because fsnotify/fsnotify was in fact named go-fsnotify/fsnotify before being renamed:

fsnotify/fsnotify@9297c46

As a consequence, there are actual projects in the wild attempting to import this project, and breaking right now. Once there is code here, they will transparently build the diverging code, silently. Unless there is some agreement here, I may have to block this repository, which is a first for gopkg.in.

Ironically, the rename creates exactly the sort of breakage that gopkg.in is meant to avoid in the first place. But we of course can't prevent people from moving the code around.

[xjtian]

I recently had a build break with the same 404 @jodok posted above. It seems like until this repository is taken down, dep ensure will continue to fail.

Out of curiosity, @niemeyer if this was the only place fsnotify resolved to on gopkg.in, how did this dependency ever resolve properly? Does gopkg.in define some kind of fallback repository URL which points at fsnotify/fsnotify?

[niemeyer]

@xjtian I've just responded to that above. Code worked because this was the real repository name before it got renamed. Nothing will fix your code to point to the right location until the code is fixed, because the repository location was changed by the maintainers of fsnotify/fsnotify.

[go-fsnotify]

Hi, I just pushed a readme on master to explain what is going on. This account/repository were registered to block potential dependency takeover attacks. The account is in the process of being delegated to Nathan.

[niemeyer]

@go-fsnotify Thanks, but who are you?

Note that there's no reason to register an account to own an organization, and delegate administration to a different user.

[ldionmarcil]

Hey, the go-fsnotify org is now under @nathany 's control. I'll write a short piece about Go dependency injection later this month, I'll be sure to mention it on Gopher's slack.

[niemeyer]

@nathany?

[fabxc]

I'm struggling to understand what all of this is about. It sounds like someone intentionally broke everyone depending on gopkg.in/fsnotify to prove a point?

[bwplotka]

The question is: Whoever did that.. have you tried to reach anyone about this exploit prior breaking everyone depending on gopkg.in/fsnotify? Maintainers of gopkg.in or fsnotify?

The problem is quite bad now because the whole redirect was to gracefully perform the migration to new import. Now it's almost impossible.

@nathany if you got control of this org, maybe it's worth to clone the fsnotify here temporarily as mitigation?

[DamareYoh]

So uh, Im glad that someone on the white hat side of things plugged up this security hole over some bad actor.

However, this was poorly handled and has led to preventable frustration. First, theres absolutely no reason why this repo remained undocumented and empty for well over 8 hours. The lack of a readme or any contact from the opener of this repo caused no small amount of chaos and confusion in consumers of fsnotify. Also, that the maintainer of gopkg.in was not also notified immediately prior to opening this repo is also sloppy because they had equal or greater stake in this security problem than the maintainer of fsnotify.

If you must break a bunch of people’s stuff in the name of claiming the discovery of a vulnerability, please do a better job of making your intentions known to the people you broke next time.

[niemeyer]

@DamareYoh We have no evidence that the intentions were actually good (or bad), and in terms of data points we still haven't heard from @nathany.

I'll patch gopkg.in today to make it redirect to the proper repository.

[DamareYoh]

@niemeyer, on that, I went digging in the gophers slack yesterday, and I found this: https://gophers.slack.com/archives/C029RQSEE/p1521584215000430

So @nathany seems to be aware of the problem, and has been in contact with the opener of this repo.

But still, the communication is obnoxiously poor. And for seemingly no reason.

[ldionmarcil]

Hi, I registered the account yesterday in order to test a theory (which I honestly thought would fail) where it is possible to register an account that is being used by a Github "redirect". I expected Github would block this and it clearly did not. I instantly notified the maintainer (through Twitter and Email) and we started the process of delegating the account in order to mitigate eventual breakage. I think Nathan has been AFK for over 12 hours now, so that is why the communication is poor. I'm not in control of this org/repo any longer.

As you can see in the Slack converstion linked previously, I was first told to fork the legitimate repository, but that would have been silently poisoning projects (with legitimate code). This would be a half-measure, since the project would eventually be out-of-sync. Silently forking the repository would also mean that a large amount of projects would never notice that the projects they are pulling does not come from the right repository. I think this is why I was then told to put a notice message stating why things were broken, and how to fix it (to the best of my understanding at the time).

Fixing this right now: If you use gopkg, simply fix your imports to point to the fsnotify/fsnotify.v1 repository.
gopkg.in/fsnotify.v1 → gopkg.in/fsnotify/fsnotify.v1

Again, sorry I broke your builds, this could have been handled better. I'm working on enumerating all vulnerable packages and will coordinate with @niemeyer and vulnerable package owners in order to mitigate this once and for all. When this will be done, the plan is to send it to Github to make it no longer possible to register accounts that are redirects, mitigating this type of situation. This is especially bad considering someone could be transparently serving malicious code through this repository.

[DamareYoh]

Thanks for the clarification. In turn, I'd also like to clarify that my concern is not that the builds were broken (I think breaking them in this case was the right thing to do) but that it took 8 hours for us to find out why they were broken.

[niemeyer]

@ldionmarcil Agreed, you forking the repository would have been even worse as people would wonder if the content was legitimate. To be clear, I'm not blaming you either. The point I made was that really we have no actual evidence about whether the intention is good or not because the only data we have is the conversation here, which was started by the community after the fact.

About the coordination with me, note that this all unrelated to gopkg.in, in that any repository that moves away and leaves the original URL unattended will have the exact same issue. Having gopkg.in here actually helps because we can move people back to the right location, but I have no plans for doing that regularly.

[guineveresaenger]

Because our project relies on pinning an older version of revel, which is now broken, I'm going to spend several days figuring out how to work around fixing the breaking changes to our code base. After spending a whole day trying to figure out why my app was no longer working.

Vulnerabilities are a bad thing, but I would like to add my strong displeasure at the way this went down and is still being left in current broken state.

[niemeyer]

@guineveresaenger Clearly I'm all for stability and although it's not an issue in gopkg.in, I'll add a workaround to fix a problem today so you don't need to fix everything else there.

With that said, I'm personally giving the creator of that problem the benefit of the doubt. We have no proof that the intention was good, but we also have no proof that the intention was bad. Registering a project in GitHub is very quick and uncontroversial, and easy to miss the dimension of the problem created for others.

[guineveresaenger]

@niemeyer appreciated; thanks! I'll follow this thread.

[isra17]

@guineveresaenger you can simply fork the pinned version you need and apply the one-line fix for this issue. Just saved you a few day of work ;)

[niemeyer]

Okay, https://gopkg.in/fsnotify.v1 now redirects to the renamed repository.

I also took the chance to move the server into Google Compute Engine. So please let me know if you see any issues related to this or otherwise.

[guineveresaenger]

@isra17 unfortunately it's not a direct dependency. I can submit a patch to the version though. :)

[niemeyer]

@guineveresaenger Take your time now.. that redirect won't go away while gopkg.in exists.

[guineveresaenger]

@niemeyer I can't seem to go get that:

go get gopkg.in/fsnotify/fsnotify.v1
# golang.org/x/sys/unix
../../../golang.org/x/sys/unix/syscall_darwin.go:49:9: undefined: RawSockaddrDatalink

[niemeyer]

@guineveresaenger That's an error outside of the package. The repository retrieval and compilation is working fine here, at least.

[nathany]

Thanks @niemeyer for adding the gopkg.in redirect.

Apologies for the poor communication on this. It was my mistake for relying on the GitHub redirect and not considering the security implications.

See also fsnotify/fsnotify#108 for the original reasoning behind moving away from gopkg.in and version numbers in import paths (this happened over two years ago).

[niemeyer]

I'm restating this here since it's where the information around this problem is being centralized.

@ldionmarcil stated this elsewhere:

Hi. I am in the process of reporting this to affected organizations. Yes, you can mitigate this by pointing directly to github.com/fsnotify/fsnotify. You can still use gopkg.in if you need a specific version of this project in order to fetch the repository but be sure to always use gopkg.in/{org_name}/{repo_name}.

This is completely incorrect information, and helps no one. Not even you @ldionmarcil, as I bet you'd rather be providing correct information in your comments as a security researcher.

There's zero benefit in using "go-project/project.v1" vs. simply "project.v1" for repositories that are named like this. The problem that affected the "fsnotify" occurred because the project renamed their repository from "go-fsnotify" to "fsnotify". If they were using "gopkg.in/go-fsnotify/fsnotify" as the import path or even "github.com/go-fsnotify/fsnotify", and renamed the repository, the exact same problem would happen.

So the problem is completely unrelated to gopkg.in, and the main lesson here is also general: do not leave your existing URL unattended if you know the community is trusting it to be there. In fact, that's true even if you have a custom domain name.

[ldionmarcil]

I agree that discussing on the other repository is not productive. Let's take it back here.

I think there is value in mentioning that gopkg.in will append go- to the repository name if it is not specified. This is the sort of behaviour that can lead developers into not realizing (or forgetting) that using the gopkg.in/{repo_name} import value is actually pointing to github.com/go-{repo_name}/. It took literally two years for people to notice that their gopkg.in/fsnotify imports were actually pointing to a non-existing organization.

Had the developers been recommending gopkg.in/{org_name}/{repo_name} initially, users of the library could have noticed the mismatch between their import statements and the Github organization. You are free to think I am saying nonsense, but I will continue to state that the default behaviour of gopkg.in is weird and it will definitely lead to this type of scenario again in the future.

You say the problem is "completely unrelated to gopkg.in", but I am not sure of that. As I said, the abstraction of the organization name, derived from the repository name, is definitely odd and I feel like it had a role to play in this not being observed for over two years.

** I edited this because I stated something that was wrong in regards to instant breakage following the organizational rename

[niemeyer]

The entirety of your message is incorrect, @ldionmarcil.

The message above was edited and softened, so this is now unnecessarily harsh.

[ldionmarcil]

You seem to think I am putting blame on the gopkg.in project, and I'm not sure why. What I am stating is that the default behaviour when no organization/username is used is weird, and I stand by that. I'm not saying you caused this, or anything close to that. I recommended that users use the full {name}/{package} for a reason: it is immensely more clear where your package actually comes from. I don't appreciate the hostility, especially since I am trying to verbalize the behaviour of your project.

[niemeyer]

Hey, I was the one defending your back a few message above, remember?

It's not about blame, and I couldn't care less if you like gopkg.in or not. It's about factually incorrect statements such as the one I quoted above. Your messages were also edited several times after I said to please do some research, because they were also factually incorrect.

So I'm simply asking to please stop and do some research so you can understand better what happened, so that people can trust you in the future, and so that I don't need to go after your traces fixing these bogus comments.

Thank you!

[AlekSi]

If anyone still having problems and uses dep – that worked for me:

[[override]]
  name = "gopkg.in/fsnotify.v1"
  source = "github.com/fsnotify/fsnotify"