Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change import path to github.com/ugorji/go-codec #43

Open
ugorji opened this Issue Jan 8, 2015 · 20 comments

Comments

Projects
None yet
7 participants
@ugorji
Copy link
Owner

ugorji commented Jan 8, 2015

It would be nice if the import path mapped directly to a URL to the source code.

Currently, it doesn't, and for no good reason. Initially, we used a repository name of go, with the
hope of putting multiple packages into the same repository. However, in retrospect, that is not a
good idea; one repository should only hold one package, with possible sub-packages which relate
directly to that package.

The plan of action is:

  • Use github to change repository name from go to go-codec
  • Move all files from the codec sub-directory of the repository to the top-level directory
  • Keep the codec subdirectory there, and create a codec/fail.go file

The new codec/fail.go file will contain

  • package docs that tell users the new import path and where to find docs
  • init method that panics with an appropriate error message
  • possibly use the // import label to allow go get fail early

We hope to do this soon, likely by the end of january.

@dmitshur

This comment has been minimized.

Copy link

dmitshur commented Mar 25, 2015

This sounds like a good change.

We hope to do this soon, likely by the end of january.

Any updates?

@ugorji

This comment has been minimized.

Copy link
Owner Author

ugorji commented Mar 30, 2015

Sorry, no updates yet.

I'm not yet convinced that the benefits outweigh the compatibility cost.

@warpfork

This comment has been minimized.

Copy link

warpfork commented May 16, 2015

+1 to rename, fwiw.

@drewwells

This comment has been minimized.

Copy link

drewwells commented Jun 25, 2015

Also, forking your project becomes less practical, since it requires github.com/{myname}/go is available.

@ugorji ugorji added the enhancement label Oct 29, 2015

@ebehar

This comment has been minimized.

Copy link

ebehar commented May 17, 2016

Adding on to other comments above from last year, remedying the compatibility change is quite easy, and renaming the repository would be much cleaner for consumers of it.

I feel like everybody should be able to search and replace using their favorite tools to change the import paths pretty easily.

If that's concerning though, could you alternately fork the repo to go-codec, move things around to make it the way you want it, and then make go-codec the canonical repo for these codecs? That way consumers of the package can migrate to the new repo name on their own time, and the old repo can be deleted eventually.

@ugorji ugorji added the wontfix label Dec 10, 2017

@ugorji

This comment has been minimized.

Copy link
Owner Author

ugorji commented Dec 10, 2017

Currently, per sourcegraph, over 400 repositories depend on this library. Making such a change will be extremely disruptive to folks.

Unfortunately, I have thought long and hard about this, and there's no clean way to do it without un-necessary disruption. I liken this to backward-compatibility: sometimes it sucks, but it is necessary to carry forward.

Closing for now.

@arizvisa

This comment has been minimized.

Copy link

arizvisa commented Nov 28, 2018

The following was referenced in #275.

This was left unanswered in the prior closed issue, but is there some reason why you don't just keep the original repository name, and just add another repo as go-codec?

You can setup git to push to two different remotes at the same time so it really doesn't cost you anything (such as having to double-maintain), and since the repository name is optional (a user can choose to use the suggested name or the prior backwards-compat name), it really doesn't affect users/stale-projects and hence doesn't affect backwards compatibility during building in any way.

But this way at least people have an option to migrate to the new name over time. A number of your forks exist explicitly for renaming it to "go-codec" or "go-msgpack", so your users have definitely recognized that your initial project name "go" was a mistake. You could even point people to the https://gopkg.in url of your project in your docs to include the version as part of the package name.

This library is pretty much yours so you can do whatever you want with it of course, but there will definitely come a time when you will stop maintaining this project. And if this project follows the traditional open-source lifecycle, then someone else will likely inherit it. If somebody else implements a project called "go-codec" then they also have the option to impersonate your project. Before that happens, however, it'd be nice to get a solution for that the namespace collision available before that point happens I guess.

Anyways, it's all up to you of course. I just saw the comments/discussion that happened and don't think that aliasing the name of a project written in a relatively young language such as go is as intrusive as one might believe.

(edited to remove some unnecessary verbiage)

@ugorji

This comment has been minimized.

Copy link
Owner Author

ugorji commented Dec 1, 2018

In retrospect, I am sort of happy that this change wasn't done, for the following reasons:

  • Single (mono) repo for storing all go libraries/tools, analogous to github.com/golang/tools
  • Compatibility: every user has to change their code
  • Source code churn: changing the package name involves moving everything one directory up, which completely messes up the git history

Let me hit on each one by one

Single Repo

With modules, it is possible now to have multiple packages inside the same repository.

Also, we now have some precedent with the https://github.com/golang/tools, which is a single repository that hosts the source for multiple tools managed by the core go team, including blog, cover, godoc, imports, present, playground, etc. Module support will help version the different tools, even though they all are hosted in a single github repo.

This also aligns with the way I hear code is stored in google and facebook, and the react library, in a single mono-repo.

The analogy is that my go code that I share with the world is stored at github.com/ugorji/go, and includes the codec library, codecgen tool, etc.

That's one part.

Compatibility

If we change the package name, then every user has to update their code, or we split the userbase into old and new. We see how that has not worked out well when folks do it, and my package is not as popular that everyone will happily bite the bullet.

Look at https://godoc.org/github.com/ugorji/go/codec (scroll to the bottom): it says that my package is now used by 2214 packages. That is way too much to force a change to. And we still have no defined way to make announcements.

Source code churn

Changing the package name involves moving everything one directory up, which completely messes up the git history. I don't want to do it unless I have to.

For these reasons, making the change is a no. Folks are generally ok with it, and the ecosystem (with modules, etc) now encourages this model of a single repo containing multiple packages - unlike before where this wasn't truly supported.

Hope this helps explain the position.

In the future, I hope a service comes that provides true canonical package names in a flexible way. Currently, the "import paths metadata" doesn't support this, as it still expects that the canonical name and the repo have the same base path. Consequently, I can use it to map domainName/go/codec to github.com/ugorji/go/codec, but not to map domainName/go-codec. I would hope that the "godoc.org" service provides something like this, but it requires more flexible support from the go tool first.

I may follow up on this with the go tools teams in the future.

@ugorji

This comment has been minimized.

Copy link
Owner Author

ugorji commented Dec 1, 2018

@arizvisa ^^ (see comment above)

@arizvisa

This comment has been minimized.

Copy link

arizvisa commented Dec 2, 2018

Fair enough. Extending on your points though:

  • Single-Repo
    The difference is that golang/tools is a number of tools/commands with shared code, whereas your go repo is (at the moment) for a single library. This'd be consistent if your other go project (go-codec-benchmark) would be in this same repo, but you keep them as separate distinct projects. You can retain git history and leave each reference as a submodule so that everything is accessible from the same place. But again, it's just something to think about as you develop more successful libraries in the future.

  • Compatibility
    It's not actually a change, but rather an alias. This is just due to your naming of the project as go-codec which is just different from everything that references it themselves. But you made your point. It is just aesthetics after all. ;)

  • Source code churn
    You shouldn't need to move everything up which'll force users to change code. You can even keep it as a submodule (an alias to the actual project) and in no way require users to shuffle code around. To ease your development you can push to both places at the same time. (Commit history can also be retained using git filter-branch when moving it into a separate github project.) But, this way the git history for each different go project can still be tracked and everything under user/go could be what you'd consider "stable" (in a sense).

Tbh, I prefer your method of using user/go to organize all go code libraries underneath it. As this way the separation of development/experimental and stable branches can be controlled by synchronizing a submodule of "stable" code under one's user/go repository. This can be used to deal with versioning in a project repo and since go get doesn't recognize tags/branches. This solves the problem of go get always just fetching the latest commit and simply ignoring what we tag in git. Of course, though, the work for this only makes sense when there's more than a 1-man army working on a project.

As an example, like even tho you want to complete deprecate go-msgpack (and you've made a lot of effort to deprecate that library). To retain backwards compat (even tho I think your effort has been successful), you could keep a submodule in user/go as user/go/msgpack which references go-msgpack, and another one user/go/codec that references go-codec. This way you have no fears with users who're still using go-msgpack.

Anyways, thanks for your work on go-codec, it's a great little library.

(edited to mention go-msgpack and to format bulletpoints)

@ugorji

This comment has been minimized.

Copy link
Owner Author

ugorji commented Apr 6, 2019

@arizvisa

Honestly, my biggest concerns are source code churn and compatibility.

Compatibility

If I change the repo name, then I am suggesting that folks change the import path in their codebase. This effectively splits the ecosystem, which go doesn't allow. I cannot have the same codebase support github.com/ugorji/go-codec/codec and github.com/ugorji/go/codec . This is even more critical with the advent of modules, where you MUST specify the module name in the go.mod file. This is why the "import path" comment was created since go 1.4, so that you could have an "alias" but enforce a single import path.

Source code churn

If I change the import path, then I WOULD like to remove the codec subdirectory, and move everything one directory up. That means that "github.com/ugorji/go/codec" would move to "github.com/ugorji/go-codec" with package name "codec". This would cause unnecessary source code churn, and cannot be solved while solving compatibility above i.e. it goes against compatibility above.

@ugorji

This comment has been minimized.

Copy link
Owner Author

ugorji commented Apr 6, 2019

I will open this issue back up, because it will be nice to solve it somehow. But if no clear solution is found, then I will just leave it as a reminder and let it be the legacy issue that cannot be solved.

@ugorji ugorji reopened this Apr 6, 2019

@ugorji

This comment has been minimized.

Copy link
Owner Author

ugorji commented Apr 6, 2019

I am currently considering a middle ground - but am not sold on doing it yet.

  1. Change repo name to go-codec
  2. github will transparently redirect go to go-codec
  3. Add an import path so that it MUST be imported as github.com/ugorji/go/codec
  4. DONE

This way, we change the repo name to match the function of the repo. Yet we maintain compatibility and have no source code churn.

I do not want to use submodules because they bring inconvenience, but seen now and unknown. Some of these concerns are nicely articulated at the bottom of https://github.blog/2016-02-01-working-with-submodules/

@ugorji

This comment has been minimized.

Copy link
Owner Author

ugorji commented Apr 8, 2019

Updated by 74d13ae

@dcarbone

This comment has been minimized.

Copy link

dcarbone commented Apr 10, 2019

This needs to be addressed asap. You pushed a breaking change in early march (moving the location of RawToString breaks packages that tried to use literal construction i.e. &codec.MsgpackHandle{RawToString: true}) that is causing me endless headaches. I have a dependency that uses this package as a dependency and while I have an accepted pr that addresses this (uses new() -> set syntax instead), it has not been merged into a stable tag.

This is a VERY big issue now that you have released another tag with this breaking change. Because you define your import path as github.com/ugorji/go, but do not have any actual code there, neither dep nor mod will allow me to constrain this repo properly.

edit dep will, thanks to the [[override]] block, but mod just produced headaches. I really hope something can be done here.

I am sorry if I sound harsh, but I have been fighting this for a month now and I grow weary of it.

@ugorji

This comment has been minimized.

Copy link
Owner Author

ugorji commented Apr 12, 2019

@dcarbone While I can understand the tendency to lash out when frustrated, I cannot accept this here. Next time you need to sound harsh and lash out, close a door and scream. Then compose yourself and make your point. It will go down much nicer next time.

This is the way this works, and anyone that has submitted a bug report can attest to this. You find an issue, you describe it, I resolve it typically within a few days. There's a reason why the bug count is typically about 0 (save for these reminder bugs I keep around).

Do us all a favor. Next time something bothers you, articulate your issue swiftly - and watch it get resolved very quickly. If you had brought the issue to my attention earlier, we might have resolved it before it stayed for so long. The main issues here is that I forgot that folks would use the literal constructors, and so that code is broken. That is a big shame - I wish someone had raised the flag early since I didn't realize it.

About the other thing i.e. dep and mod having an issue with the module path of github.com/ugorji/go - articulate your issue well in a new bug and I can try to address it. It may be as simple as putting a holder file in the module path.

But I can't afford this ridiculousness you did above. Behavior like that serves only as a detriment to the entire community, as it discourages folks from doing good work for free.

Hope this message reaches you well.

@dcarbone

This comment has been minimized.

Copy link

dcarbone commented Apr 12, 2019

I believe I articulated my issues very concisely. Using go mod, it is not possible to constrain this package properly as there is no code present in the path defined by the mod file. In addition, you cannot merely use a typical [[constrain]] block in a Gopkg.toml because, again, there is no code in the root of the repo. The same goes for Glide. All of these issue would be fixed if you were to change the package path as defined in this issue.

If you really want, I will open separate issues.

@ugorji

This comment has been minimized.

Copy link
Owner Author

ugorji commented Apr 14, 2019

I am currently considering a middle ground - but am not sold on doing it yet.

  1. Change repo name to go-codec
  2. github will transparently redirect go to go-codec
  3. Add an import path so that it MUST be imported as github.com/ugorji/go/codec
  4. DONE

This way, we change the repo name to match the function of the repo. Yet we maintain compatibility and have no source code churn.

I do not want to use submodules because they bring inconvenience, but seen now and unknown. Some of these concerns are nicely articulated at the bottom of https://github.blog/2016-02-01-working-with-submodules/

Many tools e.g. travis-ci, etc do not work well with a repo name different from the import path. It prevents us from getting this testing with multiple versions of go officially. Due to this, I will revert the repo name back to go.

@dmitshur

This comment has been minimized.

Copy link

dmitshur commented Apr 14, 2019

travis-ci, etc do not work well with a repo name different from the import path

Travis CI supports this well via the go_import_path field. See https://docs.travis-ci.com/user/languages/go/#go-import-path.

Look at .travis.yml of github.com/go4org/go4 for an example of a repository with a vanity import path using it.

@ugorji ugorji referenced this issue Apr 15, 2019

Closed

go-get error #289

@ugorji

This comment has been minimized.

Copy link
Owner Author

ugorji commented Apr 15, 2019

Keeping the repo name consistent with the package path has resolved a spurious issue arising in #289 (see #289 (comment) ).

This is a good reason to leave well enough alone.

@dmitshur ^^

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.