New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let's try a monorepo (or two)! #1

Closed
slimsag opened this Issue Mar 5, 2016 · 5 comments

Comments

Projects
None yet
3 participants
@slimsag
Member

slimsag commented Mar 5, 2016

Hi all! I've noticed over a long period of time that the excessive amount of repositories that Azul3D has is quite confusing for new users. Additionally, our versioning scheme does not match other Go packages and has been a huge source of confusion for users as well.

How it's caused us trouble

We've also been dealing with this issue in various ways for a long time:

And also not listed above, is the high development cost on me personally to ensure that we are keeping repositories in-sync with one another. Changes to azul3d.org/gfx.v2 require changes in a separate repo at azul3d.org/keyboard.v2, we must use govers etc. to update version numbers, and it's all quite confusing for newcomers.

What the future looks like

  • One github.com/azul3d/engine repository, and one github.com/azul3d/examples repository.
  • go get -u azul3d.org/engine
  • go get -u azul3d.org/examples/...
  • No more package versions.
    • The official way to do this by the Go team is to use vendoring. I highly approve of this approach, and it means users can get new fixes and features when they want them (rather than relying on me to release them first).
    • We will NOT try to avoid API-breaking changes. We will document these, and maybe even continue to use semver outside of our import paths, but otherwise not.
    • This means PRs that are merged will instantly be usable by users, instead of being delayed for long periods of time until the next release (sometimes months).
  • One issue tracker for all repositories.
    • I view this as a huge positive, but others might not like the noise. "Closed as unfortunate." :)

Potential Downsides:

  • The repository is large once merging all commits, 80MB+. This is because we were naive and committed binary files like wav files, textures, etc.
    • To fix this, we will switch to proper storage for these binary files, in specific https://git-lfs.github.com/
    • This does mean we will lose all old commit history (or would otherwise have to rewrite all of it, too much work / not worth it).
    • Users will now have to install git-lfs in order to: run the examples, tests, install native/al or native/freetype packages. Other packages are not affected. git-lfs is useful for end-user projects, too, for storing resources in git like textures, etc.
  • One issue tracker for all repositories.
    • I actually view this as a positive, but YMMV.

Other notes:

Installing git-lfs is actually very useful outside of Azul3D, too. If you're doing gamedev, you will not want to commit your assets to your repository in order to avoid the size growing outrageously each time you modify an asset. And it's very easy to install:

Linux:

curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
sudo apt-get install git-lfs
git lfs install

OS X (requires brew install):

brew install git-lfs
git lfs install

Windows (requires choco install):

choco install git-lfs.install
git lfs install

Current Status

  1. The plan is in action! We're moving forward!
  2. The monorepo at https://github.com/azul3d/engine is finalized.
  3. The monorepo at https://github.com/azul3d/examples is finalized (old repo at https://github.com/azul3d-legacy/examples).
  4. Old repos have been migrated to https://github.com/azul3d-legacy.
  5. Old repo's issues have been migrated to https://github.com/azul3d/engine/issues.
  6. The website has been updated to remove all versioning-related content, plus added a new blog post describing what has happened.

@slimsag slimsag changed the title from Let's try a monorepo! to Let's try a monorepo (or two)! Mar 5, 2016

@slimsag slimsag self-assigned this Mar 5, 2016

This was referenced Mar 6, 2016

@slimsag slimsag referenced this issue Mar 6, 2016

Closed

Multiple Windows #98

slimsag added a commit to azul3d/website that referenced this issue Mar 6, 2016

- Remove versioning page.
- Remove roadmap page (out of date, bring it back later maybe).
- Update packages list to be more of a summary.

Helps azul3d/engine#1
@dmitshur

This comment has been minimized.

Show comment
Hide comment
@dmitshur

dmitshur Mar 6, 2016

Contributor

👏 👍

Contributor

dmitshur commented Mar 6, 2016

👏 👍

@mewmew

This comment has been minimized.

Show comment
Hide comment
@mewmew

mewmew Mar 6, 2016

Member

+1 indeed.

This is definitely moving in the right direction, and while we don't know yet of all the hassles it will create, it certainly solves a number of long-standing issues with the old repo management.

Specifically, I approve of dropping the use of versioned import paths. They have caused much more pain than that created by the potential problems they may have been set out to solve.

I'll dig in a bit deeper in the next month or so, when I start to get more time on my hands.

Until then, best of luck and thanks for keeping us up to date :)

Cheers

  • u
Member

mewmew commented Mar 6, 2016

+1 indeed.

This is definitely moving in the right direction, and while we don't know yet of all the hassles it will create, it certainly solves a number of long-standing issues with the old repo management.

Specifically, I approve of dropping the use of versioned import paths. They have caused much more pain than that created by the potential problems they may have been set out to solve.

I'll dig in a bit deeper in the next month or so, when I start to get more time on my hands.

Until then, best of luck and thanks for keeping us up to date :)

Cheers

  • u

mewmew added a commit to mewspring/hashutil that referenced this issue Mar 6, 2016

mewmew added a commit to mewkiz/flac that referenced this issue Mar 6, 2016

@mewmew

This comment has been minimized.

Show comment
Hide comment
@mewmew

mewmew Mar 7, 2016

Member

Another potential downside to the merge of all repos into a single repository is that it decreases the granularity with which permissions may be distributed to developers and contributors of Azul3d.

In the previous setup, my brother and I had read/write access to the audio repositories of the Azul3d GitHub organization. Now, I can no longer close audio related issues, which creates an unnecessary administrative overhead that I have to ping @slimsag each time (e.g. #79 (comment), #76 (comment) and #71 (comment)).

Any thoughts on how we may resolve this without giving contributors to Azul3d full access to the entire project?

Member

mewmew commented Mar 7, 2016

Another potential downside to the merge of all repos into a single repository is that it decreases the granularity with which permissions may be distributed to developers and contributors of Azul3d.

In the previous setup, my brother and I had read/write access to the audio repositories of the Azul3d GitHub organization. Now, I can no longer close audio related issues, which creates an unnecessary administrative overhead that I have to ping @slimsag each time (e.g. #79 (comment), #76 (comment) and #71 (comment)).

Any thoughts on how we may resolve this without giving contributors to Azul3d full access to the entire project?

@slimsag

This comment has been minimized.

Show comment
Hide comment
@slimsag

slimsag Mar 7, 2016

Member

@mewmew Good point. I am not too concerned about contributors doing anything crazy, so I've gone ahead and given all prior contributors access to the engine repository. I think we are all good/responsible OSS citizens, so we should be fine.

Member

slimsag commented Mar 7, 2016

@mewmew Good point. I am not too concerned about contributors doing anything crazy, so I've gone ahead and given all prior contributors access to the engine repository. I think we are all good/responsible OSS citizens, so we should be fine.

@slimsag

This comment has been minimized.

Show comment
Hide comment
@slimsag

slimsag Mar 11, 2016

Member

Closing for now, if anyone runs into new issues regarding this please feel free to create a new one :)

Member

slimsag commented Mar 11, 2016

Closing for now, if anyone runs into new issues regarding this please feel free to create a new one :)

@slimsag slimsag closed this Mar 11, 2016

@mewmew mewmew referenced this issue May 24, 2016

Closed

meta: Reorganization of repositories #174

9 of 9 tasks complete

dmitshur added a commit to shurcooL/Go-Package-Store that referenced this issue Sep 17, 2016

presenter/github: Remove azul3d semver support.
This was originally added for semver-using azul3d.org packages. After
azul3d/engine#1, they've all moved into a
monorepo that no longer uses semver:

> ### What the future looks like
>
> - No more package versions.

So it's no longer necessary for most azul3d.org packages.

The only remaining package that needs this is the semver package
itself, which is not enough reason to keep this here (a single
package that is not actively developed).

See 2e0d1ee#commitcomment-19066368 for context. /cc @slimsag
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment