Merge LibLaserCut into VisiCut repository?

[phdoerfler]

We want a Monorepo (I do, and I hope you do, too)

I'd like to merge the LibLaserCut repository into this one. This can be easily done simply by pushing from LibLaserCut to this one, preserving the full history and everything.

Now, why would I want a Monorepo?

It might just be me, but I find dealing with multiple repositories annoying and rather time consuming, even more so for sub-repositories.
For instance, creating pull requests that depend on a modification in the other repository are prone to fail their CI check.
Refactorings that affect the whole project are impossible to do in atomically with multiple repositories.

A monorepo

• provides a single source of truth
• makes it easier share and reuse code easily
• makes it easier to manage dependencies (e.g., if you make a change, what else will be impacted?)
• allows to make atomic changes (e.g., one operation to make a change across multiple projects)

What this isn't

I'm not talking about combining both artefacts into one. Both VisiCut and LibLaserCut can and should still remain two individual maven artefacts / projects / poms that can be published and depended on independently.
I'd have a subdirectory for LibLaserCut in which it, together with its old change sets, can live happily ever after.

Is there a way back if we ever wanted to have separate repositories again?

Yes, all the changes will still be there and can easily be pushed back to a different repository. History, comment, everything will be preserved.

So, just in the unlikely case that one day there will be a need to have the added complexity of dealing with multiple repositories, the whole thing can be undone and LibLaserCut with its history can be moved back to where it came from.

Thoughts?

[t-oster]

Hi, fine with me. Can you give me concrete instructions or even a pull request?

[phdoerfler]

I wanna file a PR and I think I'm gonna have time for that on the 26th.

[t-oster]

and we should update the readme of the LibLaserCut repo afterwards that development has moved to the VisiCut repo

[phdoerfler]

Indeed, very good point!

[mgmax]

I'm not sure and think we should discuss this a little further before breaking stuff. Looking at your arguments:

• single source of truth: if you see LibLaserCut as "yet another library", then it's not different from other dependencies. If you see it as tightly coupled, I get the point that a monorepo saves you the burden of dealing with git submodules. One such tight coupling comes implicitly from the way we do serialization: The internal class structure (private variables) of LaserDevice sub-classes is written to the VisiCut XML config files. However, this could be solved by a custom serialization mechanism.
• share and reuse code easily: The dependency between LibLaserCut and VisiCut is intentionally a one-way street, keeping LibLaserCut simple and independent of Java-GUI-libraries, so that it would also work on Android. Separate repositories clearly enforce this boundary. Currently, it's easy to include LibLaserCut as a submodule in any other project, e.g., if we one day want to use as back-end for a web lasercutting app. This would be a lot worse if the whole VisiCut GUI stuff is also included, though one could argue that nobody has ever used LibLaserCut outside VIsiCut (?).
• easier to manage dependencies: I don't get the point. Currently, the split repository enforces that LibLaserCut builds with way less dependencies than VisiCut.
• atomic changes that affect both VisiCut and LibLaserCut: Yes, here a monorepo is definitely better. However, these changes are quite rare, often small, and have worked okay in the past because LibLaserCut is completely independent of VisiCut. You can just change things in LibLaserCut, get that published, and then fix VisiCut to work with the new LibLaserCut API. The only inconvenience is that the pull requests must be done in the right order.

Some things that may not work properly, depending on how the transition to monorepo is implemented:

• git blame
• references to old commit IDs from GitHub issues / commit messages / code comments
• references to GitHub issue numbers #123
• including LibLaserCut as a "subfolder" (submodule) in other projects

This should also be checked for the backwards migration: Will git blame, commit IDs etc. still work?

Overall, I'm not yet convinced that the solution will bring more benefit than harm.

[t-oster]

As @phdoerfler said they would stay two separate maven artifacts with their dependencies and just be managed in the same git repo.
We would keep all the separation but get rid of the submodule stuff and doube-pull-requests, which really confuses many developers.

You could still clone the repo and just build liblasercut which you can use as maven dependency in other projects.
But visicut doesn't make sense without liblasercut and since it's not in maven central it needs to be built on the developer's machine so IMHO it makes totally sense to combine the repos if we can keep history.

[phdoerfler]

there is indeed one point, "references to GitHub issue numbers #123" which I want to address: Issues can be transferred from one repository to another:
https://docs.github.com/en/github/managing-your-work-on-github/transferring-an-issue-to-another-repository

Unfortunately the document does not mention how issue numbers are translated but I wouldn't be surprised if Github was smart about this. After all, they do know what exact issue which changeset used to refer to, so it wouldn't be impossible to remap that and I have seen them being surprisingly smart when it comes to issue linking in the past in similar cases though I can't name an example currently. What I want to say: It might actually just work. We will see.

[t-oster]

Note to myself: read https://gist.github.com/b7988c2ab919cb01a9954816a5dec25b

[mgmax]

It would be good to test this somewhere else first. If the issue labels are not correctly mapped, then the PR will create a hundred "this issue was mentioned in #..." comments. What will definitely break is the issue mapping outside of Github because issue IDs are not unique. So if a commit says "Change foo to fix #...", then a normal Git GUI or command-line user will not know if this refers to LibLaserCut or VisiCut. New developers will not know about the old separate LibLaserCut issue tracker.

[t-oster]

That's a valid point. Technically one could create a s script, prefixing all "#" commit messages with the repository slug. But I don't know if that's worth it or if that's more a theoretical issue.

[mgmax]

Changing the commit messages would change the commit IDs, which is also not so great.

[t-oster]

yes, I would be talking about completely rewriting history. Of cause all now existing forks have to be rebased, however there can't be any conflict and any future fork would have the correct commit messages. As for commit references in the code itself, I would let a commit directly follow the "re-include submodule" commit with all commit references in the code corrected.

[phdoerfler]

I'm still on this, time has been rather tight lately, but after frustrating research (seriously how is everything so much more involved in git than in mercurial) I have found that git filter-branch is worth a closer look for moving files into different directories. This being said, I'm inclined to just simply pull the changes in a branch with parent -1, move them into a subdirectory and then merge with master.

[phdoerfler]

I am indeed still on this. I've written a small ruby script to automate this to allow me to experiment without having to remember to run all the commands in the right order, because I'm no computer but my computer is and it's so much better at this than I am.
And it's nearly done. In fact, it correctly merges the master of LibLaserCut into a subdirectory of VisiCut (it's just a series of git mv). It does so preserving all commit hashes and everything. I decided against git filter-branch for simplicity's sake. This will append a big heap of commits at the end of the VisiCut history, though. However, those will soon be forgotten after a few commits on VisiCut itself, I assume.
If we'd rather have the LibLaserCut commits appear in chronological order, interweaved with the VisiCut commits, then I believe git filter-branch is necessary. So if you rather want that, speak up please.

Anyways: The main thing that's missing is to do that for the other branches, too. Not just for master. After that I'll put up a fork so y'all can take a look at it.

[mgmax]

Sounds good, with the downside that it will break all issue references from LibLaserCut: if the commit message or code comment says "fix #600", then this will link to the wrong repository and also create "spam" comments in unrelated issues ("This issue was mentioned by user Xyz in commit abcd"). Probably, even creating your fork will create such spam comments. Use case: - Something breaks in LibLaserCut - git blame points to a commit from two years ago whose message says "fixes #600" - the link points to a completely unrelated issue of VisiCut instead, creating confusion

[phdoerfler]

I don’t expect a fork to trigger messages, that would be really dumb. And yeah I do see the issue of broken links. I’d of course prefer if they stayed intact, too, but even if they didn’t (they don’t have to break. Github could be smart about this, maybe they are, I don’t know. Yet.) that’s a worthy sacrifice in my opinion.