RFC: Bringing back the monorepo for rust services and main app #207
Conversation
neilkakkar
changed the title
There was a problem hiding this comment.
Only want to say big 👍 there are always downsides to a monorepo but in our current upsides the benefits far out way the cons (and we can always reverse it later).
|
|
||
| Effectively, nothing should change for anyone developing on `hog-rs` currently, and people developing on main `posthog` repo should get more options to hook things up. | ||
|
|
||
| 1. [ ] Introduce a `rust/` top-level folder that has all rust services |
There was a problem hiding this comment.
If we go with a rust top level it should make the whole "only run the CI for this code" quite straightforward.
There was a problem hiding this comment.
Also one thought - you could do this in the hog-rs repo first to test out that it works as expected.
There was a problem hiding this comment.
For a True Monorepo™ it would also be great to put everything which is not rust//plugin-server//plugin-transpiler/ under web/ – since it doesn't make sense that the plugin server and such are under the Django server in the hierarchy. Architecturally, they're all on the same level. (This is scope creep)
|
|
||
| I'm not 100% sure yet how we want to migrate things over, or over what timeline. | ||
|
|
||
| Given hog-rs deploys happen manually right now, it seems de-risked enough to do it in one fell-swoop to minimise keeping-code-in-two-places-in-sync-pains. So I'd propose: |
There was a problem hiding this comment.
All sounds good, except this. TIL we have to deploy hog-rs manually. If this was embedded within posthog/posthog then I would have assumed that deployments were automatic with the rest of the codebase. With hog-rs being a separate repo, I would have intentionally sought out how it got deployed
There was a problem hiding this comment.
This is also likely a non-issue considering who edits hog-rs code at the moment - e.g. typically pipeline team who are already aware of this
There was a problem hiding this comment.
Correct - eventually this will be automated, but for now it's good enough for hog-rs editors 👍
| Given hog-rs deploys happen manually right now, it seems de-risked enough to do it in one fell-swoop to minimise keeping-code-in-two-places-in-sync-pains. So I'd propose: | ||
|
|
||
| 1. [ ] Once we get alignment on the design above, we create a branch where we copy over current code, and try implementing the above things. If we need to merge to test (possible with CI shiz), copy over everything but the docker image build workflow. | ||
| 2. [ ] Once the branch/folder is ready with the minimal required things, copy over all new code into the branch, merge, and stop CI in hog-rs. This does mean all existing PRs would need to move over as well. |
There was a problem hiding this comment.
Maybe you already planned to do this, but it would be nice to pull in the actual git repo so that we have accurate history, rather than just copying the code over and adding it.
I think the steps are roughly like:
- In
posthog,git remote addthehog-rsrepo - Checkout the
mainbranch fromhog-rs - Move everything into a
rustdir and commit that, so that the only thing in the directory isrust - Check out a
posthogbranch git merge the-hog-rs-branch --allow-unrelated-historiesshould in theory do a merge commit that just drops therustdirectory into place, but with all of our history
(I riffed these notes and could be wrong, but I did do this when I merged our 2 rust repos into one. I can help if there are issues with the list above.)
|
not all the things have been done, but monorepo is here now, so merging this, we can incrementally improve the rest of the things when we want/need to! |
Looking for alignment on how to do this, and help on doing some of these changes