Bring Your Own Repo Discussion

[eliotwrobson]

A thread for discussing technical details of the BYO repo plan @nwalters512 .

[nwalters512]

Here's roughly what I have in mind:

• There would be a PrairieLearn GitHub app. Users could authenticate with GitHub and then install the app on either a personal account or an organization as a whole.
• Once the app is installed, users could select a repo for their course from a course admin page.
  • We'd need to figure out what the process of cutting over from one repo to another is. Do we rely on folks syncing all files between them out-of-band? Should we offer to force-push any existing contents to the new repo?
• We'd need to update our syncing code that does git operations (clone/pull/push) to use credentials that we'd presumably get from GitHub (I've never built a GitHub integration before, so I'm not 100% sure how that works).
• Wherever relevant, we should design this with other source control platforms in mind. While we should only support GitHub in the first iteration of this, we shouldn't make any choices that would make it difficult for us to expand to build integrations for GitLab/Bitbucket/etc. in the future.

@mwest1066 when we had discussed BYOR in the past, we threw around the idea of this being an enterprise-only feature (the assumption being that this feature would appeal most to either really large institutions, or to us as the operators of large multi-tenant installations). What are your current thoughts about that?

[ffund]

I generally prefer something that is at the repo level and not account/organization level. (I understand that you can specify which repos an app has permission for, but I still don't like that it's installed on account/organization.)

[nwalters512]

Ah!! I completely forgot about deploy keys (and even then I forgot that you can configure them with write permissions). That might be ideal. We could even have PL generate the keypair and give the user the public key to add to their source control service. Excellent idea @echuber2.

[eliotwrobson]

Looking around, deploy keys seem like a really flexible solution and very feasible to implement given the current infrastructure of PL. Not to get too far ahead of ourselves, but if that's the route we want to go, what kind of next steps should we take? Draw up design docs of some type?

[nwalters512]

That seems reasonable, yeah! We'd typically use an RFC for discussing new things like this. Don't feel like you have to copy the structure of the existing ones if you don't want to; just getting stuff written down into a form that we can discuss and iterate on (PR comments are great!) is the important bit.

[eliotwrobson]

Sounds good! We'll get started as soon as we get everyone onboarded on our dev team.

[nwalters512]

This discussion feels like it's related to the discussion of using GitHub submodules that @jesus-villalobos brought up on Slack. That would be tricky with PL's current model of "machine user that can access every single repo" since course A could declare course B as a submodule and get access to its contents. But if we were working with user-based credentials, we'd get access to everything the user has access to, making submodules much easier to implement. Notably, this would not be super straightforward with deploy tokens, unless you collected multiple deploy tokens (one for the main repo, one per submodule).

[echuber2]

Related discussion: #5907

There are potential issues with submodules but also several alternatives to them, so I'll just leave some resources here:

• Why submodules are bad (probably one of many similar articles)
  https://codingkilledthecat.wordpress.com/2012/04/28/why-your-company-shouldnt-use-git-submodules/

Alternative: git subtree

• git subtree tool (originally a community contribution)
  https://manpages.debian.org/testing/git-man/git-subtree.1.en.html
• subtree merge strategy for git (official and fully manual)
  https://git-scm.com/book/en/v2/Git-Tools-Advanced-Merging#_subtree_merge

More info about subtree (tool or strategy)

• https://stackoverflow.com/questions/32407634/when-to-use-git-subtree
• https://www.atlassian.com/git/tutorials/git-subtree
• https://medium.com/@porteneuve/mastering-git-subtrees-943d29a798ec

Other alternatives

I'm not sure of the relative strengths of these tools. Some of these predate the subtree option.

• Google Repo tool (from Android tooling)
  https://source.android.com/docs/setup/develop/repo
• braid
  https://github.com/cristibalan/braid
• git-subrepo
  https://github.com/ingydotnet/git-subrepo