Description
What would you like to be added:
Some sort of generic git service in prow that normalizes incoming webhooks into a generic format that hook and other plugins can understand, as well as provides a generic interface for performing git-related actions such as commenting on PRs, etc.
My initial proposal is that this git service essentially be another plugin that implements a number of generic git-related APIs, and that the name/location of this service be configurable so that we can support out-of-tree providers. In addition, it's probably worthwhile to think about decomposing the notions of code review and issue tracking into separate provider APIs, since not all providers implement these functions, although presumably prow will only ever be useful in combination with a git provider that at least provides basic code review features. But it would be nice to support, e.g., code review tool X in combination with issue tracker Y. We currently have something like this with the loose coupling of the Gerrit adapter and reporter. It would be great if we could make this sort of thing generic and extensible.
We discussed this briefly on the sig-testing call of November 6, and my hope is that this issue will be an asynchronous continuation of that discussion. I am also working on a more formal proposal that I plan to present on the weekly sig-testing call some time in the near future.
Why is this needed:
Currently, prow is tightly coupled to GitHub. Making it more generic would give the various k8s projects more flexibility in how they craft their workflows, as well as making prow more accessible to projects in the broader open source community and beyond.
/kind feature
Activity
BenTheElder commentedon Nov 14, 2018
cc @cjwagner @krzyzacy
/area prow
yastij commentedon Nov 15, 2018
we might want to think about an interface that providers might want to implement.
wbrefvem commentedon Nov 20, 2018
@yastij Yeah, there seems to be some consensus around this.
wbrefvem commentedon Nov 20, 2018
Update
Here are some more concrete ideas on how to tackle this:
AssignIssue,CommentOnPR, etc..protofiles available for users who want to implement their own service.midnightconman commentedon Feb 9, 2019
This PR #11075 was just merged, it gets us closer to this goal.
fejta-bot commentedon May 10, 2019
Issues go stale after 90d of inactivity.
Mark the issue as fresh with
/remove-lifecycle stale.Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with
/close.Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale
yastij commentedon May 10, 2019
/remove-lifecycle stale
/lifecycle frozen
townsendb commentedon May 28, 2019
Hi @BenTheElder . Is this proposal on the radar? I'm using GitLab and trying to use the next gen serverless features of Jenkins-x but blocked by this issue in prow.
Many thanks.
McShauno commentedon Jun 12, 2019
Ok, fine ... I'll write it. Give me a couple of weeks.
¯_(ツ)_/¯
BenTheElder commentedon Jun 12, 2019
@townsendb Hi, Sorry I missed this comment in all my github notifications somehow... I don't really work on prow currently (kind mainly) though I'd love to see this happen. I'm not sure where this is at.
@cjwagner & @stevekuznetsov are good current points of contact for prow.
11 remaining items
abdennour commentedon Jan 10, 2021
we love prow. But, we are using bitbucket cloud.
Any change ?
cjwagner commentedon Jan 11, 2021
@abdennour No change, Alvaro's comment is still accurate. PRs are welcome if you'd like to contribute to this!
spiffxp commentedon Aug 9, 2021
/sig testing
vsoch commentedon Apr 24, 2023
I'm definitely interested in this use case (and new to prow but looking to try it out!) At the labs we use GitLab heavily, and it would be great to have this integration. The degree to which I could help will probably come down to if it's easy to set up a development environment I think.
vsoch commentedon Apr 24, 2023
Also the README says it was going to be deleted in February but... it still lives!! cue Frankenstein monster noises

BenTheElder commentedon Apr 24, 2023
There really need to be more concrete proposals, this thread is pretty vague.
prow has many aspects and most of them are inherently GitHub specific (eg everything related to webhook handling and label based chatops) which theoretically could be abstracted away, but at this point would take a lot of work and require a more detailed proposal
vsoch commentedon Apr 24, 2023
I'm not familiar with how proposals are done here - akin to a KEP?
I was reading here: https://docs.prow.k8s.io/docs/life-of-a-prow-job/
And at least from a high level, the abstractions like hooks, comments, are things that can be mapped to other code review places too. I definitely don't disagree about it being a lot of work, but (at face value) doesn't seem impossible?
@BenTheElder are these the best docs https://docs.prow.k8s.io/docs/build-test-update/ for a developer to get started trying things out?
BenTheElder commentedon Apr 24, 2023
Depends on the scope of the change, typically a lightweight google doc is enough to start discussion, which should be shared to the SIG in the mailinglist and slack.
It's not impossible, but it's a lot of work and as-is we're having difficulty getting much more scoped changes the Kubernetes project itself would benefit from reviewed / approved promptly, there's limited bandwidth from the current prow OWNERS.
I don't work on prow itself often anymore and when I do I make small fixes that don't require spinning up an instance.
vsoch commentedon Apr 24, 2023
Interesting - I would guess there is a long history behind this, but why does prow have to live under Kubernetes here proper? If it’s holding the project back and adding extra churn for developers that doesn’t make a lot of sense.
BenTheElder commentedon Apr 24, 2023
Prow exists because Kubernetes needed some CI tooling, which started as a webhook system for commands against jenkins and then a merge robot. It eventually turned to running CI workloads on Kubernetes itself.
Usage for other purposes beyond Kubernetes came much later. It still primarily exists for Kubernetes and supports workflows used by the project.
That's not what I meant. I meant that the existing owners are currently already struggling to keep up with incoming feature work so I'm warning that it may not be a great time to propose a very large change.
vsoch commentedon Apr 24, 2023
Gotcha, and thank you for the details! It could be then that a better avenue to pursue would be creating something like prow, but separate from it and with a focus on more general abstractions for components.
If you (or anyone in this thread) know if other tools in this space that are for creating your own CI/CD on Kubernetes please share! When you do a general Google search it typically shows articles about CI/CD with respect to continuous deployment to Kubernetes, and not making your own system (that could work with it).
kfox1111 commentedon Apr 24, 2023
There was a fork of Prow I think a while back that eventually became Lighthouse? https://github.com/jenkins-x/lighthouse/
I wanna say it was intentionally done to try and add more then github support while not burdening the k8s developers with features they didn't need.
Might be what your looking for?
BenTheElder commentedon May 30, 2024
I don't think we've had the bandwidth for this, but in any case prow is now split out to github.com/kubernetes-sigs/prow and future feature discussions should be held there.