Roll out CLABot to Kubernetes

[caniszczyk]

This issue will track progress of rolling out the CLA Bot fully to Kubernetes (see this issue for previous discussion: cncf/demo#7) and this Google doc (https://docs.google.com/document/d/1LIwyn5PDSYu8kh-7hH8688lt1cvwATkZmhec13nhp1c/edit#).

This blocks issue: kubernetes/kubernetes#27796

[caniszczyk]

Hey @emsearcy, can you add https://github.com/kubernetes/pr-bot as another repo to the CNCF CLABot so we can move the testing into the Kubernetes organization.

[dankohn]

Cc @bgrant0607 @foxish @monadic

[foxish]

Thanks @dankohn. Chris and I have communicated via email and we've granted access to the pr-bot repository in the kubernetes org.

[foxish]

Looks like our first step has been successful. Thanks @caniszczyk.
Before we start on implementing the label behavior in our bot, I'd like to discuss some aspects of your system to understand how they might work after we switch to the CNCF bot.

1. When no CLA is found, we apply the cla: no label and wait for the person to sign and comment back with I signed it!.
   • Can we expect the cla/linuxfoundation status on the PR to change in response to a new comment? (see Document how to create more than one node in the cluster. kubernetes/website#627 (comment))
   • What are the things that would trigger a re-evaluation of the CLA status? Does it happen on every update to the PR?
2. Is it possible to figure out from the status if there are multiple commits in a PR from different authors?

• In case of this, we typically want to post a different comment to the PR, based on whether all authors have signed the CLA (see Configure webhook kubernetes/kubernetes#30923 (comment)), or not.

[emsearcy]

(sorry about the extra notifications)

An update for @dankohn from a meeting I had with @foxish.

• webhooks are on target to be ready by the end of the week.
• commenting on a PR to update the status should not be needed, as our system will re-check open PRs immediately upon users signing CLAs. That said, any update to a PR will cause the webhook to fire too
• Currently, it is not possible from within the status API message we post to get granular information about multiple different failures (authors without CLAs or commits without authorship), however the status of each commit in the PR could be used to determine this (github doesn't have a special failure for the PR itself, it just shows the status of the last commit in the series for the PR).

[emsearcy]

(meant to send this yesterday but looks like I never hit "Comment")

Example pull request with multiple authors (pass and fail) including a commit lacking a GitHub author:
https://github.com/linuxfoundation-cla/webhook-test/pull/1

(Github only uses the status of the last commit in the series, so we propagate failures through to children commits, where the message for a given commit is the status of that commit, plus a count of failed authors from earlier commits in the patch series.)

Is this workable for the mungebot you are running to set labels and post messages?

[emsearcy]

OK, the pull request web service is live now. We have monitoring set up for it—if you want to monitor it yourself you can hit a simple ping responder which is separate from the github API endpoint but in the same Go service:
https://identity.linuxfoundation.org/lfcla/ping (should return 200, contents "OK")

cncf/demo and kubernetes/pr-bot have had the polling Perl script shutdown, and have been configured with a web hook so that the new LF CLA check will immediately post statuses.

I have not yet set it up to update the statuses when the user signs the CLA, so "I signed it" is, for the moment, still needed.

[foxish]

@emsearcy Sounds good! The example PR you showed looks good as well. The mungebot can handle the labels and comments. We would just need the status to be set, which appears to be happening. I'll set up the bot to do the CLA labeling and commenting later today on pr-bot. Thanks.

cc @bgrant0607 @dankohn

[foxish]

@emsearcy I just tried a test deployment on pr-bot. Please see https://github.com/kubernetes/pr-bot/pull/3#issuecomment-243205552
I signed the CLA, and commented on the PR (I signed it!) but I don't see any change in the status cla/linuxfoundation. The CLA page tells me I'm authorized to contribute. Shouldn't the webhook be firing immediately and updating that status?

cc @bgrant0607

[foxish]

Update: In an email discussion, we found that there was an issue with the way the CLA check is implemented currently on the CNCF side. We will continue with deployment once the issues are addressed.

[foxish]

@emsearcy @caniszczyk Any updates so far on fixing the issue we found in the CNCF contributor validation?

cc @dankohn

[dankohn]

@emsearcy @caniszczyk Could you please provide an update on the status here?

[caniszczyk]

I believe @emsearcy will followup soon, he was out on vacation for the last week.

Thanks for your patience.

[emsearcy]

Thank you for your patience. I've been communicating with our developer team on this and they have completed implementing and testing a hotfix for the issue. They'll deploy it 6pm EDT today.

[foxish]

@emsearcy Is the fix live at this point? Can we proceed with migrating the CLA system?

[emsearcy]

Yes, the fix for email validation behavior on GitHub-based account registration is live.

[dankohn]

@emsearcy Could you please provide an overview on the status. What are the open issues preventing a rollout to K8s or other projects? Are there nice-to-haves we want to add in the future?

[foxish]

The rollout is now complete and the kubernetes and kubernetes-incubator organizations are using the CNCF CLA. Thanks @caniszczyk, @emsearcy and @dankohn for all the help.

[bgrant0607]

Thanks all!

[mbohlool]

Can we add this check to kubernetes-client org too?