[SPARK-33282] Migrate from deprecated probot autolabeler to GitHub labeler action #30244
Conversation
|
ok to test |
kbendick
changed the title
|
Kubernetes integration test starting |
|
Kubernetes integration test status success |
|
Kubernetes integration test starting |
|
Kubernetes integration test status success |
|
Kubernetes integration test starting |
|
Kubernetes integration test status success |
|
Kubernetes integration test starting |
|
Kubernetes integration test starting |
|
Kubernetes integration test status success |
…/streaming/**/* to allow it to match directories with that pattern that are not at the repo root
|
Kubernetes integration test status success |
kbendick
changed the title
|
Kubernetes integration test starting |
|
I created an issue in the |
… support for all & any matches
… any in the labeler workflow file
| - "repl/**/*" | ||
| - "bin/spark-shell*" | ||
| SQL: | ||
| #- any: ["**/sql/**/*", "!python/pyspark/sql/avro/**/*", "!python/pyspark/sql/streaming.py", "!python/pyspark/sql/tests/test_streaming.py"] |
There was a problem hiding this comment.
This is how the any should look when it's supported in a release upstream.
|
Kubernetes integration test status success |
|
Thanks for working on this @kbendick. Let me know when you think it's ready to merge. We can merge and see how it gose. |
|
Kubernetes integration test starting |
|
Kubernetes integration test status success |
|
Kubernetes integration test starting |
|
Kubernetes integration test starting |
|
Kubernetes integration test status success |
|
Kubernetes integration test status failure |
|
cc @nchammas FYI |
kbendick
changed the title
Thanks @HyukjinKwon! This is ready to be merged. It might need a small tweak or two given it was a pretty complicated config relative to the others that I've done so far but I would say it's tested and ready to merge so people can finally have their labels back. 👍 |
|
Let me merge and see how it goes. Thanks for this work and thanks for the followup in advance. BTW would you mind file a separate JIRA to follow up? |
|
Merged to master. |
|
I created an issue for updating the labeler with |
|
Just a reminder :-). It might be great to double check if |
What changes were proposed in this pull request?
This PR removes the old Probot Autolabeler labeling configuration, as the probot autolabeler has been deprecated. I've updated the configs in Iceberg and in Avro, and we also need to update here. This PR adds in an additional workflow for labeling PRs and migrates the old probot config to the new format. Unfortunately, because certain features have not been released upstream, we will not get the exact behavior as before. I have documented where that is and what changes are neeeded, and in the associated ticket I've also discussed other options and why I think this is the best way to go. Definitely a follow up ticket is needed to get the original behavior back in these few cases, but PRs have not been labeled for almost a month and so it's probably best to get it right 95% of the time and occasionally have some UI related PRs labeled as
COREwhile the issue is resolved upstream and/or further investigated.Why are the changes needed?
The probot autolabeler is dead and will not be maintained going forward. This has been confirmed with github user [at]mithro in an issue in their repository.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
To test this PR, I first merged the config into my local fork. I then edited it several times and ran tests on that.
Unfortunately, I've overwritten my fork with the apache repo in order to create a proper PR. However, I've also added the config for the same thing in the Iceberg repo as well as the Avro repo.
I have now merged this PR into my local repo and will be running some tests on edge cases there and for validating in general:
I've also discovered that we're likely not killing github actions that run (like large tests etc) when users push to their PR. In most cases, I see that a user has to mark something as "OK to test", but it still seems like we might want to discuss whether or not we should add a cancellation step In order to save time / capacity on the runners. If so desired, we would add an action in each workflow that cancels old runs when a
pushaction occurs on a PR. This will likely make waiting for test runners much faster iff tests are automatically rerun on push by anybody (such as PMCs, PRs that have been marked OK to test, etc). We could free a large number of resources potentially if a cancellation step was added to all of the workflows in the Apache account (as github action API limits are set at the account level).Admittedly, the fact that the "old" workflow runs weren't cancelled could admittedly be because of the fact that I was working in a fork, but given that there are explicit actions to be added to the start of workflows to cancel old PR workflows and given that we don't have them configured indicates to me that likely this is the case in this repo (and in most
apacherepos as well), at least under certain circumstances (e.g. repos that don't have "Ok to test"-like webhooks as one example).This is a separate issue though, which I can bring up on the mailing list once I'm done with this PR. Unfortunately I've been very busy the past two weeks, but if somebody else wanted to work on that I would be happy to support with any knowledge I have.
The last Apache repo to still have the probot autolabeler in it is Beam, at which point we can have Gavin from ASF Infra remove the permissions for the probot autolabeler entirely. See the associated JIRA ticket for the links to other tickets, like the one for ASF Infra to remove the dead probot autolabeler's read and write permissions to our PRs in the Apache organization.