[Observability] add reporting mechanism when mirroring community pipelines. #8676
Conversation
| def main(args): | ||
| action_url = f"https://github.com/{GITHUB_REPO}/actions/runs/{GITHUB_RUN_ID}" | ||
| if args.status == "success": | ||
| hub_path = "https://huggingface.co/datasets/diffusers/community-pipelines-mirror/tree/main" |
There was a problem hiding this comment.
is there a way to report the actual path it mirrored to?
for example the workflow https://github.com/huggingface/diffusers/actions/runs/9489094498/job/26149588467#step:8:19 is "successful" but wrong: you can see that
CHECKOUT_REF: refs/tags/v0.29.0
PATH_IN_REPO:
it pushed to [https://huggingface.co/datasets/diffusers/community-pipelines-mirror/tree/main/.]
compared with this one that's actually run correctly, the path is https://huggingface.co/datasets/diffusers/community-pipelines-mirror/tree/main/v0.29.0 https://github.com/huggingface/diffusers/actions/runs/9495944132/job/26169275902#step:8:16
There was a problem hiding this comment.
The PATH_IN_REPO logic is here:
So, I took PATH_IN_REPO which is the simplest one to operate with to craft the hub_path.
Wauplin
left a comment
Wauplin
left a comment
There was a problem hiding this comment.
Thanks for adding this @sayakpaul!
| - name: Report success status | ||
| if: ${{ success() }} | ||
| run: | | ||
| pip install requests && python utils/notify_community_pipelines_mirror.py --status=success |
There was a problem hiding this comment.
Haven't tried it myself but requests should already be installed since we've installed huggingface_hub + uploaded stuff to the Hub using huggingface_hub (as it's a success).
I'm fine with reinstalling it though. I understand the point of having it for failures.
| pip install requests && python utils/notify_community_pipelines_mirror.py --status=success | |
| python utils/notify_community_pipelines_mirror.py --status=success |
…lines. (#8676) * add reporting mechanism when mirroring community pipelines. * remove unneeded argument * get the actual PATH_IN_REPO * don't need tag
What does this PR do?
Adds a reporting mechanism for the community pipeline mirroring workflow. This is important to prevent any failure cases of the mirror for whatever reasons. We rely on this Hugging Face Hub repository to pull in the community pipelines since https://github.com/huggingface/diffusers//issues/8417.
Example message: https://huggingface.slack.com/archives/C06LR0CSR6H/p1719214396872319.