HBASE-25033 : Provide an option to upload RC to home.apache.org#2438
HBASE-25033 : Provide an option to upload RC to home.apache.org#2438virajjasani wants to merge 4 commits intoapache:masterfrom
Conversation
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
| } | ||
|
|
||
| function is_svn_ci_disabled { | ||
| [[ "$DRY_RUN" = 1 ]] || [[ "$IGNORE_SVN_CI" = 1 ]] |
There was a problem hiding this comment.
(reworded) is this change for using -ignore-svn-ci with -f together? it does not seem like we can use it without dry-run
There was a problem hiding this comment.
is this change for using -ignore-svn-ci with -f together?
That's true, ignore-svn-ci is only useful when used in combination with -f, otherwise it is sort of no-op because dry-mode anyways don't perform svn ci.
| If none specified, runs tag, then publish-dist, and then publish-release. | ||
| 'publish-snapshot' is also an allowed, less used, option. | ||
| -x debug. do less clean up. (env file, gpg forwarding on mac) | ||
| -i [ignore-svn-ci] |
There was a problem hiding this comment.
Rather than ignoring svn, what if the script could accept a flag to upload to a path in the user's home directory on https://home.apache.org. The template could also be updated to point to this https url. This way, distribution of RC artifacts remains codified by the scripts, for committers with and without PMC rights.
There was a problem hiding this comment.
I tried considering it, the only issue is user will have to manually generate ssh private/public keys from docker container and copy public key over to home.apache.org 's ~/.ssh/authorized_keys every time they use this script in order to perform password-less scp operation from container :(
There was a problem hiding this comment.
Hmm that could be a bigger change, but not insurmountable: rather than copy keys, i think the ssh-agent could be used to forward credentials into the docker environment, similar to how it's done with gpg-agent.
There was a problem hiding this comment.
Actually, the same authentication mechanism (ssh keys) should ideally be used for interacting with svn as well. Maybe this require infra to update the svn repository somehow.
There was a problem hiding this comment.
i think the ssh-agent could be used to forward credentials into the docker environment, similar to how it's done with gpg-agent.
This definitely sounds reasonable, but if we just ignore svn ci, user might have other options available too (maybe upload to some apache resource via sftp or maybe git checkin or manually upload via ui to an apache resource?), not that I can think of much other than uploading to h.a.o/p.a.o that I am aware of, however something might come up in future and then password-less scp might become stale. I believe, having artifacts available locally for committers without PMC rights might be enough for them to decide on next step?
Even if we prefer automation around RC upload, we are not sure that all committers will have public_html under their respective home dir in home.apache.org, hence, I was considering option to take full path in argument (e.g do-release-docker.sh -u home.apache.org:/home/vjasani/public_html/2.3.2RC4/), which I believe should be acceptable. Moreover, h.a.o's authorized_keys should be updated with public key (just one time activity, not a big deal).
Overall, nothing wrong with ssh-agent based approach that we can explore but I am just thinking if it is worth having an optional additional docker container for long term.
@ndimiduk what would you prefer?
Actually, the same authentication mechanism (ssh keys) should ideally be used for interacting with svn as well. Maybe this require infra to update the svn repository somehow.
Let me explore svn auth, i don't have much context here. Thank you for bringing this up.
There was a problem hiding this comment.
Btw if we are good with uploading to h.a.o/p.a.o, perhaps we might not need ssh-agent either. We can mount ~/.ssh directly in docker image and that should help run password-less scp without issues.
There was a problem hiding this comment.
Sorry for too much back and forth here, was giving some thoughts on what seems better and flexible option for committers, and to answer my own question, I believe as long as we know of h.a.o upload as one of the available ways for committers to upload artifacts, we should utilize it (until better options are available).
Will try mounting local .ssh to container and using password-less scp to upload RC over h.a.o. Will try to document as much as possible (as part of -h).
virajjasani
changed the title
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
busbey
left a comment
busbey
left a comment
There was a problem hiding this comment.
Please forward access to the ssh agent to the container instead of mounting the whole .ssh dir.
I believe ssh agent socket forwarding even works on docker desktop for mac so you shouldn't need to use the gpg proxy container gymnastics.
5 participants
No description provided.