Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-40727][INFRA] Add merge_spark_docker_pr.py #1

Closed
wants to merge 3 commits into from

Conversation

Yikun
Copy link
Member

@Yikun Yikun commented Oct 10, 2022

What changes were proposed in this pull request?

This patch add the merge_spark_docker_pr.py to help to merge spark-docker commits and resolve spark JIRA issue.

The script is from https://github.com/apache/spark/blob/ef837ca71020950b841f9891c70dc4b29d968bf1/dev/merge_spark_pr.py

And change spark to spark-docker: e4107a7

Why are the changes needed?

Help to merge spark-docker commits.

Does this PR introduce any user-facing change?

No, dev only

How was this patch tested?

will merge it by using itself

@Yikun
Copy link
Member Author

Yikun commented Oct 10, 2022

cc @HyukjinKwon @zhengruifeng

if branch == "master":
return versions[0]
else:
branch_ver = branch.replace("branch-", "")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will we have a branch here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I think only maintain a master branch with all release dockerfiles is enough.

So, only branch == "master" is valid in here, I can remove the fix_version_from_branch I think

And we should also remove L571:

    pick_prompt = "Would you like to pick %s into another branch?" % merge_hash
    while input("\n%s (y/n): " % pick_prompt).lower() == "y":
        merged_refs = merged_refs + [cherry_pick(pr_num, merge_hash, latest_branch)]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this script is almost same as Apache Spark's repo, let's just keep them as are.
If the diff is already large, we could remove these codes out.

Copy link
Member Author

@Yikun Yikun Oct 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's almost same diff is here: e4107a7. So just let me remove L571 is enough (to remove redundant backport process):

    pick_prompt = "Would you like to pick %s into another branch?" % merge_hash
    while input("\n%s (y/n): " % pick_prompt).lower() == "y":
        merged_refs = merged_refs + [cherry_pick(pr_num, merge_hash, latest_branch)]

@Yikun
Copy link
Member Author

Yikun commented Oct 10, 2022

This patch has been merged by it self

$ ./merge_spark_docker_pr.py
git rev-parse --abbrev-ref HEAD
Which pull request would you like to merge? (e.g. 34): 1

=== Pull Request #1 ===
title	[SPARK-40727][INFRA] Add merge_spark_docker_pr.py
source	Yikun/merge_script
target	master
url	https://api.github.com/repos/apache/spark-docker/pulls/1

Proceed with merging pull request #1? (y/n): y
git fetch apache-github pull/1/head:PR_TOOL_MERGE_PR_1
From github.com:apache/spark-docker
 * [new ref]         refs/pull/1/head -> PR_TOOL_MERGE_PR_1
git fetch apache master:PR_TOOL_MERGE_PR_1_MASTER
From github.com:apache/spark-docker
 * [new branch]      master     -> PR_TOOL_MERGE_PR_1_MASTER
 * [new branch]      master     -> apache/master
git checkout PR_TOOL_MERGE_PR_1_MASTER
Switched to branch 'PR_TOOL_MERGE_PR_1_MASTER'
['git', 'merge', 'PR_TOOL_MERGE_PR_1', '--squash']
['git', 'log', 'HEAD..PR_TOOL_MERGE_PR_1', '--pretty=format:%an <%ae>', '--reverse']
Enter primary author in the format of "name <email>" [Yikun Jiang <yikunkero@gmail.com>]:
git config --get user.name
git config --get user.email
['git', 'commit', '--author="Yikun Jiang <yikunkero@gmail.com>"', '-m', '[SPARK-40727][INFRA] Add merge_spark_docker_pr.py', '-m', '### What changes were proposed in this pull request?\r\nThis patch add the merge_spark_docker_pr.py to help to merge `spark-docker` commits and resolve spark JIRA issue.\r\n\r\nThe script is from https://github.com/apache/spark/blob/ef837ca71020950b841f9891c70dc4b29d968bf1/dev/merge_spark_pr.py\r\n\r\nAnd change `spark` to `spark-docker`: https://github.com/apache/spark-docker/commit/e4107a74d348656041612ff68a647c6051894240\r\n\r\n### Why are the changes needed?\r\nHelp to merge spark-docker commits.\r\n\r\n\r\n### Does this PR introduce _any_ user-facing change?\r\nNo, dev only\r\n\r\n\r\n### How was this patch tested?\r\nwill merge it by using itself', '-m', 'Closes #1 from Yikun/merge_script.', '-m', 'Authored-by: Yikun Jiang <yikunkero@gmail.com>\nSigned-off-by: Yikun Jiang <yikunkero@gmail.com>']

Merge complete (local ref PR_TOOL_MERGE_PR_1_MASTER). Push to apache? (y/n): y
git push apache PR_TOOL_MERGE_PR_1_MASTER:master
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Delta compression using up to 8 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 7.63 KiB | 3.82 MiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
To github.com:apache/spark-docker.git
   c5b015a..fa2d1a5  PR_TOOL_MERGE_PR_1_MASTER -> master
git rev-parse PR_TOOL_MERGE_PR_1_MASTER
Restoring head pointer to merge_script
git checkout merge_script
Switched to branch 'merge_script'
git branch
Deleting local branch PR_TOOL_MERGE_PR_1
git branch -D PR_TOOL_MERGE_PR_1
Deleting local branch PR_TOOL_MERGE_PR_1_MASTER
git branch -D PR_TOOL_MERGE_PR_1_MASTER
Pull request #1 merged!
Merge hash: fa2d1a59

Would you like to update an associated JIRA? (y/n): y
Enter a JIRA id [SPARK-40727]:
=== JIRA SPARK-40727 ===
summary		Add merge_spark_docker_pr.py to help merge commit
assignee	Yikun Jiang
status		Reopened
url		https://issues.apache.org/jira/browse/SPARK-40727

Enter comma-separated fix version(s) [3.4.0]:
Successfully resolved SPARK-40727 with fixVersions=['3.4.0']!

JIRA https://issues.apache.org/jira/browse/SPARK-40727 was also closed as expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants