New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Next 1.8.1 sync of pubished staging/ repos #53808

Closed
sttts opened this Issue Oct 12, 2017 · 13 comments

Comments

Projects
None yet
3 participants
@sttts
Contributor

sttts commented Oct 12, 2017

Now that https://github.com/kubernetes/kubernetes/releases/tag/v1.8.1 exists, I propose to do the next push of the staging repos.

As for the previous pushes:

We restrict the push to master and the 1.8 branches because #53583 blocks 1.7 right now (and probably similarly for 1.6, 1.5 if we want to keep them up-to-date at all).

Note that this time the push command is git push --follow-tags, which pushes the kubernetes-* tags. Also note that this does not include semver tags for client-go like v5.0.0. We will create this manually once the push is done.

We made good progress with repairing the publishing bot for nightly pushes kubernetes/test-infra#4134, but it isn't merged yet into test-infra and redeployed. Though the good news is that it is running for some weeks now with nightly pushes to the private forks.

@eparis

This comment has been minimized.

Show comment
Hide comment
@eparis
Member

eparis commented Oct 12, 2017

@sttts

This comment has been minimized.

Show comment
Hide comment
@sttts

sttts Oct 12, 2017

Contributor

@eparis I updated the push script to use --follow-tags instead of --tags. The later would push also the non-1.8 tags.

Contributor

sttts commented Oct 12, 2017

@eparis I updated the push script to use --follow-tags instead of --tags. The later would push also the non-1.8 tags.

@sttts

This comment has been minimized.

Show comment
Hide comment
@sttts

sttts Oct 12, 2017

Contributor

Double checked with a test-repo that --follow-tags is what we want: https://github.com/sttts/client-go-test/releases.

Contributor

sttts commented Oct 12, 2017

Double checked with a test-repo that --follow-tags is what we want: https://github.com/sttts/client-go-test/releases.

@caesarxuchao

This comment has been minimized.

Show comment
Hide comment
@caesarxuchao

caesarxuchao Oct 12, 2017

Member

Having second thought on whether we should maintain two lines of tags in client-go at: the kubernetes-v1.x.y and the semver v5.u.v. It is confusing for users.

Instead of that, we can let the robot output the mapping of tags into a file, and a human can manually create the semver (v5.u.v) tags on client-go. And update the table in README to reflect the latest mapping. cc @cheftako

Member

caesarxuchao commented Oct 12, 2017

Having second thought on whether we should maintain two lines of tags in client-go at: the kubernetes-v1.x.y and the semver v5.u.v. It is confusing for users.

Instead of that, we can let the robot output the mapping of tags into a file, and a human can manually create the semver (v5.u.v) tags on client-go. And update the table in README to reflect the latest mapping. cc @cheftako

@caesarxuchao

This comment has been minimized.

Show comment
Hide comment
@caesarxuchao

caesarxuchao Oct 13, 2017

Member

Using the test script provided by sttts, I verified that the tips of the those branches were working, though for the code-generator repo, the tips are ahead of the commits listed in the "push script" by one day.

Other than the client-go tag issues, this is good to go.

Member

caesarxuchao commented Oct 13, 2017

Using the test script provided by sttts, I verified that the tips of the those branches were working, though for the code-generator repo, the tips are ahead of the commits listed in the "push script" by one day.

Other than the client-go tag issues, this is good to go.

@sttts

This comment has been minimized.

Show comment
Hide comment
@sttts

sttts Oct 13, 2017

Contributor

It is confusing for users.

I see the kubernetes-* tags as quite the opposite: they document our semver tags. Moreover, we won't turn the former into releases. So the semver tags will show up very prominently in Github.

Contributor

sttts commented Oct 13, 2017

It is confusing for users.

I see the kubernetes-* tags as quite the opposite: they document our semver tags. Moreover, we won't turn the former into releases. So the semver tags will show up very prominently in Github.

@caesarxuchao

This comment has been minimized.

Show comment
Hide comment
@caesarxuchao

caesarxuchao Oct 13, 2017

Member

@eparis I think it's good to go!

Member

caesarxuchao commented Oct 13, 2017

@eparis I think it's good to go!

@eparis

This comment has been minimized.

Show comment
Hide comment
@eparis

eparis Oct 13, 2017

Member

https://gist.github.com/anonymous/34a32396daf6616d33e0cbbe5637c647
Failed.

$ ./script.sh 
+ git clone git@github.com:sttts/code-generator.git
Cloning into 'code-generator'...
remote: Counting objects: 2516, done.
remote: Compressing objects: 100% (185/185), done.
remote: Total 2516 (delta 102), reused 224 (delta 74), pack-reused 2222
Receiving objects: 100% (2516/2516), 2.97 MiB | 0 bytes/s, done.
Resolving deltas: 100% (1007/1007), done.
+ pushd code-generator
/tmp/code-generator /tmp
+ git fetch git@github.com:sttts/code-generator.git
From github.com:sttts/code-generator
 * branch            HEAD       -> FETCH_HEAD
+ git push --follow-tags git@github.com:kubernetes/code-generator.git aaa11be3549de11b8de2664a393fa12edb7feb99:refs/heads/master
Counting objects: 298, done.
Delta compression using up to 16 threads.
Compressing objects: 100% (160/160), done.
Writing objects: 100% (298/298), 78.60 KiB | 0 bytes/s, done.
Total 298 (delta 142), reused 252 (delta 100)
remote: Resolving deltas: 100% (142/142), completed with 42 local objects.
To github.com:kubernetes/code-generator.git
   7749115..aaa11be  aaa11be3549de11b8de2664a393fa12edb7feb99 -> master
 * [new tag]         kubernetes-1.8.0 -> kubernetes-1.8.0
 * [new tag]         kubernetes-1.8.0-beta.0 -> kubernetes-1.8.0-beta.0
 * [new tag]         kubernetes-1.8.0-beta.1 -> kubernetes-1.8.0-beta.1
 * [new tag]         kubernetes-1.8.0-rc.1 -> kubernetes-1.8.0-rc.1
 * [new tag]         kubernetes-1.8.1-beta.0 -> kubernetes-1.8.1-beta.0
 * [new tag]         kubernetes-1.9.0-alpha.0 -> kubernetes-1.9.0-alpha.0
 * [new tag]         kubernetes-1.9.0-alpha.1 -> kubernetes-1.9.0-alpha.1
+ popd
/tmp
+ pushd code-generator
/tmp/code-generator /tmp
+ git fetch git@github.com:sttts/code-generator.git
From github.com:sttts/code-generator
 * branch            HEAD       -> FETCH_HEAD
+ git push --follow-tags git@github.com:kubernetes/code-generator.git aae45cec4d74909e8ba6252be4ab4bacd7f2da67:refs/heads/release-1.8
Counting objects: 143, done.
Delta compression using up to 16 threads.
Compressing objects: 100% (68/68), done.
Writing objects: 100% (143/143), 32.08 KiB | 0 bytes/s, done.
Total 143 (delta 52), reused 136 (delta 49)
remote: Resolving deltas: 100% (52/52), completed with 2 local objects.
To github.com:kubernetes/code-generator.git
   42f0582..aae45ce  aae45cec4d74909e8ba6252be4ab4bacd7f2da67 -> release-1.8
 * [new tag]         kubernetes-1.8.1 -> kubernetes-1.8.1
 * [new tag]         kubernetes-1.8.2-beta.0 -> kubernetes-1.8.2-beta.0
+ popd
/tmp
+ git clone git@github.com:sttts/sample-apiserver.git
Cloning into 'sample-apiserver'...
remote: Counting objects: 12446, done.
remote: Compressing objects: 100% (369/369), done.
remote: Total 12446 (delta 228), reused 362 (delta 157), pack-reused 11883
Receiving objects: 100% (12446/12446), 9.44 MiB | 0 bytes/s, done.
Resolving deltas: 100% (7544/7544), done.
+ pushd sample-apiserver
/tmp/sample-apiserver /tmp
+ git fetch git@github.com:sttts/sample-apiserver.git
From github.com:sttts/sample-apiserver
 * branch            HEAD       -> FETCH_HEAD
+ git push --follow-tags git@github.com:kubernetes/sample-apiserver.git 338f1e3529a24a18a959035c4e876a46d54c1a60:refs/heads/master
Counting objects: 717, done.
Delta compression using up to 16 threads.
Compressing objects: 100% (341/341), done.
Writing objects: 100% (717/717), 394.44 KiB | 0 bytes/s, done.
Total 717 (delta 473), reused 577 (delta 337)
remote: Resolving deltas: 100% (473/473), completed with 181 local objects.
To github.com:kubernetes/sample-apiserver.git
 * [new tag]         kubernetes-1.6.10 -> kubernetes-1.6.10
 * [new tag]         kubernetes-1.6.11 -> kubernetes-1.6.11
 * [new tag]         kubernetes-1.6.11-beta.0 -> kubernetes-1.6.11-beta.0
 * [new tag]         kubernetes-1.6.12-beta.0 -> kubernetes-1.6.12-beta.0
 * [new tag]         kubernetes-1.7.6 -> kubernetes-1.7.6
 * [new tag]         kubernetes-1.7.7 -> kubernetes-1.7.7
 * [new tag]         kubernetes-1.7.7-beta.0 -> kubernetes-1.7.7-beta.0
 * [new tag]         kubernetes-1.7.8 -> kubernetes-1.7.8
 * [new tag]         kubernetes-1.7.8-beta.0 -> kubernetes-1.7.8-beta.0
 * [new tag]         kubernetes-1.7.9-beta.0 -> kubernetes-1.7.9-beta.0
 * [new tag]         kubernetes-1.8.0 -> kubernetes-1.8.0
 * [new tag]         kubernetes-1.8.0-beta.0 -> kubernetes-1.8.0-beta.0
 * [new tag]         kubernetes-1.8.0-beta.1 -> kubernetes-1.8.0-beta.1
 * [new tag]         kubernetes-1.8.0-rc.1 -> kubernetes-1.8.0-rc.1
 * [new tag]         kubernetes-1.8.1-beta.0 -> kubernetes-1.8.1-beta.0
 * [new tag]         kubernetes-1.9.0-alpha.0 -> kubernetes-1.9.0-alpha.0
 * [new tag]         kubernetes-1.9.0-alpha.1 -> kubernetes-1.9.0-alpha.1
 ! [rejected]        338f1e3529a24a18a959035c4e876a46d54c1a60 -> master (fetch first)
error: failed to push some refs to 'git@github.com:kubernetes/sample-apiserver.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
Member

eparis commented Oct 13, 2017

https://gist.github.com/anonymous/34a32396daf6616d33e0cbbe5637c647
Failed.

$ ./script.sh 
+ git clone git@github.com:sttts/code-generator.git
Cloning into 'code-generator'...
remote: Counting objects: 2516, done.
remote: Compressing objects: 100% (185/185), done.
remote: Total 2516 (delta 102), reused 224 (delta 74), pack-reused 2222
Receiving objects: 100% (2516/2516), 2.97 MiB | 0 bytes/s, done.
Resolving deltas: 100% (1007/1007), done.
+ pushd code-generator
/tmp/code-generator /tmp
+ git fetch git@github.com:sttts/code-generator.git
From github.com:sttts/code-generator
 * branch            HEAD       -> FETCH_HEAD
+ git push --follow-tags git@github.com:kubernetes/code-generator.git aaa11be3549de11b8de2664a393fa12edb7feb99:refs/heads/master
Counting objects: 298, done.
Delta compression using up to 16 threads.
Compressing objects: 100% (160/160), done.
Writing objects: 100% (298/298), 78.60 KiB | 0 bytes/s, done.
Total 298 (delta 142), reused 252 (delta 100)
remote: Resolving deltas: 100% (142/142), completed with 42 local objects.
To github.com:kubernetes/code-generator.git
   7749115..aaa11be  aaa11be3549de11b8de2664a393fa12edb7feb99 -> master
 * [new tag]         kubernetes-1.8.0 -> kubernetes-1.8.0
 * [new tag]         kubernetes-1.8.0-beta.0 -> kubernetes-1.8.0-beta.0
 * [new tag]         kubernetes-1.8.0-beta.1 -> kubernetes-1.8.0-beta.1
 * [new tag]         kubernetes-1.8.0-rc.1 -> kubernetes-1.8.0-rc.1
 * [new tag]         kubernetes-1.8.1-beta.0 -> kubernetes-1.8.1-beta.0
 * [new tag]         kubernetes-1.9.0-alpha.0 -> kubernetes-1.9.0-alpha.0
 * [new tag]         kubernetes-1.9.0-alpha.1 -> kubernetes-1.9.0-alpha.1
+ popd
/tmp
+ pushd code-generator
/tmp/code-generator /tmp
+ git fetch git@github.com:sttts/code-generator.git
From github.com:sttts/code-generator
 * branch            HEAD       -> FETCH_HEAD
+ git push --follow-tags git@github.com:kubernetes/code-generator.git aae45cec4d74909e8ba6252be4ab4bacd7f2da67:refs/heads/release-1.8
Counting objects: 143, done.
Delta compression using up to 16 threads.
Compressing objects: 100% (68/68), done.
Writing objects: 100% (143/143), 32.08 KiB | 0 bytes/s, done.
Total 143 (delta 52), reused 136 (delta 49)
remote: Resolving deltas: 100% (52/52), completed with 2 local objects.
To github.com:kubernetes/code-generator.git
   42f0582..aae45ce  aae45cec4d74909e8ba6252be4ab4bacd7f2da67 -> release-1.8
 * [new tag]         kubernetes-1.8.1 -> kubernetes-1.8.1
 * [new tag]         kubernetes-1.8.2-beta.0 -> kubernetes-1.8.2-beta.0
+ popd
/tmp
+ git clone git@github.com:sttts/sample-apiserver.git
Cloning into 'sample-apiserver'...
remote: Counting objects: 12446, done.
remote: Compressing objects: 100% (369/369), done.
remote: Total 12446 (delta 228), reused 362 (delta 157), pack-reused 11883
Receiving objects: 100% (12446/12446), 9.44 MiB | 0 bytes/s, done.
Resolving deltas: 100% (7544/7544), done.
+ pushd sample-apiserver
/tmp/sample-apiserver /tmp
+ git fetch git@github.com:sttts/sample-apiserver.git
From github.com:sttts/sample-apiserver
 * branch            HEAD       -> FETCH_HEAD
+ git push --follow-tags git@github.com:kubernetes/sample-apiserver.git 338f1e3529a24a18a959035c4e876a46d54c1a60:refs/heads/master
Counting objects: 717, done.
Delta compression using up to 16 threads.
Compressing objects: 100% (341/341), done.
Writing objects: 100% (717/717), 394.44 KiB | 0 bytes/s, done.
Total 717 (delta 473), reused 577 (delta 337)
remote: Resolving deltas: 100% (473/473), completed with 181 local objects.
To github.com:kubernetes/sample-apiserver.git
 * [new tag]         kubernetes-1.6.10 -> kubernetes-1.6.10
 * [new tag]         kubernetes-1.6.11 -> kubernetes-1.6.11
 * [new tag]         kubernetes-1.6.11-beta.0 -> kubernetes-1.6.11-beta.0
 * [new tag]         kubernetes-1.6.12-beta.0 -> kubernetes-1.6.12-beta.0
 * [new tag]         kubernetes-1.7.6 -> kubernetes-1.7.6
 * [new tag]         kubernetes-1.7.7 -> kubernetes-1.7.7
 * [new tag]         kubernetes-1.7.7-beta.0 -> kubernetes-1.7.7-beta.0
 * [new tag]         kubernetes-1.7.8 -> kubernetes-1.7.8
 * [new tag]         kubernetes-1.7.8-beta.0 -> kubernetes-1.7.8-beta.0
 * [new tag]         kubernetes-1.7.9-beta.0 -> kubernetes-1.7.9-beta.0
 * [new tag]         kubernetes-1.8.0 -> kubernetes-1.8.0
 * [new tag]         kubernetes-1.8.0-beta.0 -> kubernetes-1.8.0-beta.0
 * [new tag]         kubernetes-1.8.0-beta.1 -> kubernetes-1.8.0-beta.1
 * [new tag]         kubernetes-1.8.0-rc.1 -> kubernetes-1.8.0-rc.1
 * [new tag]         kubernetes-1.8.1-beta.0 -> kubernetes-1.8.1-beta.0
 * [new tag]         kubernetes-1.9.0-alpha.0 -> kubernetes-1.9.0-alpha.0
 * [new tag]         kubernetes-1.9.0-alpha.1 -> kubernetes-1.9.0-alpha.1
 ! [rejected]        338f1e3529a24a18a959035c4e876a46d54c1a60 -> master (fetch first)
error: failed to push some refs to 'git@github.com:kubernetes/sample-apiserver.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
@sttts

This comment has been minimized.

Show comment
Hide comment
@sttts

sttts Oct 13, 2017

Contributor

We have PRs manually merged into the published repos after I did my fork (sample-apiserver, kube-aggregator, apimachinery, metrics). Will regenerate the master branch and post the update when it's ready.

In general, I wonder whether we should move all files back into the staging/ directories to avoid those situations. Of course, this increases the latency to get changes in. It's easier to click on merge in the Github UI than waiting for the bot. But the current situation always has this danger, though admittedly much less if the bot is running nightly based directly on the kubernetes repos (instead of another fork).

Contributor

sttts commented Oct 13, 2017

We have PRs manually merged into the published repos after I did my fork (sample-apiserver, kube-aggregator, apimachinery, metrics). Will regenerate the master branch and post the update when it's ready.

In general, I wonder whether we should move all files back into the staging/ directories to avoid those situations. Of course, this increases the latency to get changes in. It's easier to click on merge in the Github UI than waiting for the bot. But the current situation always has this danger, though admittedly much less if the bot is running nightly based directly on the kubernetes repos (instead of another fork).

@caesarxuchao

This comment has been minimized.

Show comment
Hide comment
@caesarxuchao

caesarxuchao Oct 13, 2017

Member

admittedly much less if the bot is running nightly based directly on the kubernetes repos (instead of another fork).

To be clear, if the robot is running, the problem only occurs if the manual merge happens at the same time as a sync.

Member

caesarxuchao commented Oct 13, 2017

admittedly much less if the bot is running nightly based directly on the kubernetes repos (instead of another fork).

To be clear, if the robot is running, the problem only occurs if the manual merge happens at the same time as a sync.

@sttts

This comment has been minimized.

Show comment
Hide comment
@sttts

sttts Oct 13, 2017

Contributor

To be clear, if the robot is running, the problem only occurs if the manual merge happens at the same time as a sync.

And it's not really a danger actually as it is self-healing the next day.

Contributor

sttts commented Oct 13, 2017

To be clear, if the robot is running, the problem only occurs if the manual merge happens at the same time as a sync.

And it's not really a danger actually as it is self-healing the next day.

@sttts

This comment has been minimized.

Show comment
Hide comment
@sttts

sttts Oct 16, 2017

Contributor

@eparis @caesarxuchao I have updated the branches on my private fork and the gist link in the PR description.

@caesarxuchao I have updated kubernetes/test-infra#4134 with a fix for a Godeps.json bug, included in the newly generated branched already.

Contributor

sttts commented Oct 16, 2017

@eparis @caesarxuchao I have updated the branches on my private fork and the gist link in the PR description.

@caesarxuchao I have updated kubernetes/test-infra#4134 with a fix for a Godeps.json bug, included in the newly generated branched already.

@sttts

This comment has been minimized.

Show comment
Hide comment
@sttts

sttts Oct 16, 2017

Contributor

Pushed. Thanks @eparis! 🚀 🎉

Contributor

sttts commented Oct 16, 2017

Pushed. Thanks @eparis! 🚀 🎉

@sttts sttts closed this Oct 16, 2017

@sttts sttts referenced this issue Oct 16, 2017

Closed

Tag client-go 5.0.x #54003

0 of 2 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment