Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Begin supporting etcd3 #20504

Closed
lavalamp opened this issue Feb 2, 2016 · 70 comments
Closed

Begin supporting etcd3 #20504

lavalamp opened this issue Feb 2, 2016 · 70 comments
Assignees
@timothysc @lavalamp @hongchaodeng @wojtek-t
Labels
area/etcd priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery.
Milestone
v1.6

Comments

@lavalamp
Copy link
Member

lavalamp commented Feb 2, 2016

There's probably already an issue about this, but none of my searches could find it.

I know @hongchaodeng (and @xiang90?) are interested in doing much of the work. @krousey may be able to help.

Off the top of my head, we'll need:

  • Flag enable etcd 2 or etcd 3, support both for a significant period of time.
  • Adjust storage.Interface as needed.
  • Adjust deployment scripts, again we need both to work in parallel for one, possibly two, point releases.
  • Provide update scripts/instructions to switch over a running cluster, ideally with no downtime.
@lavalamp lavalamp added priority/backlog Higher priority than priority/awaiting-more-evidence. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. labels Feb 2, 2016
@xiang90
Copy link
Contributor

xiang90 commented Feb 2, 2016

@lavalamp yea. @hongchaodeng and me will start the work soon.

@smarterclayton anyone at Redhat is interested in this?

@HardySimpson anyone from Huawei?

@smarterclayton
Copy link
Contributor

We are indeed interested in it - I'm probably going to be very heavily involved, as is @derekwaynecarr and @ncdc, and @timothysc as well.

@krousey
Copy link
Contributor

krousey commented Feb 2, 2016

@xiang90 Is there documentation for the v3 api? I've only been able to find the v2 api. Or is it the same?

@xiang90
Copy link
Contributor

xiang90 commented Feb 2, 2016

@krousey

https://github.com/coreos/etcd/blob/master/Documentation/rfc/v3api.md

It is a gRPC defined service API. Check the proto directly. Thanks!

@krousey
Copy link
Contributor

krousey commented Feb 2, 2016

@xiang90 I'm curious about the Range api. I see that I can pin it to a resource version, and define a half-open range of [start, end). And the response has a boolean indicating that there is more. I assume you expect people to redo the same RangeRequest with the same resource version, but update the start to the key of the last element they saw? Like [last_seen, end)? Would that mean we'd see the same key/value pair twice?

(Also, this is probably not the best place to have this discussion. Where would be more appropriate?)

@timothysc
Copy link
Member

Given that I did the 2.2 client update, the touch points should all be isolated sans kube2sky. I didn't think it was ready till march/april. @xiang90

/cc @kubernetes/sig-scalability

@xiang90
Copy link
Contributor

xiang90 commented Feb 2, 2016

@krousey

Would that mean we'd see the same key/value pair twice?

You can do [lasts_seen+1, end)

(Also, this is probably not the best place to have this discussion. Where would be more appropriate?)

You can always open an issue in etcd repo.

@smarterclayton
Copy link
Contributor

We should get a proposal going for the process of transforming, testing, and ensuring that it is not disruptive to the other 1.3 work. We are probably going to push etcd3 hard for scale reasons, and now that the client is coming together there's no better time to get going than now.

@krousey krousey mentioned this issue Feb 2, 2016
Closed
@HardySimpson
Copy link
Contributor

@kevin-wangzefeng @magicwang-cn @alfred-huangjian

@magicwang-cn
Copy link
Contributor

great news!

@wojtek-t
Copy link
Member

wojtek-t commented Feb 3, 2016

I'm also interested in it and may be able to help with that a bit...

@timothysc
Copy link
Member

With the storage work done, you should be able to wholesale prototype a new back-end and not have to transform the existing. The testing jig will need to be updated though.

@derekwaynecarr
Copy link
Member

+1

@wojtek-t
Copy link
Member

wojtek-t commented Feb 4, 2016

Definitely - it would be super helpful.

@adohe-zz adohe-zz mentioned this issue Feb 4, 2016
Closed
@magicwang-cn
Copy link
Contributor

@sunshine-zhd1229

@wojtek-t
Copy link
Member

I have met with @hongchaodeng last week and they are just starting doing the prototype roughly now. Once they have a prototype, the will start doing the real work. I will be coordinating with them and helping them from the k8s side.

@wojtek-t
Copy link
Member

So assigning to me, although probably most of the work will be done by CoreOS folks.

@wojtek-t wojtek-t self-assigned this Feb 15, 2016
@timothysc
Copy link
Member

@wojtek-t please /cc me on reviews.

@wojtek-t
Copy link
Member

@timothysc sure
Although, it will probably take few weeks before the first one. But sure - I will cc you.

@timothysc
Copy link
Member

thx.

@hongchaodeng hongchaodeng mentioned this issue Mar 3, 2016
Closed
@hongchaodeng
Copy link
Contributor

I opened #22448 for a specific discussion of next step. Please feel free to discuss any thoughts there.

@wojtek-t
Copy link
Member

wojtek-t commented Mar 4, 2016

We should probably deduplicate issues. @lavalamp - are you fine with closing this one?

@lavalamp
Copy link
Member Author

lavalamp commented Mar 4, 2016

I agree we should discuss it in one place, but #22448 is the dup ;)

The requirements in my first comment are not echoed in #22448, so I would prefer to close #22448.

k8s-github-robot pushed a commit that referenced this issue Nov 2, 2016
Merge pull request #35784 from wojtek-t/tweak_restore_script
94a6538 
Automatic merge from submit-queue

Support events in restore script

Ref #20504
This was referenced Nov 2, 2016
Merged
Closed
k8s-github-robot pushed a commit that referenced this issue Nov 2, 2016
Merge pull request #36062 from wojtek-t/etcd_migration_logs
2bbc801 
Automatic merge from submit-queue

Extend etcd migration logs

Ref #20504
k8s-github-robot pushed a commit that referenced this issue Nov 2, 2016
Merge pull request #35896 from wojtek-t/target_version
09a4262 
Automatic merge from submit-queue

Expose etcd version.

Ref #20504

This PRs makes use of the following two env variables
- ETCD_IMAGE (if set) allows to use a custom etcd image.
- ETCD_VERSION (if set) allows you to use custom version of etcd. The main purpose of using it may be rollback of etcd v3 API, where we need 3.0.* image, but are rolling back to 2.3.7.
@wojtek-t wojtek-t mentioned this issue Nov 2, 2016
Merged
k8s-github-robot pushed a commit that referenced this issue Nov 3, 2016
Merge pull request #36090 from wojtek-t/fix_migration_collisions
606a2dd 
Automatic merge from submit-queue

Fix collisions between different etcds during migration.

Ref #20504
This was referenced Nov 4, 2016
Merged
Merged
Merged
Merged
Merged
k8s-github-robot pushed a commit that referenced this issue Nov 4, 2016
Merge pull request #36228 from wojtek-t/storage_backend_changes
8363c55 
Automatic merge from submit-queue

Prepare for easy change to etcd3 storage backend

Ref #20504
k8s-github-robot pushed a commit that referenced this issue Nov 6, 2016
Merge pull request #36240 from wojtek-t/quota_bytes_backend
b7512d9 
Automatic merge from submit-queue

Increase quota-bytes for etcd in v3 mode

Ref #20504
@wojtek-t wojtek-t mentioned this issue Nov 7, 2016
Merged
k8s-github-robot pushed a commit that referenced this issue Nov 7, 2016
Merge pull request #36343 from wojtek-t/bump_to_3_0_14
83c5a1c 
Automatic merge from submit-queue

Bump etcd to 3.0.14 in tests

Ref #20504
k8s-github-robot pushed a commit that referenced this issue Nov 7, 2016
Merge pull request #36218 from wojtek-t/backup_before_migration
4b66d80 
Automatic merge from submit-queue

Backup before migration

Do backup before etcd migration.

Ref #20504
k8s-github-robot pushed a commit that referenced this issue Nov 8, 2016
Merge pull request #36232 from wojtek-t/cherrypick_etcd3
ece834b 
Automatic merge from submit-queue

Cherrypick etcd3 changes

Ref #20504

From production pov, this PR is supposed to be no-op (since we are not switching the defaults still).
shyamjvs pushed a commit to shyamjvs/kubernetes that referenced this issue Dec 1, 2016
Merge pull request kubernetes#36232 from wojtek-t/cherrypick_etcd3
fdb3622 
Automatic merge from submit-queue

Cherrypick etcd3 changes

Ref kubernetes#20504

From production pov, this PR is supposed to be no-op (since we are not switching the defaults still).
k8s-github-robot pushed a commit that referenced this issue Jan 5, 2017
Merge pull request #36229 from wojtek-t/bump_etcd_version
f4a8713 
Automatic merge from submit-queue (batch tested with PRs 36229, 39450)

Bump etcd to 3.0.14 and switch to v3 API in etcd.

Ref #20504

**Release note**:

```release-note
Switch default etcd version to 3.0.14.
Switch default storage backend flag in apiserver to `etcd3` mode.
```
@timothysc timothysc modified the milestones: v1.6, v1.4-nonblocking Jan 5, 2017
@timothysc
Copy link
Member

going to close this one now, we can open up issues as needed now.

@lavalamp
Copy link
Member Author

lavalamp commented Jan 9, 2017

Great work everyone! Much appreciated!

@smarterclayton
Copy link
Contributor

smarterclayton commented Jan 9, 2017 via email

@timothysc
Copy link
Member

Although my initial estimate of summer 2016 were way off :)

Yeah, my early 2016 was HIGHLY optimistic.
"No one expects the Spanish inquisition" ~ M. Python

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@timothysc timothysc

@lavalamp lavalamp

@hongchaodeng hongchaodeng

@wojtek-t wojtek-t

Labels
area/etcd priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery.
Projects
None yet
Milestone
v1.6
Development

No branches or pull requests