Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

discoveryserver: implement a minimal discovery API on top of v3 APIs #10627

Open
wants to merge 29 commits into
base: master
from

Conversation

5 participants
@philips
Copy link
Contributor

commented Apr 11, 2019

To make the etcd discovery service maintainable by the CNCF (coreos/discovery.etcd.io#63) we should make it easy to run on the v3 API and storage.

This discoveryserver package implements enough of the v2 API alongside the existing discovery.etcd.io API. I have tested it just now with the hack/insta-discovery bring-up script and it worked enough to do that.

This PR is operating under a deadline. One of the AWS nodes that is running discovery.etcd.io is scheduled for deletion on April 16th. So, ideally I can get a Kubernetes cluster up, an etcd Operator backed etcd cluster, and this code all running by end of week.

@philips

This comment has been minimized.

Copy link
Contributor Author

commented Apr 11, 2019

Oh, and the reason I would like to put this in the tree here is because trying to vendor the v2store and v2v3 pkgs was a nightmare.

@hexfusion hexfusion self-assigned this Apr 11, 2019

@philips philips force-pushed the philips:discovery branch 3 times, most recently from afda038 to eb72223 Apr 11, 2019

@philips

This comment has been minimized.

Copy link
Contributor Author

commented Apr 13, 2019

Everything is working and deployed here for demonstration purposes. My testing shows it works just fine: https://discovery.ifup.org/new

@xiang90

This comment has been minimized.

Copy link
Contributor

commented Apr 14, 2019

@philips I skimmed this PR. Overall, looks reasonable good, just a few nits.

@philips

This comment has been minimized.

Copy link
Contributor Author

commented Apr 15, 2019

@codecov-io

This comment has been minimized.

Copy link

commented Apr 18, 2019

Codecov Report

Merging #10627 into master will increase coverage by 0.42%.
The diff coverage is 69.39%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #10627      +/-   ##
==========================================
+ Coverage   71.51%   71.94%   +0.42%     
==========================================
  Files         394      406      +12     
  Lines       36658    36988     +330     
==========================================
+ Hits        26217    26610     +393     
+ Misses       8595     8479     -116     
- Partials     1846     1899      +53
Impacted Files Coverage Δ
discoveryserver/handlers/robots.go 0% <0%> (ø)
discoveryserver/handlers/home.go 0% <0%> (ø)
discoveryserver/timeprefix/time.go 100% <100%> (ø)
discoveryserver/handlers/httperror/httperror.go 100% <100%> (ø)
discoveryserver/handlers/ctx.go 100% <100%> (ø)
discoveryserver/metrics/metrics.go 100% <100%> (ø)
discoveryserver/handlers/garbage.go 56.25% <56.25%> (ø)
discoveryserver/handlers/token.go 58.66% <58.66%> (ø)
discoveryserver/handlers/health.go 65.21% <65.21%> (ø)
discoveryserver/handlers/new.go 73.97% <73.97%> (ø)
... and 43 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8146e1e...e0ea2c5. Read the comment docs.

@philips philips force-pushed the philips:discovery branch 5 times, most recently from 4b56423 to c891b68 Apr 18, 2019

philips and others added some commits Apr 10, 2019

CHANGELOG-3.4: add json-iterator/go change
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
discovery: use _config instead of config dir
_config is part of the spec, fix this
discovery: handle PUT/DELETE/GET
This handles PUT/DELETE/GET via the v2v3 storage API.
discovery: handle watch commands correctly
With this commit hack/insta-discovery/discovery now works to spin up a
three node cluster!

philips added some commits Apr 11, 2019

discoveryserver: rename from discovery
Make this more clear what it is.
discoveryserver: remove socket activation code
Remove this code as it isn't needed when running on Kubernetes
discoveryserver: fixup for new etcd embed
Remove no longer exported field SnapCount
discoveryserver: set Etcd-Index header
This is necessary for the watch behavior
discoveryserver: put dockerfile in root
The dockerfile needs the full tree context for vendor so put it in the
root.
discoveryserver: handle watches for discovery client
This makes it so watches work sufficient for a discovery client.
discoveryserver: set home to github docs
The CoreOS docs are no longer maintained. Point the home to the github
docs on how to use the discovery service.
discoveryserver: lowercase all errors
follow Go styling and lowercase all of the error messages
discoveryserver: return from default method
We don't want to continue on an unrecognized method, return from the
function.
discoveryserver: better error message on failed serialization
This could happen on all verbs, not just GET.
discoveryserver: improve log messages from TokenHandler
Include the verb to ease debugging. This was really annoying when trying
to get the integration tests running again
discoveryserver: get integration tests running again
Remove v2 specific stuff, and away it goes.
discoveryserver: add garbage collection and tests
Add periodic GC proteted by a mutex in the server. Also add a simple
integration test.
README: remove docker instructions
Replace with Kube instructions for discovery.etcd.io
discoveryserver: cleanup errors from staticcheck
staticcheck identified deprecated package and unused variables. Fix.
discoveryserver: handlers: use etcd auto sync
Ensure that the client is syncing with the known etcd machines regularly
discoveryserver: use custom metrics registry
Use a custom metrics registry so this program doesn't pickup all of the
random guages registered via init functions in the rest of etcd.

@philips philips force-pushed the philips:discovery branch from e0ea2c5 to e280055 Apr 27, 2019

@philips

This comment has been minimized.

Copy link
Contributor Author

commented May 2, 2019

The semaphore tests are bogus and this service is in production. Can someone help review and merge this?

@xiang90

This comment has been minimized.

Copy link
Contributor

commented May 2, 2019

@philips I will give it a look over the weekend.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.