Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add UDP input back #2751

Merged
merged 1 commit into from
Jun 4, 2015
Merged

Conversation

renan-strauss
Copy link
Contributor

Hi there,

Seeing that my previous pull request is not likely to get merged because of the big refactoring going on, here's a PR to add UDP support back (not sure why it was removed) in alpha1. I hope this will be helpful, and I'm looking forward to the next release ! (got rc31 running in production and still seeing some issues as well as feeling the lack of CQs).

Keep up the good work,
Renan.

  • CHANGELOG.md updated
  • Rebased/mergable
  • Tests pass
  • Sign CLA (if not already signed)

@corylanou
Copy link
Contributor

@renan- Thanks for the work. The reason we removed it was we are not going to support json on this endpoint. We are only going to support the new line protocol here likely. I'll get @pauldix to confirm though to be sure. Regardless, I imagine at the very least we would do something similar to the http endpoint and detect json vs. line protocol to support both.

@pauldix
Copy link
Member

pauldix commented Jun 3, 2015

Yeah, I'd prefer to have UDP support only the line protocol. Which would mean it would need configuration options like the Graphite and CollectD input plugins. Set which database and retention policy you're writing into.

See #2696 for discussion on the merits of line protocol over JSON.

@renan-strauss
Copy link
Contributor Author

Thanks for the quick answers guys ! I desperately need UDP support for my use case (to avoid any latency when writing data) so would you be interested in another PR supporting line protocol instead of JSON ?

@pauldix
Copy link
Member

pauldix commented Jun 3, 2015

@renan- that would be great, we'd gladly take it. Going to cut RC32 tomorrow so if you can get it in time, it'll be included! Thanks again for helping out!

@pauldix
Copy link
Member

pauldix commented Jun 3, 2015

@renan- oh, another thing to look at is to have it use the batcher that graphite and collectd now use.

@renan-strauss
Copy link
Contributor Author

Thanks for the tip, starting to work on it.

@renan-strauss renan-strauss force-pushed the udp-support-alpha1 branch 2 times, most recently from 0f27ecf to df60fa5 Compare June 3, 2015 17:33
@renan-strauss
Copy link
Contributor Author

Updated the code, now waiting for your feedback.

@@ -77,6 +78,7 @@ type Config struct {
Graphites []graphite.Config `toml:"graphite"`
Collectd collectd.Config `toml:"collectd"`
OpenTSDB opentsdb.Config `toml:"opentsdb"`
UDP udp.Config `toml:"udp"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add a test in config_test.go for this addition of the config.

@corylanou
Copy link
Contributor

@renan- Thanks for the hard work! Certainly headed in the correct direction. Please let me know if you have any questions. I know not all of our services are in perfect shape yet either, but feel free to look at collectd to get an idea of some of the patterns we are currently using (I reference some as well in my comments).

@renan-strauss renan-strauss force-pushed the udp-support-alpha1 branch 8 times, most recently from b203245 to 4bf9784 Compare June 3, 2015 21:55
@renan-strauss renan-strauss force-pushed the udp-support-alpha1 branch 2 times, most recently from f0dd608 to 1f102f1 Compare June 3, 2015 22:03
@renan-strauss
Copy link
Contributor Author

@corylanou You're welcome! I updated the PR once again based on your comments. I still wonder if the retention policy should be in config or always set to the default one. One more thing: what about ConsistencyLevel value when creating a WritePointsRequest ?

@renan-strauss renan-strauss changed the title Add UDP service back in alpha1 Add UDP input back Jun 3, 2015
@@ -1010,6 +1010,9 @@ func NewConfig() *run.Config {
c.HTTPD.Enabled = true
c.HTTPD.BindAddress = "127.0.0.1:0"
c.HTTPD.LogEnabled = testing.Verbose()

c.UDP.BindAddress = "127.0.0.1:0"
c.UDP.Database = "db0"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We actually don't want to add these. This would enable the UDP endpoint for every test. We use this function to create a base config for the tests to use, and then turn on specific endpoints in the tests themselves. Simply revert the change to this file and we should be fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests on CircleCI won't pass without these (panic because of the error returned by ListenAndServe when BindAddress or Database are empty)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the real problem here is that the server isn't observing if UDP is enabled, and actually adding it to the services. I'm guessing a couple of our endpoints have that problem. I'll merge this, and then do another PR that fixes services starting that aren't enabled.

@corylanou
Copy link
Contributor

@renan- Awesome! Thanks for all the work. Very much appreciated! Had one last minor comment. Looks like something changed that is preventing a merge so you'll have to rebase quick to make this mergeable.

For now let the default RP be used. It can be a feature we add in the future to specify an RP. If we do, all endpoints should likely support it as well.

Same with consistency level. feels like that should be a config option as well, but I'm fine with this PR not addressing it. Can be an easy feature to add.

Thanks again for the quick turn around. Hopefully we'll get this merged for tomorrow's RC.

corylanou added a commit that referenced this pull request Jun 4, 2015
@corylanou corylanou merged commit fba576b into influxdata:master Jun 4, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants