Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Byi fix timeseries #14

Merged
merged 3 commits into from Apr 24, 2019
Merged

Byi fix timeseries #14

merged 3 commits into from Apr 24, 2019

Conversation

bin3377
Copy link

@bin3377 bin3377 commented Apr 23, 2019

Fix the problem: only one point goes into pipeline in one HTTP request.
The problems are:

  1. parse plug-ins need to implement parse(text) with yield return with time and record (e.g. hash)
    refer to:
    https://docs.fluentd.org/v0.12/articles/plugin-development#parser-plugins
    https://github.com/fluent/fluentd/tree/master/lib/fluent/plugin
  2. since single HTTP request may contains a list of TimeSeries not one, add a wrapper to make it a hash and also change datapoint plugin to handle it

update unit test make sure the time stamp and records are correctly bring in.

verified with private docker image on stag. the timeseries now giving points in fixed interval

log.trace "protobuf::parse - in: (#{text.bytesize}/#{inflated.bytesize}), out: #{decoded.timeseries.length}"
record = {}
record[KEY_TIMESERIES] = decoded.timeseries
yield Fluent::EventTime.now, record
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we override the time eventually using the value in record itself?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. this time is only used in fluentd pipeline

@lei-sumo
Copy link
Contributor

lei-sumo commented Apr 24, 2019

cc @ggarg2906sumo This is supposed to fix the missing metrics issue.

@bin3377 bin3377 merged commit cd942c8 into master Apr 24, 2019
@bin3377 bin3377 deleted the byi-fix-timeseries branch April 24, 2019 02:32
@bin3377
Copy link
Author

bin3377 commented Apr 24, 2019

@ggarg2906sumo new image pushed to dockerhub now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants