Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert heartbeat to new publisher pipeline #4591

Merged
merged 3 commits into from Jul 3, 2017

Conversation

Projects
None yet
4 participants
@urso
Copy link
Collaborator

commented Jul 2, 2017

Requires: #4554

  • install EventMetadata processing with beat.Client on connect
  • now Task and Job have different return type:
    • a Task can optionally return event fields of type common.MapStr
    • a Job returns/reports full events by wrapping the tasks events into a beat.Event type with additional annotations and start timestamp
    • unexport some 'local' types
  • slight enhancements to fields/tags testing in heartbeat system tests
  • add 'processors' setting to heartbeat monitors (requires doc update)

@urso urso force-pushed the urso:pipeline/heartbeat branch from ea13047 to 5ed4fd4 Jul 3, 2017

urso added some commits Jul 2, 2017

Convert heartbeat to new publisher pipeline
- install EventMetadata processing with beat.Client on connect
- split in return types between Task and Job:
  - a Task can optionally return event fields of type common.MapStr
  - a Job returns/reports full events by wrapping the Tasks return into a
    beat.Event type with additional annotations
- unexport some 'local' types

@urso urso force-pushed the urso:pipeline/heartbeat branch from 5ed4fd4 to 4724c76 Jul 3, 2017

@urso urso added review and removed in progress labels Jul 3, 2017

@ruflin

ruflin approved these changes Jul 3, 2017

@@ -66,11 +62,12 @@ func (bt *Heartbeat) Run(b *beat.Beat) error {

<-bt.done

bt.manager.Stop()

This comment has been minimized.

Copy link
@ruflin

ruflin Jul 3, 2017

Collaborator

could also be done as a defer statement on line 46 I think? Is there a reason you use bt.manager instead of manager`

This comment has been minimized.

Copy link
@urso

urso Jul 3, 2017

Author Collaborator

line 46 is another function

t.cancel = m.manager.jobControl.Add(t.config.Schedule, id, job)
processors, err := processors.New(t.config.Processors)
if err != nil {
logp.Critical("Fail to load monitor processors: %v", err)

This comment has been minimized.

Copy link
@ruflin

ruflin Jul 3, 2017

Collaborator

Not for this PR but thinking what we should do in this case? Alternatively we could return an error and move the creating of the jobs into a second for loop. Means all jobs only get created if all tasks work. Not sure what the user expects.

This comment has been minimized.

Copy link
@urso

urso Jul 3, 2017

Author Collaborator

This loop is about creating tasks, after merging the tasks settings (via json file) with the actual monitor settings. That is, this loop also drives monitors config reloading and is to be replaced is metricbeat/filebeat config reloading. In the filebeat/metricbeat case, the reloading is less flexible in handline settings, allowing us to load the configs once and not per task.

@urso urso force-pushed the urso:pipeline/heartbeat branch from 4724c76 to bc9c4c9 Jul 3, 2017

@ruflin

ruflin approved these changes Jul 3, 2017

func (f funcTask) Run() (common.MapStr, []TaskRunner, error) { return f.run() }

/*

This comment has been minimized.

Copy link
@ruflin

ruflin Jul 3, 2017

Collaborator

Leftover?

@ruflin ruflin merged commit 6d8a87a into elastic:master Jul 3, 2017

4 checks passed

CLA Commit author is a member of Elasticsearch
Details
beats-ci Build finished.
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@urso urso referenced this pull request Jul 3, 2017

Closed

Publisher Pipeline #4598

22 of 22 tasks complete

@dedemorton dedemorton referenced this pull request Jul 25, 2017

Closed

Beats doc updates in 6.0 #4540

42 of 42 tasks complete

@dedemorton dedemorton referenced this pull request Dec 14, 2017

Closed

Beats doc updates in 6.x #5632

37 of 37 tasks complete

@dedemorton dedemorton removed the needs_docs label Jan 22, 2018

@dedemorton

This comment has been minimized.

Copy link
Contributor

commented Jan 22, 2018

@urso I'm removing the needs_docs label here because you've already added the processors docs. If additional doc work is required, please open an issue against the docs.

@urso urso deleted the urso:pipeline/heartbeat branch Feb 19, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.