Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark the shipper output's conversion from publisher.Event #32081

Closed
Tracked by #34 ...
cmacknz opened this issue Jun 23, 2022 · 3 comments · Fixed by #32273
Closed
Tracked by #34 ...

Benchmark the shipper output's conversion from publisher.Event #32081

cmacknz opened this issue Jun 23, 2022 · 3 comments · Fixed by #32273
Assignees
Labels
Agent Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team v8.4.0

Comments

@cmacknz
Copy link
Member

cmacknz commented Jun 23, 2022

When beats are run under agent, they will soon be able to use the the new Elastic agent data shipper. Events are sent to the shipper using the shipper gRPC output which involves conversion from publisher.Event to the shipper event format.

func toShipperEvent(e publisher.Event) (*sc.Event, error) {
meta, err := convertMapStr(e.Content.Meta)
if err != nil {
return nil, fmt.Errorf("failed to convert event metadata to protobuf: %w", err)

Write a set of Go benchmarks for this conversion and attempt to minimize the the overhead of this conversion process. This conversion will be directly in the data path of every beat and we should be careful to avoid introducing performance regressions when using the shipper. At a minimum the benchmarks should help developers avoid making the current performance worse as the protocol evolves.

@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@cmacknz cmacknz changed the title Benchmark the shipper output's conversation from publisher.Event Benchmark the shipper output's conversion from publisher.Event Jun 23, 2022
@jlind23
Copy link
Collaborator

jlind23 commented Jul 4, 2022

@cmacknz any reason why you added the 8.5-candidate label here?

@cmacknz
Copy link
Member Author

cmacknz commented Jul 5, 2022

Removed it, it's already in 8.4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Agent Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team v8.4.0
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants