Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add the Elastic product origin header when talking to Elasticsearch or Kibana. #29966

Merged
merged 5 commits into from
Jan 25, 2022
Merged

Add the Elastic product origin header when talking to Elasticsearch or Kibana. #29966

merged 5 commits into from
Jan 25, 2022

Conversation

cmacknz
Copy link
Member

@cmacknz cmacknz commented Jan 24, 2022 •
edited
Loading

What does this PR do?

Adds the x-elastic-product-origin header when communicating with Elasticsearch or Kibana.

Why is it important?

If Elasticsearch deprecates an API we use in the future, this header will prevent Kibana from presenting the warning to the user. It will also tag the Elasticsearch deprecation log message with the header value to more easily attribute deprecated API usage to products

Note that there is no known usage of deprecated Elasticsearch APIs in beats currently.

Checklist

  • [ x ] My code follows the style guidelines of this project
  • [ x ] I have commented my code, particularly in hard-to-understand areas
    - [ ] I have made corresponding changes to the documentation
    - [ ] I have made corresponding change to the default configuration files
  • [ x ] I have added tests that prove my fix is effective or that my feature works
    - [ ] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Run any beat and inspect the headers attached to requests. For example, filebeat test output will make a request that should have the new headers attached.

I ran filebeat (as an arbitrary choice) locally and inspected the HTTP requests to Elasticsearch with Wireshark to confirm the expected header was attached. I haven't exhaustively tested all beats, or all HTTP interactions.

@cmacknz cmacknz added enhancement backport-v8.0.0 Automated backport with mergify Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team backport-v7.17.0 Automated backport with mergify labels Jan 24, 2022
@cmacknz cmacknz requested a review from ruflin January 24, 2022 15:00
@cmacknz cmacknz self-assigned this Jan 24, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jan 24, 2022
@elasticmachine
Copy link
Collaborator

elasticmachine commented Jan 24, 2022 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-01-25T15:52:46.544+0000

  • Duration: 118 min 26 sec

  • Commit: 498c2b2

Test stats 🧪

Test Results
Failed 0
Passed 47951
Skipped 4284
Total 52235

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@ruflin ruflin requested a review from ph January 25, 2022 12:11
ruflin
ruflin reviewed Jan 25, 2022
View reviewed changes
Copy link
Member

@ruflin ruflin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall change LGTM. Did not run it locally for testing. @ph Would be good to get an additional pair of eyes on this.

@cmacknz
Copy link
Member Author

cmacknz commented Jan 25, 2022

I ran filebeat (as an arbitrary choice) locally and inspected the HTTP requests to Elasticsearch with Wireshark to confirm the expected header was attached. I haven't exhaustively tested all beats, or all HTTP interactions.

ph
ph suggested changes Jan 25, 2022
View reviewed changes
Copy link
Contributor

@ph ph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cmacknz You can Remove Fleet, it's already implemented in fleet-server at https://github.com/elastic/fleet-server/blame/d435007d9ad40f9f7b5ed02b7a722612faffbda8/internal/pkg/config/output.go#L149. The implement is different since we don't want to increase coupling with libbeat.

@cmacknz cmacknz merged commit 5f3dd3e into elastic:master Jan 25, 2022
mergify bot pushed a commit that referenced this pull request Jan 25, 2022
@cmacknz
Add the Elastic product origin header when talking to Elasticsearch o…
25540cb 
…r Kibana. (#29966)

Set the beats product origin header by default when communicating with Elasticsearch or Kibana.

(cherry picked from commit 5f3dd3e)

# Conflicts:
#	metricbeat/module/kibana/settings/settings.go
#	metricbeat/module/kibana/stats/stats.go
#	metricbeat/module/kibana/status/status.go
@mergify mergify bot mentioned this pull request Jan 25, 2022
Merged
mergify bot pushed a commit that referenced this pull request Jan 25, 2022
@cmacknz
Add the Elastic product origin header when talking to Elasticsearch o…
176bd03 
…r Kibana. (#29966)

Set the beats product origin header by default when communicating with Elasticsearch or Kibana.

(cherry picked from commit 5f3dd3e)
@mergify mergify bot mentioned this pull request Jan 25, 2022
Merged
This was referenced Jan 25, 2022
Closed
Open
cmacknz added a commit that referenced this pull request Jan 25, 2022
@mergify @cmacknz
[7.17](backport #29966) Add the Elastic product origin header when ta…
ca2bd68 
…lking to Elasticsearch or Kibana. (#30000)

Add the Elastic product origin header when talking to Elasticsearch or Kibana. (#29966)

Set the beats product origin header by default when communicating with Elasticsearch or Kibana.

(cherry picked from commit 5f3dd3e)

# Conflicts:
#	metricbeat/module/kibana/settings/settings.go
#	metricbeat/module/kibana/stats/stats.go
#	metricbeat/module/kibana/status/status.go

Co-authored-by: Craig MacKenzie <craig.mackenzie@elastic.co>
cmacknz added a commit that referenced this pull request Jan 25, 2022
@mergify @cmacknz
Add the Elastic product origin header when talking to Elasticsearch o…
4c5fe14 
…r Kibana. (#29966) (#30001)

Set the beats product origin header by default when communicating with Elasticsearch or Kibana.

(cherry picked from commit 5f3dd3e)

Co-authored-by: Craig MacKenzie <craig.mackenzie@elastic.co>
@simitt simitt mentioned this pull request Jan 26, 2022
Closed
@simitt simitt mentioned this pull request Jan 26, 2022
Merged
v1v added a commit that referenced this pull request Jan 28, 2022
@v1v
Merge remote-tracking branch 'upstream/7.17' into v1v-patch-1
c28c548 
* upstream/7.17: (30 commits)
  [7.17](backport #29966) Add the Elastic product origin header when talking to Elasticsearch or Kibana. (#30000)
  [Heartbeat] Change size of data on ICMP packet (#29948) (#29978)
  Add clarification about enableing dashboard loading (#29985) (#29989)
  Improve aws-s3 gzip file detection to avoid false negatives (#29969) (#29974)
  ci: docker login step for pulling then pushing (#29960) (#29963)
  x-pack/auditbeat/module/system/socket: get full length path and arg from /proc when not available from kprobe (#29410) (#29958)
  [Automation] Update elastic stack version to 7.17.0-ab4975a2 for testing (#29956)
  [Automation] Update elastic stack version to 7.17.0-1bd58b32 for testing (#29938)
  [7.17](backport #29913) [Metricbeat] gcp.gke: fix overview dashboard (#29914)
  [7.17](backport #29605) Fix annotation enrichment (#29834)
  [Automation] Update elastic stack version to 7.17.0-e1efbe3a for testing (#29922)
  [Automation] Update elastic stack version to 7.17.0-68da5d12 for testing (#29904)
  [7.17][Heartbeat] Defer monitor / ICMP errors to monitor runtime / ES (backport #29413) (#29896)
  Merge pull request from GHSA-rj4h-hqvq-cc6q
  [7.17](backport #29681) Change docker image from CentOS 7 to Ubuntu 20.04 (#29817)
  Fix YAML indentation in `parsers` examples (#29663) (#29894)
  [Automation] Update elastic stack version to 7.17.0-079761a0 for testing (#29864)
  Fix Filebeat dissect processor field tokenization in documentation (#29680) (#29883)
  Enable require_alias for Bulk requests for all actions when target is a write alias (#29879)
  Update Index template loading guide to use the correct endpoint (#29869) (#29877)
  ...
yashtewari pushed a commit to build-security/beats that referenced this pull request Jan 30, 2022
@cmacknz @yashtewari
Add the Elastic product origin header when talking to Elasticsearch o…
03d4620 
…r Kibana. (elastic#29966)

Set the beats product origin header by default when communicating with Elasticsearch or Kibana.
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
@v1v
Merge remote-tracking branch 'upstream/7.17' into v1v-patch-1
82291ab 
* upstream/7.17: (30 commits)
  [7.17](backport elastic#29966) Add the Elastic product origin header when talking to Elasticsearch or Kibana. (elastic#30000)
  [Heartbeat] Change size of data on ICMP packet (elastic#29948) (elastic#29978)
  Add clarification about enableing dashboard loading (elastic#29985) (elastic#29989)
  Improve aws-s3 gzip file detection to avoid false negatives (elastic#29969) (elastic#29974)
  ci: docker login step for pulling then pushing (elastic#29960) (elastic#29963)
  x-pack/auditbeat/module/system/socket: get full length path and arg from /proc when not available from kprobe (elastic#29410) (elastic#29958)
  [Automation] Update elastic stack version to 7.17.0-ab4975a2 for testing (elastic#29956)
  [Automation] Update elastic stack version to 7.17.0-1bd58b32 for testing (elastic#29938)
  [7.17](backport elastic#29913) [Metricbeat] gcp.gke: fix overview dashboard (elastic#29914)
  [7.17](backport elastic#29605) Fix annotation enrichment (elastic#29834)
  [Automation] Update elastic stack version to 7.17.0-e1efbe3a for testing (elastic#29922)
  [Automation] Update elastic stack version to 7.17.0-68da5d12 for testing (elastic#29904)
  [7.17][Heartbeat] Defer monitor / ICMP errors to monitor runtime / ES (backport elastic#29413) (elastic#29896)
  Merge pull request from GHSA-rj4h-hqvq-cc6q
  [7.17](backport elastic#29681) Change docker image from CentOS 7 to Ubuntu 20.04 (elastic#29817)
  Fix YAML indentation in `parsers` examples (elastic#29663) (elastic#29894)
  [Automation] Update elastic stack version to 7.17.0-079761a0 for testing (elastic#29864)
  Fix Filebeat dissect processor field tokenization in documentation (elastic#29680) (elastic#29883)
  Enable require_alias for Bulk requests for all actions when target is a write alias (elastic#29879)
  Update Index template loading guide to use the correct endpoint (elastic#29869) (elastic#29877)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ruflin ruflin ruflin left review comments

@ph ph ph approved these changes

Assignees

@cmacknz cmacknz

Labels
backport-v7.17.0 Automated backport with mergify backport-v8.0.0 Automated backport with mergify enhancement Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@cmacknz @elasticmachine @ph @ruflin