Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docker-compose override file for Traffic Monitoring #2224

Merged
merged 4 commits into from Feb 21, 2022

Conversation

tstadel
Copy link
Member

@tstadel tstadel commented Feb 21, 2022

Sometimes it's invaluable to inspect and monitor the traffic between ui, rest_api and elasticsearch: E.g. ensuring request correlation, debugging elasticsearch problems, etc.). Till now there's no easy way to monitor traffic. A simple docker-compose override file makes it super easy for anyone without the need to fiddle with the own environment.

Proposed changes:

  • add docker-compose override file which enables HTTP traffic monitoring between containers using mitmproxy
  • add use_system_proxy param to ElasticSearchDocumentStore to force use of RequestsHttpConnection in elasticsearch client which automatically uses the system proxy (e.g. HTTP_PROXY and HTTPS_PROXY env variable in linux)

Usage:
Run

docker-compose -f docker-compose[-gpu].yml -f docker-compose.mitm.yml up

Watch traffic on localhost:8081.
(You might need to rebuild rest_api docker container, to enable use_system_proxy)

Status (please check what you already did):

  • First draft (up for discussions & feedback)
  • Final code

@tstadel tstadel marked this pull request as ready for review February 21, 2022 12:56
Copy link
Contributor

@ZanSara ZanSara left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh no, another parameter to the __init__ method of ElasticsearchDocumentStore!! 😄

Just kidding, it's the right thing to do here. However, __init__ parameters are really proliferating. I'll add this PR as yet another reason to review docstores' architecture.

@ZanSara ZanSara mentioned this pull request Feb 21, 2022
Copy link
Member

@julian-risch julian-risch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 👍

@tstadel tstadel merged commit b2e6dcc into master Feb 21, 2022
@tstadel tstadel deleted the monitor_docker_traffic branch February 21, 2022 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants