Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make log routing for containers flexible #179

Merged
merged 6 commits into from Jan 31, 2020

Conversation

megastef
Copy link
Contributor

Making log routing more flexible:

  • support multiple elasticsearch urls in LOGS_RECEIVER_URL container label/env var
  • support multiple elasticsearch urls in logs-receiver-url Kubernetes annotation
  • support list of pre-configured ES output plugins by setting LOGS_DESTINATION=configName,configName2

Example 1:
Run nginx and ship logs to local elasticsearch and to sematext cloud.

	docker run -d \
	-e LOGS_RECEIVER_URL="https://elasticsearch:9200/logs,https://logsene-receiver.sematext.com/YOUR_TOKEN_HERE" \
	-p 8282:80 nginx

Logagent will ship logs to both servers.

Example 2
Assume we have two elasticsearch output plugins configured in loggent.conf:

output: 
  semtatext:
    module: elasticsearch 
    url: https://logsene-receiver.sematext.com 
    index: YOUR_LOGS_TOKEN
  local-elk:
   module: elasticsearch 
   url: http://elasticsearch:9200
   index: logs

Then you can run a container and specify log destinations.

docker run -d -e LOGS_DESTINATION="sematext,local-elk" -p 8282:80 nginx

@otisg
Copy link
Member

otisg commented Jan 29, 2020

@megastef can you send the PR for docs, too?

Copy link
Contributor

@adnanrahic adnanrahic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only two things I noticed, the parseReceiverList functions can be refactored to use higher-order functions, and it should be put in a dedicated file so we can require it where it's needed to avoid code duplication.

lib/plugins/input/docker/dockerInspect.js Outdated Show resolved Hide resolved
lib/plugins/output-filter/kubernetes-enrichment.js Outdated Show resolved Hide resolved
@megastef
Copy link
Contributor Author

@otisg yes, I'm working on the PR for Logagent docs.

@megastef megastef merged commit 7a6372f into master Jan 31, 2020
@megastef megastef deleted the make_log_routing_for_containers_flexible branch January 31, 2020 13:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants