Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

group entries #12

Closed
sebastianwebber opened this issue Feb 3, 2018 · 4 comments
Closed

group entries #12

sebastianwebber opened this issue Feb 3, 2018 · 4 comments
Assignees
@deankarn
Labels
question

Comments

@sebastianwebber
Copy link

Hi there,

there is possible group log entries? Im trying to print multiple log entries for a unique action, eg:

INFO running command 1
INFO output is bla
INFO running command 2
INFO output is bla

but when I use the same logic in multiple goroutines, become insane:

INFO running command 1
INFO running command 1
INFO running command 1
INFO output is bla
INFO running command 1
INFO running command 1
INFO output is bla
INFO output is bla
ERROR error for command is : foo bar
INFO output is bla
INFO running command 2
INFO running command 2
INFO output is bla
INFO output is bla
INFO running command 2
INFO output is bla

Any ideia about this issue?

Thanks
@deankarn
Copy link
Contributor

deankarn commented Feb 5, 2018 •
edited

@sebastianwebber this is really just because of how multithreaded/distributed applications work, the best way is to assign a unique ID, preferably one that’s time stamp sortable, to the start of the logging using WithFields and that will be printed with all your log entries allowing you to filter and sort.

I am updating this lib to handle this better.

@deankarn deankarn self-assigned this Feb 5, 2018
@sebastianwebber
Copy link
Author

sebastianwebber commented Feb 5, 2018 •
edited

HI @joeybloggs !

I don't know how solve this, but sorting a timestamp does not appear to be a solution. Take a look at the example bellow:

2018-02-05 14:26:26.451ms INFO  app=super-app1    Executing: docker container run --publish=8001:8081 --env APP_ENV=homo --restart=unless-stopped --detach --name super-
2018-02-05 14:26:11.234ms ERROR app=super-app1    Error running a command: exit status 1
2018-02-05 14:26:11.234ms INFO  app=super-app1    Output:
	Error: No such container: super-app1.service

2018-02-05 14:26:16.000ms INFO  app=consul     Executing: docker container run --publish=8400:8400 --publish=8500:8500 --net=host --restart=always --detach --name consul.service consul:0.7.5
2018-02-05 14:26:16.000ms INFO  app=consul     Output:
	Unable to find image 'consul:0.7.5' locally
	0.7.5: Pulling from library/consul
	709515475419: Pulling fs layer
	676ebc89eaf4: Pulling fs layer
	90884291b44b: Pulling fs layer
	258a863da0c3: Pulling fs layer
	a17c50c26b2f: Pulling fs layer
	258a863da0c3: Waiting
	a17c50c26b2f: Waiting
	676ebc89eaf4: Verifying Checksum
	676ebc89eaf4: Download complete
	709515475419: Verifying Checksum
	709515475419: Download complete
	90884291b44b: Verifying Checksum
	90884291b44b: Download complete
	709515475419: Pull complete
	676ebc89eaf4: Pull complete
	258a863da0c3: Verifying Checksum
	258a863da0c3: Download complete
	a17c50c26b2f: Verifying Checksum
	a17c50c26b2f: Download complete
	90884291b44b: Pull complete
	258a863da0c3: Pull complete
	a17c50c26b2f: Pull complete
	Digest: sha256:7fa3365242fca70d63e8e9737f4f1ac8687987d04bbdb7287bc80ea813e624ca
	Status: Downloaded newer image for consul:0.7.5
	45540090ad82e13cdffbb8c5192eb4f4282463e4aefa0f3a0275479e385e53d3

2018-02-05 14:26:16.000ms INFO  Configuring nginx proxy for consul application
2018-02-05 14:26:16.000ms WARN  app=consul     Missing configuration for this app
2018-02-05 14:26:26.450ms INFO  app=super-app2 Executing: docker container run --publish=8090:8080 --env APP_ENV=approval --restart=unless-stopped --detach --name super-app2.service registry.super-site.com/super-app2:approval
2018-02-05 14:26:26.450ms ERROR app=super-app2 Error running a command: exit status 125
2018-02-05 14:26:26.450ms INFO  app=super-app2 Output:
	Unable to find image 'registry.super-site.com/super-app2:approval' locally
	docker: Error response from daemon: Get https://registry.super-site.com/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers).
	See 'docker run --help'.

2018-02-05 14:26:26.450ms INFO  Configuring nginx proxy for super-app2 application
2018-02-05 14:26:26.450ms INFO  app=super-app2 Saving /Users/seba/tmp/nginx/homo-dashboard.super-site.com.conf file
2018-02-05 14:26:26.450ms ERROR open /Users/seba/tmp/nginx/homo-dashboard.super-site.com.conf: no such file or directory
2018-02-05 14:26:26.451ms INFO  app=super-app1    Executing: docker container run --publish=8001:8081 --env APP_ENV=homo --restart=unless-stopped --detach --name super-app1.service registry.super-site.com/super-app1:approval
2018-02-05 14:26:26.451ms ERROR app=super-app1    Error running a command: exit status 125
2018-02-05 14:26:26.451ms INFO  app=super-app1    Output:
	Unable to find image 'registry.super-site.com/super-app1:approval' locally
	docker: Error response from daemon: Get https://registry.super-site.com/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers).
	See 'docker run --help'.

2018-02-05 14:26:26.451ms INFO  Configuring nginx proxy for super-app1 application
2018-02-05 14:26:26.451ms INFO  app=super-app1    Saving /Users/seba/tmp/nginx/homo-report.super-site.com.conf file
2018-02-05 14:26:26.451ms ERROR open /Users/seba/tmp/nginx/homo-report.super-site.com.conf: no such file or directory
2018-02-05 14:26:26.451ms INFO  Restarting services
2018-02-05 14:26:26.451ms INFO  Executing: service nginx restart
2018-02-05 14:26:26.537ms INFO  Output:
	Redirecting to /bin/systemctl restart nginx.service

2018-02-05 14:26:26.537ms INFO  Done.

Note that I use a field to "group the messages" but, how they are running in a separate goroutine they happen almost at the same time.

To solve my problem, i'm thinking sort messages by field, but this turns fields obligatory. This make me think that sort by group its not the better idea for it.

The example bellow is sorted. Take a look:

2018-02-05 14:26:16.000ms INFO  app=consul     Executing: docker container run --publish=8400:8400 --publish=8500:8500 --net=host --restart=always --detach --name consul.service consul:0.7.5
2018-02-05 14:26:16.000ms INFO  app=consul     Output:
	Unable to find image 'consul:0.7.5' locally
	0.7.5: Pulling from library/consul
	709515475419: Pulling fs layer
	676ebc89eaf4: Pulling fs layer
	90884291b44b: Pulling fs layer
	258a863da0c3: Pulling fs layer
	a17c50c26b2f: Pulling fs layer
	258a863da0c3: Waiting
	a17c50c26b2f: Waiting
	676ebc89eaf4: Verifying Checksum
	676ebc89eaf4: Download complete
	709515475419: Verifying Checksum
	709515475419: Download complete
	90884291b44b: Verifying Checksum
	90884291b44b: Download complete
	709515475419: Pull complete
	676ebc89eaf4: Pull complete
	258a863da0c3: Verifying Checksum
	258a863da0c3: Download complete
	a17c50c26b2f: Verifying Checksum
	a17c50c26b2f: Download complete
	90884291b44b: Pull complete
	258a863da0c3: Pull complete
	a17c50c26b2f: Pull complete
	Digest: sha256:7fa3365242fca70d63e8e9737f4f1ac8687987d04bbdb7287bc80ea813e624ca
	Status: Downloaded newer image for consul:0.7.5
	45540090ad82e13cdffbb8c5192eb4f4282463e4aefa0f3a0275479e385e53d3

2018-02-05 14:26:16.000ms INFO  Configuring nginx proxy for consul application
2018-02-05 14:26:16.000ms WARN  app=consul     Missing configuration for this app
2018-02-05 14:26:11.234ms INFO  app=super-app1    Executing: docker container rm --force ureport.service
2018-02-05 14:26:11.234ms ERROR app=super-app1    Error running a command: exit status 1
2018-02-05 14:26:11.234ms INFO  app=super-app1    Output:
	Error: No such container: super-app1.service

2018-02-05 14:26:26.451ms INFO  app=super-app1    Executing: docker container run --publish=8001:8081 --env APP_ENV=homo --restart=unless-stopped --detach --name super-app1.service registry.super-site.com/super-app1:approval
2018-02-05 14:26:26.451ms ERROR app=super-app1    Error running a command: exit status 125
2018-02-05 14:26:26.451ms INFO  app=super-app1    Output:
	Unable to find image 'registry.super-site.com/super-app1:approval' locally
	docker: Error response from daemon: Get https://registry.super-site.com/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers).
	See 'docker run --help'.

2018-02-05 14:26:26.451ms INFO  Configuring nginx proxy for super-app1 application
2018-02-05 14:26:26.451ms INFO  app=super-app1    Saving /Users/seba/tmp/nginx/homo-report.super-site.com.conf file
2018-02-05 14:26:26.451ms ERROR open /Users/seba/tmp/nginx/homo-report.super-site.com.conf: no such file or directory
2018-02-05 14:26:26.450ms INFO  app=super-app2 Executing: docker container run --publish=8090:8080 --env APP_ENV=approval --restart=unless-stopped --detach --name super-app2.service registry.super-site.com/super-app2:approval
2018-02-05 14:26:26.450ms ERROR app=super-app2 Error running a command: exit status 125
2018-02-05 14:26:26.450ms INFO  app=super-app2 Output:
	Unable to find image 'registry.super-site.com/super-app2:approval' locally
	docker: Error response from daemon: Get https://registry.super-site.com/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers).
	See 'docker run --help'.

2018-02-05 14:26:26.450ms INFO  Configuring nginx proxy for super-app2 application
2018-02-05 14:26:26.450ms INFO  app=super-app2 Saving /Users/seba/tmp/nginx/homo-dashboard.super-site.com.conf file
2018-02-05 14:26:26.450ms ERROR open /Users/seba/tmp/nginx/homo-dashboard.super-site.com.conf: no such file or directory
2018-02-05 14:26:26.451ms INFO  Restarting services
2018-02-05 14:26:26.451ms INFO  Executing: service nginx restart
2018-02-05 14:26:26.537ms INFO  Output:
	Redirecting to /bin/systemctl restart nginx.service

2018-02-05 14:26:26.537ms INFO  Done.

@deankarn
Copy link
Contributor

deankarn commented Feb 9, 2018

@sebastianwebber until I finish the updates to the library, there won't be a great way to solve this.

but I was referring to the same process using a unique ID, not a timestamp such as https://github.com/segmentio/ksuid

@deankarn
Copy link
Contributor

@sebastianwebber I buried my head into updating this lib over the weekend, it's mostly the same, let me know if you run into any issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@deankarn deankarn

Labels
question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sebastianwebber @deankarn