enable logging to stdout for access logs #1683

mjeri · 2017-05-30T10:12:24Z

This PR enables sending the access logs to the stdout. It should therefore solve #1306. Consistently to TraefikLogsFile, it uses stdout now when the config for AccessLogsFile is omitted or empty.

timoreimann

Thanks! Left a few remarks.

timoreimann · 2017-05-30T14:40:37Z

middlewares/accesslog/logger.go

+		}
+		file = f
+	} else {
+		file = os.Stdout


The else case could be dropped if we set file's default to stdout above.

timoreimann · 2017-06-06T19:42:26Z

middlewares/accesslog/logger_test.go

-	if logdata, err := ioutil.ReadFile(logfilePath); err != nil {
+	if logdata, err := ioutil.ReadFile(logfilePath); err == nil {
+		assertValidLogData(t, logdata)
+	} else {
 		fmt.Printf("%s\n%s\n", string(logdata), err.Error())
 		assert.NoError(t, err)


This should be a require I think.

And if it is, we can drop the else by handling the err != nil case first.

timoreimann · 2017-06-06T19:43:35Z

middlewares/accesslog/logger_test.go

-	if logdata, err := ioutil.ReadFile(logfilePath); err != nil {
+	if logdata, err := ioutil.ReadFile(logfilePath); err == nil {
+		assertValidLogData(t, logdata)
+	} else {
 		fmt.Printf("%s\n%s\n", string(logdata), err.Error())


I think we should move the details into the last arguments of the NoError call on the next line.

Apart from saving us one line, it will also make sure that the error output and our custom details are shown together.

I compared the messages a bit with and without. I dropped it then completely as it doesn't really add value. The logdata will always be empty given there is an error and err infos we get anyway using require.NoError.

timoreimann · 2017-06-06T19:46:05Z

middlewares/accesslog/logger_test.go

+func TestNewLogHandlerOutputStdout(t *testing.T) {
+	file, err := ioutil.TempFile("", "testlogger")
+	if err != nil {
+		t.Fatalf("failed to create temp file: %s", err)


We should stick to either stdlib testing or testify but not mix.

I cleaned up the whole test case to use testify completely.

timoreimann · 2017-06-06T19:47:10Z

middlewares/accesslog/logger.go

+	if len(filePath) > 0 {
+		f, err := openAccessLogFile(filePath)
+		if err != nil {
+			return nil, err


Will just returning err give us enough context?

Good point, I added some specific error message that adds more context to the previous one.

timoreimann · 2017-06-06T20:08:58Z

middlewares/accesslog/logger_test.go

+	defer file.Close()
+	// capture stdout in a file
+	stdOut := os.Stdout
+	os.Stdout = file


Ideally, it'd be nice to use a memory-mapped file. I did a quick search and couldn't find an easy solution, however.

I also checked and saw no "native" way of doing this, whereas the current approach with TempFile is included. Therefore I would stick with the current solution.

timoreimann · 2017-06-06T20:09:34Z

middlewares/accesslog/logger_test.go

+
+	logger.ServeHTTP(&logtestResponseWriter{}, req, LogWriterTestHandlerFunc)
+
+	written, err := ioutil.ReadFile(file.Name())


We should probably require.NoError on err?

Good catch, I was a bit surprised by the compiler does not complain even though we do not use err anymore after this call. The reason is clear though, its already initialised and used before.

timoreimann · 2017-06-06T20:10:10Z

middlewares/accesslog/logger_test.go

+	logger.ServeHTTP(&logtestResponseWriter{}, req, LogWriterTestHandlerFunc)
+
+	written, err := ioutil.ReadFile(file.Name())
+	assert.NotZero(t, len(written), "expected access log message on stdout")


require maybe? Does assertValidLogData make sense if the written length is zero?

Nope it doesn't, good catch.

timoreimann · 2017-06-06T20:12:28Z

middlewares/accesslog/logger_test.go

@@ -92,7 +138,7 @@ func printLogdata(logdata []byte) string {
 	return fmt.Sprintf(


It looks like this part will be executed even in the case where all assertions succeed. Do we want that?

Hmm its the error message we have to provide in a "static" way to the assert/require invocations. Therefore we have to do it at least once for each test case. I improved the message a bit and inlined the formatting into the requireValidLogData, so that we only do it once :)

timoreimann · 2017-06-06T20:13:12Z

middlewares/accesslog/logger_test.go

@@ -82,7 +128,7 @@ func TestLogger(t *testing.T) {
 		assert.Equal(t, fmt.Sprintf("%d", len(helloWorld)), tokens[7], printLogdata(logdata))


I find the part up to and include this line somewhat hard to read. Maybe a combination of a requires and a switch statement could improve readability? WDYT?

Changed it to use require and its much clearer imho.

ldez · 2017-06-07T09:06:49Z

@marco-jantke can you resolve conflicts?

mjeri · 2017-06-07T10:00:15Z

@ldez thanks for the notice, done.
@timoreimann thanks for the review. I added the feedback, though I had to do a rebase in order to resolve the conflicts.

ldez

Could you change the behavior?

We want this behavior:

[accessLog]
# with nothing

:-> StdOut

[accessLog]
filePath = "/path/to/log/log.txt"

:-> file

# nothing at all

:-> no output

ldez · 2017-06-07T10:03:18Z

docs/toml.md

 Using the default CLF option is simple, e.g.

 ```toml
 [accessLog]
-  filePath = "/path/to/access.log"
+  filePath = "/path/to/access.log" # If empty, logs to stdout


could you remove this comment? Prefer add a line above.

ldez · 2017-06-07T10:03:21Z

docs/toml.md

 ```

 To write JSON format logs, specify `json` as the format:
 ```toml
 [accessLog]
-  filePath   = "/path/to/access.log"
+  filePath   = "/path/to/access.log" # If empty, logs to stdout


could you remove this comment? Prefer add a line above.

ldez · 2017-06-07T10:04:15Z

docs/toml.md

@@ -41,6 +41,7 @@
 # traefikLogsFile = "log/traefik.log"

 # Access logs file
+# If not defined, logs to stdout


this option is deprecated, you need to add the comment in the new option.

ldez · 2017-06-07T10:10:48Z

middlewares/accesslog/logger_test.go

-		}
-	}
-	return false
+func requireValidLogData(t *testing.T, logData []byte) {


I prefer assert to require in this case.

Having require in the function name makes it clear that a function call to it might abort the test evaluation. This is something the function name should communicate for me and guards against unexpected behaviour. WDYT?

I understand but:

it's your main assertion. For me, you don't need to mimic Testify in this case.

this method contains a mix of require and assert, you cannot guarantee it might abort.

Ok, I renamed it.

mjeri · 2017-06-07T12:41:49Z

@ldez thats exactly the behaviour we have now. Improved docs to match your description.

ldez

Good Job 👍

juliens · 2017-06-12T13:27:20Z

docs/toml.md

-Alternatively logs can be written in JSON. 
-Using the default CLF option is simple, e.g.
+Alternatively logs can be written in JSON.
+By default logs will be written to stdout and use the CLF format.


Can you change this part in order to understand that without the [accessLog] section, we will not have any accesslog ?

Moreover can you add the same information in the traefik.sample.toml.

I rephrased the configuration a bit to make it hopefully more clear and easier to read. Also I added and adjusted documentation in the traefik.sample.toml file. Good catch, didn't think about this one..

juliens · 2017-06-12T13:30:47Z

server/configuration.go

@@ -45,7 +45,7 @@ type GlobalConfiguration struct {
 	CheckNewVersion           bool                    `description:"Periodically check if a new version has been released"`
 	AccessLogsFile            string                  `description:"(Deprecated) Access logs file"` // Deprecated
 	AccessLog                 *types.AccessLog        `description:"Access log settings"`
-	TraefikLogsFile           string                  `description:"Traefik logs file"`
+	TraefikLogsFile           string                  `description:"Traefik logs file. Stdout is used when omitted or empty"`


This information must be in AccessLog.FilePath description ?

I added it here by intend, as this is actually the behaviour of the TraefikLogsFile and as it wasn't documented yet, I added this comment here as well. Nevertheless, I also extended the documentation in AccessLog.FilePath. Good point.

ldez · 2017-06-13T10:38:34Z

@marco-jantke it's time to squash 😃

timoreimann

LGTM, thanks.

emilevauge

Thanks a lot @marco-jantke !
LGTM

ldez added area/logs kind/enhancement a new or improved feature. status/1-needs-design-review labels May 30, 2017

mjeri force-pushed the 1306-access-logs-to-stdout branch from abfaf95 to cdfaa6e Compare June 6, 2017 10:28

timoreimann suggested changes Jun 6, 2017

View reviewed changes

ldez added the contributor/needs-resolve-conflicts label Jun 6, 2017

mjeri force-pushed the 1306-access-logs-to-stdout branch 2 times, most recently from ee10958 to f78d0a1 Compare June 7, 2017 09:01

mjeri force-pushed the 1306-access-logs-to-stdout branch 3 times, most recently from c6da678 to c7a1048 Compare June 7, 2017 09:58

ldez requested changes Jun 7, 2017

View reviewed changes

ldez added status/2-needs-review and removed status/1-needs-design-review labels Jun 7, 2017

ldez removed the contributor/needs-resolve-conflicts label Jun 7, 2017

ldez approved these changes Jun 8, 2017

View reviewed changes

juliens reviewed Jun 12, 2017

View reviewed changes

ldez added the contributor/waiting-for-feedback label Jun 12, 2017

enable logging to stdout for access logs

adbf680

mjeri force-pushed the 1306-access-logs-to-stdout branch from 27f5dc7 to adbf680 Compare June 13, 2017 10:40

ldez removed the contributor/waiting-for-feedback label Jun 13, 2017

timoreimann approved these changes Jun 13, 2017

View reviewed changes

ldez added this to the 1.4 milestone Jun 13, 2017

emilevauge approved these changes Jun 13, 2017

View reviewed changes

ldez merged commit 885b9f3 into traefik:master Jun 13, 2017

ldez removed the status/2-needs-review label Jun 13, 2017

timoreimann mentioned this pull request Jun 14, 2017

send traefik access logs to stdout #1306

Closed

djalal mentioned this pull request Jun 19, 2017

feature request: add a log on not matched frontend rule #1762

Closed


		logger.ServeHTTP(&logtestResponseWriter{}, req, LogWriterTestHandlerFunc)

		written, err := ioutil.ReadFile(file.Name())

		@@ -92,7 +138,7 @@ func printLogdata(logdata []byte) string {
		return fmt.Sprintf(

		@@ -82,7 +128,7 @@ func TestLogger(t *testing.T) {
		assert.Equal(t, fmt.Sprintf("%d", len(helloWorld)), tokens[7], printLogdata(logdata))

enable logging to stdout for access logs #1683

enable logging to stdout for access logs #1683

Conversation

mjeri commented May 30, 2017

timoreimann left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timoreimann Jun 6, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timoreimann Jun 6, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ldez commented Jun 7, 2017

mjeri commented Jun 7, 2017

ldez left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mjeri commented Jun 7, 2017

ldez left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ldez commented Jun 13, 2017

timoreimann left a comment

Choose a reason for hiding this comment

emilevauge left a comment

Choose a reason for hiding this comment

timoreimann Jun 6, 2017 •

edited

timoreimann Jun 6, 2017 •

edited