Fixed issue with log multiline log message being broken for multiple CloudWatch Log records #1165
Conversation
normj
force-pushed
the
normj/lambda-runtime-multiline-logging
branch
from
4f70db6 to
c55cd34
Compare
| var stdOutWriter = FileDescriptorLogFactory.GetWriter(fileDescriptorLogId); | ||
| var stdErrorWriter = FileDescriptorLogFactory.GetWriter(fileDescriptorLogId); | ||
| Initialize(stdOutWriter, stdErrorWriter); | ||
| InternalLogger.GetDefaultLogger().LogInformation("Using file descriptor stream writer for logging."); |
There was a problem hiding this comment.
Do this and ConsoleLoggerWriter.cs L53 log line appear same when logged? I believe there should be some prefix to differentiate the log lines emitted from different classes to make debugging easier.
There was a problem hiding this comment.
There is no precedence for adding a prefix when using InternalLogger.GetDefaultLogger(). If we really wanted to do something like that it would be implemented inside InternalLogger.
There was a problem hiding this comment.
It is not blocking but I think worth considering, if two logs are shown same, and dev can't figure the origin, it creates some confusion.
Libraries/src/Amazon.Lambda.RuntimeSupport/Helpers/FileDescriptorLogStream.cs
Outdated
Show resolved
Hide resolved
Libraries/src/Amazon.Lambda.RuntimeSupport/Helpers/FileDescriptorLogStream.cs
Outdated
Show resolved
Hide resolved
Libraries/src/Amazon.Lambda.RuntimeSupport/Helpers/FileDescriptorLogStream.cs
Show resolved
Hide resolved
Libraries/src/Amazon.Lambda.RuntimeSupport/Helpers/FileDescriptorLogStream.cs
Outdated
Show resolved
Hide resolved
Libraries/src/Amazon.Lambda.RuntimeSupport/Helpers/FileDescriptorLogStream.cs
Outdated
Show resolved
Hide resolved
Libraries/src/Amazon.Lambda.RuntimeSupport/Helpers/FileDescriptorLogStream.cs
Outdated
Show resolved
Hide resolved
Libraries/src/Amazon.Lambda.RuntimeSupport/Helpers/FileDescriptorLogStream.cs
Outdated
Show resolved
Hide resolved
Libraries/src/Amazon.Lambda.RuntimeSupport/Helpers/FileDescriptorLogStream.cs
Outdated
Show resolved
Hide resolved
| Array.Reverse(messageLengthBytes); | ||
| } | ||
|
|
||
| var typeAndLength = ArrayPool<byte>.Shared.Rent(8); |
…CloudWatch Log records
normj
force-pushed
the
normj/lambda-runtime-multiline-logging
branch
from
c55cd34 to
aaa032d
Compare
|
Hi, I'm still experiencing this issue on .net 6 lambda runtime. Was the fix released to all AWS environments in all AWS regions? |
|
It will take awhile for it to roll out when the next .NET patch is applied to the managed runtime. I don't know the exact Lambda rollout schedule but you can imagine code changes into Lambda are done very slowly and carefully. If you need this fix today you can follow the pattern of our top-level statement project templates where the version of Amazon.Lambda.RuntimeSupport is included in the deployment bundle. |
|
@normj Hi. I've just checked and still see that the issue is still actual. I tried updating other AWS Nuget packages (Amazon.Lambda.RuntimeSupport, Amazon.Lambda.Core) but with no success - we still have not handled exception logs multilined |
|
Opened a bug to fix that #1265 |
|
Issue still exists |
5 participants
Description of changes:
If log messages contain newlines each line gets broken up over separate CloudWatch Logs. This is particular bad for stack traces. This is due to the runtime using STDOUT and STDERR for logging and the Lambda service doesn't know the start and stop of messages other then using newlines.
To fix this problem the .NET runtime will write log messages to the Lambda telemetry file descriptor if it is available. Otherwise fallback to STDOUT and STDERR. This is similar to the Java Lambda runtime.
Besides confirming all of the existing log tests a new stress test was added to confirm no threading issues and there was no affect to performance.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.