activation logs squashed well before 10MB, and with no truncation warning

[starpit]

the openwhisk documentation states that, when logs are truncated, a truncation message will appear on the last line.

i am seeing truncation well in advance of my specified log limit (10MB), and with no truncation suffix message.

wsk activation logs e4c655fd13b6409690a593165647aaec  | wc -c | awk '{print $1/1024/1024 "MB"}'
0.343715MB

and no truncation marker at the end of the logs:

wsk activation logs e4c655fd13b6409690a593165647aaec  | tail -1
2017-02-28T03:05:16.650909172Z stdout: 03:04:28.662 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.RepositoryChainComponentMetaDataResolver] Using org.eclipse.jetty:jetty-websocket:8.1.15.v20140411 from Maven repository 'MavenRepo'

[rabbah]

@cbickel @markusthoemmes FYI.

[rabbah]

I initially thought this was a bug but was fooled by the pre-existing unit test (I had changed the limit in the action limit test from 0 to 10 and expecting it to pass but it failed - for several reasons which I addressed in #1924).

I think we need to dig into this particular activation - is it possible the action aborted abnormally?

[starpit]

this is indeed a failing activation, but the logs end short of the errors i get when i don't pass the --debug flag to gradle (i'm using gradle to run selenium in whisk)

[perryibm]

Failing due to memory usage or something else? Does it always truncate at or near the same spot?

[starpit]

the action exits "normally", at least in that i see the expected log output... unless i add the --debug flag to the gradle command line. i suppose it's possible that --debug results in memory exhausion. i'm running with 512M.

when i stick an "echo foo" as the last line of my action, under the assumption that gradle is failing, rather than logs failing, ... i don't see foo at the end. so it still seems like logs are disappearing.

also, in the curtailed-logs cases, i don't see the ultimate XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX

oh: somehow randomly this one executed to "completion" (meaning i see the XXX suffix to the log). i didn't do anything differently: 9ec68828f20048a59784a4d899ea8b88

[rabbah]

Going to close as inactionable.

1. We've added a detection mechanism to report OOM.
2. There may be something lingering here - related to The logs in activation is not complete #2548.
3. The logging is getting overhauled in Streamingly read user-logs. #2878