Stack overflow in unmanaged: IP: 0x5fad4c, fault addr: 0x7f7fc7a89b28

[mrdavidlaing]

From cityindex/csharp-bot-development-flow#8

Running an app under mono 3.0.10 on Ubuntu 12.04 using Tracker functionality from AppMetrics.Client.dll fails with the following

2013-06-26T17:26:27+0000 app.0: Stack overflow in unmanaged: IP: 0x5fad4c, fault addr: 0x7f7fc7a89cc8
2013-06-26T17:26:37+0000 app.0: Stacktrace:
2013-06-26T17:26:37+0000 app.0:
2013-06-26T17:26:37+0000 app.0:
2013-06-26T17:26:37+0000 app.0: Native stacktrace:
2013-06-26T17:26:37+0000 app.0:
2013-06-26T17:26:37+0000 app.0: Stack overflow in unmanaged: IP: 0x5fad4c, fault addr: 0x7f7fc7a89c78
2013-06-26T17:26:37+0000 app.0:         /app/app/runtimes/mono/bin/mono() [0x4a27fd]
2013-06-26T17:26:37+0000 app.0:         /app/app/runtimes/mono/bin/mono() [0x4f718f]
2013-06-26T17:26:37+0000 app.0:         /app/app/runtimes/mono/bin/mono() [0x41ce87]
2013-06-26T17:26:37+0000 app.0:         /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f7fcf302cb0]
2013-06-26T17:26:37+0000 app.0:         [0x7f7fc7a7fce0]
2013-06-26T17:26:37+0000 app.0:
2013-06-26T17:26:37+0000 app.0: Debug info from gdb:

I'll attach a test case showing which functionality causes the issue shortly.

[mrdavidlaing]

Procedure for replicating issue by running sample on Stackato:

1. Make sure you have compiled the issue138-StackOverflowException-on-Mono.sln
2. Make sure your stackato client is pointing at https://api.apps.labs.cityindex.com
3. From the a terminal change to samples/issue138-StackOverflowException-on-Mono/
4. stackato push

[19:41][mrdavidlaing@maccy001:~/Projects/fandrei/AppMetrics/samples/issue138-StackOverflowException-on-Mono(issue138-stackoverflow-exception-in-mono)]$ stackato push
Would you like to deploy from the current directory ?  [Yn]: 
Pushing application 'issue138-StackOverflowException'...
Framework:       buildpack
Runtime:         <framework-specific default>
Application Deployed URL [issue138-StackOverflowException.apps.labs.cityindex.com]: 
Application Url: issue138-stackoverflowexception.apps.labs.cityindex.com
Creating Application [issue138-StackOverflowException] ... OK
Binding service [issue138-StackOverflowException-cache] ... OK
Enter BUILDPACK_URL [git://github.com/mrdavidlaing/stackato-buildpack-mono.git]: 
  Adding Environment Variable [BUILDPACK_URL=git://github.com/mrdavidlaing/stackato-buildpack-mono.git]
Updating environment: OK
Uploading Application [issue138-StackOverflowException] ... 
  Checking for bad links: 5 OK 
  Copying to temp space: 4 OK 
  Checking for available resources:  OK                                        
  Processing resources: OK
  Packing application: OK
  Uploading (447): 100% OK
Push Status: OK
stackato.stager: Staging application 'issue138-StackOverflowException'
staging: 
staging: -----> Stackato receiving staging request
staging: -----> Mono on Stackato app detected
staging: -----> compile params: BUILDPACK_DIR: /staging/staged/buildpack BUILD_DIR: /staging/staged/app CACHE_DIR:/app/fs/issue138-StackOverflowException-cache/staging-assets
staging: -----> Installing Nginx v1.2.6
staging:        Extracting /app/fs/issue138-StackOverflowException-cache/staging-assets/nginx-1.2.6.tar.gz to /staging/staged/app/runtimes/nginx
staging: -----> Installing MONO v3.0.10
staging:        Extracting /app/fs/issue138-StackOverflowException-cache/staging-assets/mono-3.0.10.tar.gz to /staging/staged/app/runtimes/mono
staging: Caching Mozilla certs data
staging:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
staging:                                  Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  688k    0  688k    0     0   406k      0 --:--:--  0:00:01 --:--:--  406k
100 1007k    0 1007k    0     0   516k      0 --:--:--  0:00:01 --:--:--  516k
staging: -----> Copying config templates
staging: -----> Creating bin/ scripts
staging:        Writing setup.sh script
staging:        Writing start.sh script
stackato.stager: Completed staging application 'issue138-StackOverflowException'
app.0: -----> BASE_DIR: /app/app
app.0: -----> Writing config files...
app.0: -----> Ensuring /app/app/public/index.html exists
app.0: -----> Logs are:
app.0: /app/app/logs/consolerunner.log
app.0: /app/app/logs/access.log
app.0: /app/app/logs/error.log
app.0: -----> Cleanup orphaned processes...
app.0: -----> Using LOGLEVEL: ERROR [options: Error (default) | Warning | Info | Verbose | Debug ]
app.0: -----> Setting locale: en_US.UTF-8
app.0: -----> Ensuring Mono has latest SSL root certificates from Mozilla...
app.0: Mozilla Roots Importer - version 3.0.10.0
app.0: Download and import trusted root certificates from Mozilla's MXR.
app.0: Copyright 2002, 2003 Motus Technologies. Copyright 2004-2008 Novell. BSD licensed.
app.0: 
app.0: Importing certificates into user store...
http://issue138-stackoverflowexception.apps.labs.cityindex.com/ deployed to Stackato

1. Check the logs - stackato files app/logs/consolerunner.log

[19:45][mrdavidlaing@maccy001:~/Projects/fandrei/AppMetrics/samples/issue138-StackOverflowException-on-Mono(issue138-stackoverflow-exception-in-mono)]$ stackato files app/logs/consolerunner.log
POST http://foo.com
    AccessKey=accessKey
    MessageAppKey=appKey
    MessagesList=7b620253-609f-4da6-9696-7893b3142903   2013-06-27 18:28:00.7223340 System_OsName   Unix
7b620253-609f-4da6-9696-7893b3142903    2013-06-27 18:28:00.7225300 System_OsVersion    Unix 3.2.0.36
7b620253-609f-4da6-9696-7893b3142903    2013-06-27 18:28:00.7225660 System_ComputerName e59cc5f3511690c718f257ec0efa4aeb
7b620253-609f-4da6-9696-7893b3142903    2013-06-27 18:28:00.7226000 System_UserName stackato
7b620253-609f-4da6-9696-7893b3142903    2013-06-27 18:28:00.7226560 System_ClrVersion   4.0.30319.17020
7b620253-609f-4da6-9696-7893b3142903    2013-06-27 18:28:00.7237080 Exception   System.NotImplementedException: The requested feature is not implemented.\n  at issue138StackOverflowExceptiononMono.Tracker.ReportSystemInfo () [0x00000] in <filename unknown>:0 
7b620253-609f-4da6-9696-7893b3142903    2013-06-27 18:28:00.7237500 Client_WorkingSet   8
7b620253-609f-4da6-9696-7893b3142903    2013-06-27 18:28:00.7239170 Client_PrivateMemorySize    159
7b620253-609f-4da6-9696-7893b3142903    2013-06-27 18:28:00.7239570 Exception   System.NotImplementedException: The requested feature is not implemented.\n  at issue138StackOverflowExceptiononMono.Tracker.ReportPeriodicInfo () [0x00000] in <filename unknown>:0 
7b620253-609f-4da6-9696-7893b3142903    2013-06-27 18:28:00.7252050 Latency Dummy event 0.0003705
7b620253-609f-4da6-9696-7893b3142903    2013-06-27 18:28:00.7252260 Latency Dummy event 2   3.3E-06
...snip...

1. Comment out Tracker, uncomment MonoCompatableTracker and recompile
2. stackato update
3. Fetch logs again: stackato files app/logs/consolerunner.log
   Note the lack of Exceptions.

[mrdavidlaing]

@fandrei, I need some help here - I've discovered an issue in AppMetrics.Client that causes the Tracker to throw System.NotImplementedException exceptions when running on Mono on Unix.

I think that by fixing this issue we will also fix the larger Stack overflow issue.

Does that make sense?

What do you think is the best way to address the System.NotImplementedException issue I've uncovered?

[fandrei]

@mrdavidlaing I already wrote about this bug in my show-and-tell, but forgot to add it to the tracker. It doesn't prevent AppMetrics.Client from functioning properly, and that stack overflow must be caused by something completely different.

[fandrei]

Created issue for NotImplementedException: #139

Fixing stack overflow exception requires more information.

[fandrei]

Added issue for stack overflow exception as well: #140