The sos "ClrStack" command now works #437

mikem8361 · 2015-03-12T23:49:22Z

Enabled all the sos source necessary to allow ClrStack to run. Had to change how the PAL layer implemented GetCurrentThreadId() to use gettid() instead of pthread_self() so the DAC can find the vm's thread object. The tid's are the only thing available on the lldb side. OSx still uses pthread_self() and will need some future work to get ClrStack working. Fixed the PAL vsnprintf functions to properly handle the "%_._p" format string. The address in the ClrStack (and most other commands) are printed this way.

@jkotas could you review? And @sergiy-k if you get some time.

…mand now builds and almost runs. There is a problem with thread ids between the debugger/DAC and the target/VM/PAL.

…ilds and almost runs except for a thread id problem.

…related to %p). Needed to change how the PAL got the thread id (pthread_self to gettid) and implemented a simple GetThreadContext function.

…pported under windbg/windows runtime. Didn't actually support "*" for the width parameter at all for any format char.

kangaroo · 2015-03-13T06:35:25Z

@mikem8361 Do you know what lldb uses on OS X? Its probably the mach port. If not I can go dig it up.

mikem8361 · 2015-03-13T16:48:01Z

I don't know what is used for the thread id on OSx. I probably won't have time to look into OSx until later next week or so.

Sent from Windows Mail

From: Geoff Nortonmailto:notifications@github.com
Sent: ?Thursday?, ?March? ?12?, ?2015 ?11?:?35? ?PM
To: dotnet/coreclrmailto:coreclr@noreply.github.com
Cc: Michael J McLaughlinmailto:mikem@microsoft.com

@mikem8361 https://github.com/mikem8361 Do you know what lldb uses on OS X? Its probably the mach port. If not I can go dig it up.

Reply to this email directly or view it on GitHubhttps://github.com//pull/437#issuecomment-78832069.

mikem8361 · 2015-03-13T19:18:15Z

@kangaroo It would be great if you could dig any info on lldb thread ids for OSx. Thanks.

mikem8361 · 2015-03-13T19:18:53Z

@jkotas could you review? @sergiy-k if you have time.

kangaroo · 2015-03-13T19:29:30Z

@mikem8361 I'll spend some cycles on it this weekend. Can you provide a testcase and expected behavior so I can confirm my fixes?

mikem8361 · 2015-03-13T21:02:30Z

Loading the sos plugin and running the ClrStack command will test this. The directions below are obviously for Linux and will have to be adjusted for OSx.

I would like to know how you install lldb and build the plugin for OSx after Pat Gavlin’s cmake changes? Do we still need to clone and build lldb? Thanks.

mikem

I will put these directions on our “wiki” as soon as I figure out how (Matt?).

First we need to install and build with lldb 3.6 instead of 3.5 (because it has problem debugging corerun/coreclr):

sudo echo "deb http://llvm.org/apt/trusty/ llvm-toolchain-trusty-3.6 main" > /etc/apt/sources.list.d/llvm.list
sudo wget -O - http://llvm.org/apt/llvm-snapshot.gpg.key|sudo apt-key add -
sudo apt-get update
sudo apt-get install lldb-3.6 lldb-3.6-dev

Now set the LD_LIBRARY_PATH environment variable to the path of the binaries. BUILD_DIR is just for convenience.

export BUILD_DIR=/home/mikem/coreclr/binaries/Product/linux.x64.debug
export LD_LIBRARY_PATH=$BUILD_DIR

Start lldb like this:

lldb-3.6 -o "plugin load %BUILD_DIR/linux.x64.debug /libsosplugin.so" $BUILD_DIR/corerun $BUILD_DIR/HelloTux.exe

After the module coreclr is loaded/started/initialized, execute the sos commands like this (the case is significant):

(lldb) sos IP2MD 0x00007ffff231c90e
MethodDesc: 00007ffff1eb9b98
Method Name: System.Lazy1[[System.__Canon, mscorlib]].LazyInitValue() Class: 00007ffff1ed0ff8 MethodTable: 00007ffff1eb9c60 mdToken: 000000000600090C Module: 00007ffff7fec020 IsJitted: yes CodeAddr: 00007ffff231c720 Transparency: Not calculated (lldb) sos ClrStack OS Thread Id: 0x56fb (1) Child SP IP Call Site 00007FFFFFFFCD18 00007FFFF6A140BD [PrestubMethodFrame: 00007fffffffcd18] Interop.getenv_core(System.String) 00007FFFFFFFCE70 00007FFFF1EE3FE1 Interop.getenv(System.String) 00007FFFFFFFCEB0 00007FFFF1EE3F3E System.ConsolePal+TermInfo+Database.ReadDatabase() 00007FFFFFFFCEE0 00007FFFF1EE3ABD System.ConsolePal+TermInfo+Database.<.cctor>b__2() 00007FFFFFFFCF00 00007FFFF231CCBC System.Lazy1[[System.__Canon, mscorlib]].CreateValue()
00007FFFFFFFCF50 00007FFFF231C90E System.Lazy1[[System.__Canon, mscorlib]].LazyInitValue() 00007FFFFFFFCFB0 00007FFFF1EE3901 System.ConsolePal+TermInfo+Database.get_Instance() 00007FFFFFFFCFD0 00007FFFF1EE3676 System.ConsolePal+TerminalColorInfo.<.cctor>b__0() 00007FFFFFFFD0C0 00007FFFF1EE33B3 System.Lazy1[[System.ConsolePal+TerminalColorInfo, System.Console]].CreateValue()
00007FFFFFFFD160 00007FFFF1EE30B0 System.Lazy`1[[System.ConsolePal+TerminalColorInfo, System.Console]].LazyInitValue()
00007FFFFFFFD1B0 00007FFFF1EE2B9C System.ConsolePal+TerminalColorInfo.get_Instance()
00007FFFFFFFD220 00007FFFF1EE2B00 System.ConsolePal.set_ForegroundColor(System.ConsoleColor)
00007FFFFFFFD260 00007FFFF1EE269A System.Console.set_ForegroundColor(System.ConsoleColor)
00007FFFFFFFD280 00007FFFF1EE248E Program.Main()
00007FFFFFFFD668 00007FFFF65FD4F7 [GCFrame: 00007fffffffd668]
00007FFFFFFFDC90 00007FFFF65FD4F7 GCFrame: 00007fffffffdc90

You can create an alias to a simpler command only after the plugin is loaded:

command alias clrstack sos ClrStack
(lldb) clrstack
…

mikem

From: Geoff Norton [mailto:notifications@github.com]
Sent: Friday, March 13, 2015 12:30 PM
To: dotnet/coreclr
Cc: Mike McLaughlin
Subject: Re: [coreclr] The sos "ClrStack" command now works (#437)

@mikem8361 https://github.com/mikem8361 I'll spend some cycles on it this weekend. Can you provide a testcase and expected behavior so I can confirm my fixes?

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/437#issuecomment-79283332.

kangaroo · 2015-03-13T21:42:01Z

@mikem8361 I have it working on OS X:

(lldb) plugin load libsosplugin.dylib
(lldb) r
Process 41183 launched: './corerun' (x86_64)
Process 41183 stopped
* thread #1: tid = 0x3e8033, 0x000000011c5d649a, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
    frame #0: 0x000000011c5d649a
-> 0x11c5d649a:  divss  0x16(%rip), %xmm0
   0x11c5d64a2:  incl   %eax
   0x11c5d64a4:  cmpl   $0x400, %eax
   0x11c5d64a9:  jl     0x11c5d648a
(lldb) sos ClrStack
OS Thread Id: 0x3e8033 (1)
        Child SP               IP Call Site
00007FFF5FBFE1D8 000000011C5D649A Driver.Main(System.String[])
00007FFF5FBFE5B8 0000000101684F14 [GCFrame: 00007fff5fbfe5b8] 
00007FFF5FBFEBE0 0000000101684F14 [GCFrame: 00007fff5fbfebe0] 
(lldb)

Do you want to wait for review and squash before bringing in my commit?

As for instructions, the LLDB build is currently broken on OS X (it'll be in my PR to fix it), but you do need to check out LLDB from source, then you do the following prior to build:

export LLDB_LIB_DIR=/Library/Developer/CommandLineTools/Library/PrivateFrameworks/
export LLDB_INCLUDE_DIR=/path/to/checkout/of/lldb/include/

… which matches the tid from lldb

kangaroo · 2015-03-13T21:46:24Z

BTW, my patches are trivial, but here: https://github.com/kangaroo/coreclr/tree/clrstack

mikem8361 · 2015-03-13T22:01:57Z

Go ahead and create a PR for my fork.

From: Geoff Norton [mailto:notifications@github.com]
Sent: Friday, March 13, 2015 2:43 PM
To: dotnet/coreclr
Cc: Mike McLaughlin
Subject: Re: [coreclr] The sos "ClrStack" command now works (#437)

@mikem8361 https://github.com/mikem8361 I have it working on OS X:

(lldb) plugin load libsosplugin.dylib

(lldb) r

Process 41183 launched: './corerun' (x86_64)

Process 41183 stopped

thread Merge changes from TFS #1: tid = 0x3e8033, 0x000000011c5d649a, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP

frame #0: 0x000000011c5d649a

-> 0x11c5d649a: divss 0x16(%rip), %xmm0

0x11c5d64a2: incl %eax

0x11c5d64a4: cmpl $0x400, %eax

0x11c5d64a9: jl 0x11c5d648a

(lldb) sos ClrStack

OS Thread Id: 0x3e8033 (1)

    Child SP               IP Call Site

00007FFF5FBFE1D8 000000011C5D649A Driver.Main(System.String[])

00007FFF5FBFE5B8 0000000101684F14 [GCFrame: 00007fff5fbfe5b8]

00007FFF5FBFEBE0 0000000101684F14 [GCFrame: 00007fff5fbfebe0]

(lldb)

Do you want to wait for review and squash before bringing in my commit?

As for instructions, the LLDB build is currently broken on OS X (it'll be in my PR to fix it), but you do need to check out LLDB from source, then you do the following prior to build:

export LLDB_LIB_DIR=/Library/Developer/CommandLineTools/Library/PrivateFrameworks/

export LLDB_INCLUDE_DIR=/path/to/checkout/of/lldb/include/

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/437#issuecomment-79439477.

kangaroo · 2015-03-13T22:08:18Z

Done

Fixes OSx problems.

jkotas · 2015-03-13T22:37:40Z

src/ToolBox/SOS/Strike/sos.cpp

@@ -150,7 +150,7 @@ namespace sos


        if (mTypeName == NULL)
-            return L"<error>";
+            return (wchar_t *)W("<error>");


It may be to change the return type to const wchar_t * instead of adding this cast.

jkotas · 2015-03-13T22:51:32Z

LGTM

The sos "ClrStack" command now works

mikem8361 added 6 commits March 10, 2015 15:50

The "ClrStack" command and related functions now build.

5310bc8

Fix some missing variables and other build problems. The ClrStack com…

c5b03de

…mand now builds and almost runs. There is a problem with thread ids between the debugger/DAC and the target/VM/PAL.

Fix some more build problems/missing globals. The ClrStack command bu…

a9958b9

…ilds and almost runs except for a thread id problem.

The ClrStack command mostly works except for some address formating (…

0681995

…related to %p). Needed to change how the PAL got the thread id (pthread_self to gettid) and implemented a simple GetThreadContext function.

Fix PAL printf functions. Didn't handle "%*.*p" correctly which is su…

ce7dd7e

…pported under windbg/windows runtime. Didn't actually support "*" for the width parameter at all for any format char.

Merge remote-tracking branch 'upstream/master' into clrstack

6c5888e

dnfclas added the cla-not-required label Mar 12, 2015

Fix CI build failures. Undefined functions.

7f238c9

kangaroo added 2 commits March 13, 2015 14:45

Ensure we find_library on LLDB as well to find LLDB.framework on OSX

2d384e4

Change THREADSilentGetCurrentThreadId to return the pthread_threadid,…

d6fb741

… which matches the tid from lldb

Merge pull request #4 from kangaroo/clrstack

bcdca17

Fixes OSx problems.

jkotas reviewed Mar 13, 2015
View reviewed changes

Code review feedback.

e33ab2f

mikem8361 added a commit that referenced this pull request Mar 14, 2015

Merge pull request #437 from mikem8361/clrstack

39b7e46

The sos "ClrStack" command now works

mikem8361 merged commit 39b7e46 into dotnet:master Mar 14, 2015

mikem8361 deleted the clrstack branch March 14, 2015 00:03

The sos "ClrStack" command now works #437

The sos "ClrStack" command now works #437

mikem8361 commented Mar 12, 2015

kangaroo commented Mar 13, 2015

mikem8361 commented Mar 13, 2015

mikem8361 commented Mar 13, 2015

mikem8361 commented Mar 13, 2015

kangaroo commented Mar 13, 2015

mikem8361 commented Mar 13, 2015

kangaroo commented Mar 13, 2015

kangaroo commented Mar 13, 2015

mikem8361 commented Mar 13, 2015

kangaroo commented Mar 13, 2015

jkotas Mar 13, 2015

jkotas commented Mar 13, 2015

The sos "ClrStack" command now works #437

The sos "ClrStack" command now works #437

Conversation

mikem8361 commented Mar 12, 2015

kangaroo commented Mar 13, 2015

mikem8361 commented Mar 13, 2015

mikem8361 commented Mar 13, 2015

mikem8361 commented Mar 13, 2015

kangaroo commented Mar 13, 2015

mikem8361 commented Mar 13, 2015

kangaroo commented Mar 13, 2015

kangaroo commented Mar 13, 2015

mikem8361 commented Mar 13, 2015

kangaroo commented Mar 13, 2015

jkotas Mar 13, 2015

Choose a reason for hiding this comment

jkotas commented Mar 13, 2015