Support for OSX

[felixge]

Thanks for working on this!

I just wanted to try it out, but it seems like OSX is not supported yet?

$ go get -u github.com/derekparker/delve/cmd/dlv
go build github.com/derekparker/delve/proctl: no buildable Go source files in /Users/felix/code/go/src/github.com/derekparker/delve/proctl
# github.com/derekparker/delve/goreadline
38: error: use of undeclared identifier 'rl_catch_sigwinch'
38: error: use of undeclared identifier 'rl_resize_terminal'; did you mean 'rl_reset_terminal'?

[derekparker]

Thanks for checking it out!

You're correct, OSX support is not implemented yet. That is definitely the next major task I'd like to accomplish.

[nkev]

@derekparker I think a lot of people are waiting for OSX support... You have more fans than you know, Derek! :)

[kmatt]

Could readline issues on OSX be addressed with https://github.com/bobappleyard/readline ?

[derekparker]

@datasaur looks promising -- I'll take a look at it and work on integrating it. Thanks for the link.

[kmatt]

IIRC I used it somewhere else with a similar issue - but can't find at the moment.

The other build problem appears to be Linux specific files in proctl/ (?)

[derekparker]

Yeah, and that's not going to be as simple a solution. OS X has limited a lot of what the ptrace family of syscalls can do -- so a lot of Mach kernel specific stuff needs to happen there.

[cryptix]

not sure how fancy your readline has to be but this one is pure go and should cross-compile fine.

[sbinet]

there's also this one: https://github.com/peterh/liner
pure-go as well (using it for my go interpreter)

[damienklinnert]

+1 on this!

[dmitshur]

Not sure if you want +1s, but I would definitely be very interested in OS X support.

[ebfe]

For a small cross platform pure go readline replacement with autocomplete support have a look at golang.org/x/crypto/ssh/terminal.

[gillesdemey]

Patching the readline module is relatively easy, the proctl module is a bit harder.

Many of the syscall functions aren't working.

A simple syscall.PtraceRegs returns

proctl/threads_darwin_amd64.go:20: undefined: syscall.PtraceRegs

[derekparker]

@gillesdemey yes - proctl currently relies heavily on the Ptrace family of syscalls which are extremely limited on Darwin. Most Ptrace specific code will have to be modified to use Darwin specific syscalls for Delve to build on OS X.

[epelc]

+1 also a sublime text plugin would be nice

[derekparker]

To update this issue:

I've been working on isolating any Linux specific code into *_linux.go files and move more generic code into OS agnostic files. There is still some Linux specific code in proctl.go in trapWait, notably syscall.PTRACE_EVENT_CLONE which isn't available on OS X because you cannot set Ptrace options on OS X.

Along the lines of that last sentence, another thing I've been thinking about is: on Linux, notification of new threads can happen through the Ptrace API and new threads will be automatically be traced upon creation. I've been looking for something similar on OS X, maybe a mach port or something that can notify Delve of new threads, but haven't found anything. Mostly it looks like GDB and LLDB poll and check to see if any new threads have been created, and taking action at that point. Maybe somebody more familiar with Mach internals could give some insight into this.

Another interesting challenge is codesigning. I believe Delve will need an info.Plist file compiled into custom section of the binary, and currently the only way to achieve that would be gccgo.

I've been working recently towards OS X support and would love to have it in as soon as possible. If anybody has any insights into the above issues, please feel free to share them.

Lastly, if anybody wants to hack on OS X support, these links should prove helpful:

http://books.google.com/books?id=bzZO64m3iS0C&pg=PA526&lpg=PA526&dq=mach+read+traced+process+memory&source=bl&ots=JC4cYKOeB3&sig=FvRcxjFkb1uW8AVe9vymufbEVlg&hl=en&sa=X&ei=9Lx_VOfOKMSUNtPXgLgK&ved=0CDQQ6AEwAw#v=onepage&q=mach%20read%20traced%20process%20memory&f=false

https://developer.apple.com/library/mac/documentation/Security/Conceptual/authorization_concepts/02authconcepts/authconcepts.html#//apple_ref/doc/uid/TP30000995-CH205-TPXREF20

http://os-tres.net/blog/2010/02/17/mac-os-x-and-task-for-pid-mach-call/

[neelance]

Some time ago I tried to create an interface to lldb on OS X and I had some trouble with the code signing. What I remember is that you can use the codesign tool to add a signature to existing binaries. So I guess you should be able to use normal gc and then add the signature afterwards. Manpage: https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/codesign.1.html

[derekparker]

@neelance that looks interesting, however the issue is still that the codesign tool is going to look in the binary for the Info.plist section. See: https://developer.apple.com/library/mac/documentation/Security/Conceptual/CodeSigningGuide/Procedures/Procedures.html

What caught my eye there was:

5. Add the following arguments to your linker flags:
-sectcreate __TEXT __info_plist Info.plist_path

where Info.plist_path is the complete path of the Info.plist file in your project.

If codesign can take a path to an Info.plist file instead of looking for it in the __info_plist section, that would be great. Then the normal Go toolchain would work fine, and gccGo would not be needed.

[derekparker]

@neelance actually I was wrong(-ish). We can embed the Info.plist into the binary without using gccgo by using cgo and C variable attributes. This will allow us to use the normal Go toolchain + codesign on OS X.

[neelance]

I don't think we have to. The codesign man page says:

     --prefix string
             If no explicit unique identifier is specified (using the -i option), and if the implicitly generated identifier does not contain any dot (.) characters, then the given
             string is prefixed to the identifier before use. If the implicit identifier contains a dot, it is used as-is. Typically, this is used to deal with command tools without
             Info.plists, whose default identifier is simply the command's filename; the conventional prefix used is com.domain. (note that the final dot needs to be explicit).

It seems like that you don't need an Info.plist file or section if you pass the data via command line args. I'll give it a try.

[landaire]

Yeah I was going to say that I don't think GDB has one (I haven't explicitly looked at the code though...) https://sourceware.org/gdb/wiki/BuildingOnDarwin and http://wiki.lazarus.freepascal.org/GDB_on_OS_X_Mavericks_and_Xcode_5#Codesigning_gdb.

[pnasrat]

GDB embeds one in darwin-nat.c I think

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=blob;f=gdb/darwin-nat.c;h=b44dcb4d1dd013245e2cb5ee2fa6f8ef15a84401;hb=HEAD#l154

[derekparker]

@landaire @neelance - @pnasrat is right, it's embedded in gdb/darwin-nat.c, and we can embed it in exactly the same way using C variable attributes which will work with the normal Go toolchain.

[neelance]

I still don't see why we need it. I just tried codesign on one of the binaries created by gc and it worked fine.

[derekparker]

Right, codesign will work without an embedded plist file, however as far as I know, the keys in the Info.plist file we have to embed are used by OS X, like the SecurityFramework (I think) to allow us access to another Mach task.

[landaire]

@neelance did you try it specifically with delve?

As discussed in Code Requirements, the system often uses the Info.plist file of an application bundle to determine the code’s designated requirement. Although single-file tools don’t normally have an Info.plist, you can add one.

And

To see how this works in practice, assume the user has granted permission to the Apple Mail application to access a keychain item. The keychain uses Mail’s designated requirement to identify it: the keychain records the identifier (com.apple.Mail) and the signer of the application (Apple) to identify the program allowed to access the keychain item. Whenever Mail attempts to access this keychain item, the keychain looks at Mail’s signature to make sure that the program has not been corrupted, that the identifier is com.apple.Mail, and that the program was signed by Apple. If everything checks out, the keychain gives Mail access to the keychain item. When Apple issues a new version of Mail, the new version includes a signature, signed by Apple, that identifies the application as com.apple.Mail. Therefore, when the user installs the new version of Mail and it attempts to access the keychain item, the keychain recognizes the updated version as the same program and does not prompt the user for verification.

Source

I think what it's really used for in this context is the SecTaskAccess key: http://os-tres.net/blog/2010/02/17/mac-os-x-and-task-for-pid-mach-call/

This is also present in GDB's Info.plist. Note I haven't looked at delve's source to see what APIs it uses, so I could be wrong.

[neelance]

Yeah seems like you are right. The identifier, signature and requirements are stored by codesign, but the SecTaskAccess key is not. It's only in the Info.plist file, which gets secured by the signature.

[hasjoel]

I found manually codesigning gdb on OSX Yosemite to be painful. Any improvements would be appreciated by many -- I'm sure.

[derekparker]

Update on this issue: The branch darwin-support contains my current efforts towards adding OS X support. OS X support will be my absolute top priority for the time being, with 1 other nagging issue coming in at a close second.

If anyone else happens to be working on OS X support in their own free time, please base your work off of the darwin-support branch, and post any issues or arch-specific code design questions here.

[theprojectabot]

keep rocking it derek

[derekparker]

Yet another update:

I now have Delve compiling in OS X using a bunch of unimplemented stub functions. The good news is that it compiles, which means all Linux specific stuff has been sorted out and moved to the correct place. The next step is to fill in the stub functions, most of which I already have a clear idea of what the Linux -> Mach translation looks like.

Hoping to have all stubs implemented and all tests passing as soon as possible. The other nagging issue has been resolved, so basically 100% of my efforts are focused of OS X support at the moment.

[julien]

👍 great

[pavelnikolov]

You deserve a beer!

[hasjoel]

👍 from the 🔢

[theprojectabot]

YESSSS!

[derekparker]

I have just merged initial support for OS X into master. I plan on making an actual announcement tomorrow, but I'd like to give a chance for anybody watching this thread to check it out and flush out any major issues before then, if there is any.

Please note that Delve is still in beta, so there may well be general issues, however the major thing here is OS X support.

[nowk]

@derekparker Awesome stuff, Thank you!

---

Had a small issue with the make install command, $(which dlv) is return blank, but this could just be me. $(shell which dlv) resolved that for me.

[dmitshur]

This is great, thank you!

[derekparker]

@nowk thanks, fixed.

[grandmore]

looking forward to it.

[RaananHadar]

great job derek, i'll be sure to try it.

[hasjoel]

Wow, I can't wait to give this a shot. :)