New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correct framework DWARFs and symbols to workaround broken debugging 馃敟 #924

Closed
NachoSoto opened this Issue Nov 16, 2015 · 46 comments

Comments

Projects
None yet
@NachoSoto
Contributor

NachoSoto commented Nov 16, 2015

This is a follow up to #832 with details on a possible solution.

The story:

We've all been aware since the very beginning that debugging projects that include pre-compiled (Carthage) frameworks has been very broken. As Apple promised that this got better in Xcode 7.2, a few people on #832 did the test (I still can't test it myself because reasons), and realized that it's still just as broken.

Upon further investigation, looks like DWARF files use absolute paths everywhere, based on when they were compiled.

Using dwarfdump:
screen shot 2015-11-16 at 09 37 05
screen shot 2015-11-16 at 09 43 53

As you can see, they're full of paths to the derived data directory of the user who built the framework.
Needless to say, it's not just a problem because we can't share these frameworks with other devs, but also for personal use, given how extremely frequent one must clear derived data to work around Xcode.

what year is it

This is why I finally decided to file a radar, because after two years this is still utterly broken: http://www.openradar.me/23551273.

Of course, I don't expect this to be fixed (ever?), so we need a solution.

Proposed solutions

Abandon Carthage:

We can go back to working like animals, adding all dependencies as submodules. However, this just doesn't cut it, Apple, when will you see this?

  • Swift compilation times are already insanely long. Khan Academy was taking 15 minutes for a clean build before I changed Swift dependencies to pre-compiled frameworks.
  • Nested dependencies: this solution would not be feasible if you have multiple dependencies that depend on a common one. This is one of the reasons why CocoaPods was born:

CocoaPods

(Yeah, I went there, but I wanted to analyze this too)

  • Obviously deals with the dependency graph problem.
  • It's invasive and very complex, which is why Carthage was born.
  • It's still not what we want for development. It's the year 2015, we want to be able to distribute built dependencies and compile our applications in the order of seconds, not minutes.

Work around it in Carthage

I have an idea that might work. Distributing pre-compiled frameworks would still be broken (so --no-use-binaries would have to be the only option), but we'd be able to compile once and forget:

xcodebuild -sdk $SDK -project $PROJECT -configuration Release -scheme $SCHEME ONLY_ACTIVE_ARCH=NO SYMROOT=$DIR/Carthage/Build/$PLATFORM/$FRAMEWORK clean build

Notice the SYMROOT parameter. In my tests compiling with this option correctly sets the absolute paths in the DWARF to that path, instead of derived data! Which means that debugging should hopefully work.

With this, the recommended approach would be to .gitignore Carthage/Build, and using carthage update to build the frameworks.

Once we do that, one can freely clear derived data, and that won't affect the compiled frameworks.

It's still not clear that this is enough, though. It's possible that we'll also need OBJROOT, and putting everything in Carthage/Build:
screen shot 2015-11-16 at 10 25 37


Prolonged sigh.

@NachoSoto NachoSoto added this to the 1.0: Carthaginian Peace milestone Nov 16, 2015

@NachoSoto NachoSoto changed the title from Correct framework DWARFs to workaround broken debugging 馃敟 to Correct framework DWARFs and symbols to workaround broken debugging 馃敟 Nov 16, 2015

@JaviSoto

This comment has been minimized.

Show comment
Hide comment
@JaviSoto

JaviSoto Nov 16, 2015

Contributor

xcodebuild -sdk $SDK -project $PROJECT -configuration Release -scheme $SCHEME ONLY_ACTIVE_ARCH=NO SYMROOT=$DIR/Carthage/Build/$PLATFORM/$FRAMEWORK clean build

This seems like a very sensible approach 馃憤

Contributor

JaviSoto commented Nov 16, 2015

xcodebuild -sdk $SDK -project $PROJECT -configuration Release -scheme $SCHEME ONLY_ACTIVE_ARCH=NO SYMROOT=$DIR/Carthage/Build/$PLATFORM/$FRAMEWORK clean build

This seems like a very sensible approach 馃憤

@alanjrogers

This comment has been minimized.

Show comment
Hide comment
@alanjrogers

alanjrogers Nov 16, 2015

Contributor

Is it not possible to get DWARF files to use relative paths or re-write them in the DWARF file after the fact?

Contributor

alanjrogers commented Nov 16, 2015

Is it not possible to get DWARF files to use relative paths or re-write them in the DWARF file after the fact?

@alanjrogers

This comment has been minimized.

Show comment
Hide comment
@alanjrogers

alanjrogers Nov 16, 2015

Contributor

DBGSourcePath from http://lldb.llvm.org/symbols.html looks promising.

Contributor

alanjrogers commented Nov 16, 2015

DBGSourcePath from http://lldb.llvm.org/symbols.html looks promising.

@NachoSoto

This comment has been minimized.

Show comment
Hide comment
@NachoSoto

NachoSoto Nov 16, 2015

Contributor

Interesting! That could be another option.

Note though that it's possible that in order for debugging to work, we might also need the rest of files in that last screenshot.

Contributor

NachoSoto commented Nov 16, 2015

Interesting! That could be another option.

Note though that it's possible that in order for debugging to work, we might also need the rest of files in that last screenshot.

@mdiep

This comment has been minimized.

Show comment
Hide comment
@mdiep

mdiep Nov 17, 2015

Member

Ugh. Thanks for debugging this.

IIUC setting the SYMROOT root would be similar to using a custom derived data folder (#459). It's not really clear to me if/how those two things would be different. But really the intent is to not tie the derived data of the Carthage-built frameworks to the derived data of the consumer, right?

Is it not possible to get DWARF files to use relative paths or re-write them in the DWARF file after the fact?

This is a pretty interesting idea. Unfortunately I'm not finding any promising information about it.

Member

mdiep commented Nov 17, 2015

Ugh. Thanks for debugging this.

IIUC setting the SYMROOT root would be similar to using a custom derived data folder (#459). It's not really clear to me if/how those two things would be different. But really the intent is to not tie the derived data of the Carthage-built frameworks to the derived data of the consumer, right?

Is it not possible to get DWARF files to use relative paths or re-write them in the DWARF file after the fact?

This is a pretty interesting idea. Unfortunately I'm not finding any promising information about it.

@NachoSoto

This comment has been minimized.

Show comment
Hide comment
@NachoSoto

NachoSoto Nov 17, 2015

Contributor

But really the intent is to not tie the derived data of the Carthage-built frameworks to the derived data of the consumer, right

Right, for 2 reasons:

  • So that projects can keep other relevant files that are necessary for debugging.
  • So that the DWARF and other files point to absolute paths that aren't transient.
Contributor

NachoSoto commented Nov 17, 2015

But really the intent is to not tie the derived data of the Carthage-built frameworks to the derived data of the consumer, right

Right, for 2 reasons:

  • So that projects can keep other relevant files that are necessary for debugging.
  • So that the DWARF and other files point to absolute paths that aren't transient.
@chbeer

This comment has been minimized.

Show comment
Hide comment
@chbeer

chbeer Dec 2, 2015

Any news on this one? Is there any solution/feasible workaround for this? We had to crawl back to Cocoapods in our project because of this 馃槱

chbeer commented Dec 2, 2015

Any news on this one? Is there any solution/feasible workaround for this? We had to crawl back to Cocoapods in our project because of this 馃槱

@mdiep

This comment has been minimized.

Show comment
Hide comment
@mdiep

mdiep Dec 2, 2015

Member

I believe you can work around this by using --no-use-binaries.

Member

mdiep commented Dec 2, 2015

I believe you can work around this by using --no-use-binaries.

@chbeer

This comment has been minimized.

Show comment
Hide comment
@chbeer

chbeer Dec 2, 2015

You can. But that only works as long as you don't delete the DerivedData
directory.

On Wed, Dec 2, 2015 at 4:02 PM, Matt Diephouse notifications@github.com
wrote:

I believe you can work around this by using --no-use-binaries.


Reply to this email directly or view it on GitHub
#924 (comment).

chbeer commented Dec 2, 2015

You can. But that only works as long as you don't delete the DerivedData
directory.

On Wed, Dec 2, 2015 at 4:02 PM, Matt Diephouse notifications@github.com
wrote:

I believe you can work around this by using --no-use-binaries.


Reply to this email directly or view it on GitHub
#924 (comment).

@ratkins

This comment has been minimized.

Show comment
Hide comment
@ratkins

ratkins Dec 10, 2015

Contributor

I'm pretty sure this is the same issue as #785?

Contributor

ratkins commented Dec 10, 2015

I'm pretty sure this is the same issue as #785?

@NachoSoto

This comment has been minimized.

Show comment
Hide comment
@NachoSoto

NachoSoto Dec 10, 2015

Contributor

Yes

Contributor

NachoSoto commented Dec 10, 2015

Yes

@kcharwood

This comment has been minimized.

Show comment
Hide comment
@kcharwood

kcharwood Dec 11, 2015

Just to clarify...

This only affect Swift frameworks? Or does this also affect Objective-C frameworks?

kcharwood commented Dec 11, 2015

Just to clarify...

This only affect Swift frameworks? Or does this also affect Objective-C frameworks?

@ratkins

This comment has been minimized.

Show comment
Hide comment
@ratkins

ratkins Dec 12, 2015

Contributor

Apparently only Swift frameworks. From the Xcode 7.2 release notes: "Frameworks written in Swift should be compiled from source as part of the same project that depends on them to guarantee a single, consistent compilation environment. (22492040)"

This really needs to be noted in the Carthage readme. Why is downloading prebuilt Swift binaries even an option if it demonstrably doesn't work?

Contributor

ratkins commented Dec 12, 2015

Apparently only Swift frameworks. From the Xcode 7.2 release notes: "Frameworks written in Swift should be compiled from source as part of the same project that depends on them to guarantee a single, consistent compilation environment. (22492040)"

This really needs to be noted in the Carthage readme. Why is downloading prebuilt Swift binaries even an option if it demonstrably doesn't work?

@JaviSoto

This comment has been minimized.

Show comment
Hide comment
@JaviSoto

JaviSoto Dec 12, 2015

Contributor

This really needs to be noted in the Carthage readme. Why is downloading prebuilt Swift binaries even an option if it demonstrably doesn't work?

@ratkins when Carthage was conceived, and when that feature was implemented, Apple hadn't documented anywhere that this was broken. Building all of your dependencies from source every single time is a terrible idea, especially considering the long Swift build times. Being able to download pre-built frameworks from Carthage is still desirable, and I hope it will work one day.
That been said, even if you don't download pre-built frameworks, you're going to run into this issue as soon as you clear the derived data directory...

Contributor

JaviSoto commented Dec 12, 2015

This really needs to be noted in the Carthage readme. Why is downloading prebuilt Swift binaries even an option if it demonstrably doesn't work?

@ratkins when Carthage was conceived, and when that feature was implemented, Apple hadn't documented anywhere that this was broken. Building all of your dependencies from source every single time is a terrible idea, especially considering the long Swift build times. Being able to download pre-built frameworks from Carthage is still desirable, and I hope it will work one day.
That been said, even if you don't download pre-built frameworks, you're going to run into this issue as soon as you clear the derived data directory...

@ratkins

This comment has been minimized.

Show comment
Hide comment
@ratkins

ratkins Dec 12, 2015

Contributor

I get that ultimately it's Apple's bug, but I haven't had a working debugger and haven't been able to commission a new build server for six months and I had no idea why. The first sentence of this issue ("We've all been aware since the very beginning that debugging projects that include pre-compiled (Carthage) frameworks has been very broken") makes me punchy because it's demonstrably not true鈥擨 had no idea this was a known issue and it's not mentioned in the readme, which however does describe a default-on feature that doesn't work for Swift projects.

Wishing that a desirable feature works doesn't make it work. Not mentioning it doesn't work when it's known to be the case that it doesn't work wastes a lot of people's time and effort.

Contributor

ratkins commented Dec 12, 2015

I get that ultimately it's Apple's bug, but I haven't had a working debugger and haven't been able to commission a new build server for six months and I had no idea why. The first sentence of this issue ("We've all been aware since the very beginning that debugging projects that include pre-compiled (Carthage) frameworks has been very broken") makes me punchy because it's demonstrably not true鈥擨 had no idea this was a known issue and it's not mentioned in the readme, which however does describe a default-on feature that doesn't work for Swift projects.

Wishing that a desirable feature works doesn't make it work. Not mentioning it doesn't work when it's known to be the case that it doesn't work wastes a lot of people's time and effort.

@NachoSoto

This comment has been minimized.

Show comment
Hide comment
@NachoSoto

NachoSoto Dec 12, 2015

Contributor

@ratkins I don't know if you realize that every single one of us working on Carthage and other open source projects do it for free. The tone of blame with which you're coming across is not well received.

When I said "we've all been aware..." I wasn't implying that we knew from the beginning that it was the fact that they were pre-compiled what was causing the issues. Swift 1.0 was no where near 1.0, so it was hard to know what was going on. The time I put to elaborate my findings in this issue was precisely to alert everyone. You seem to imply that just because nobody (and when I say nobody, I don't mean just contributors) thought to include this in the README is for some other reason than the fact our time is very limited.

The people who have created and worked on Carthage have done that to serve the community. They don't owe anything to anyone. Even less to users who can't appreciate how much work has been put here and in other projects like CocoaPods, precisely to make our jobs easier.

I'm very sorry you didn't find Carthage to your liking. You can help make it better by clicking on the "fork" button. Otherwise, sending messages like that is not helping anyone.

Contributor

NachoSoto commented Dec 12, 2015

@ratkins I don't know if you realize that every single one of us working on Carthage and other open source projects do it for free. The tone of blame with which you're coming across is not well received.

When I said "we've all been aware..." I wasn't implying that we knew from the beginning that it was the fact that they were pre-compiled what was causing the issues. Swift 1.0 was no where near 1.0, so it was hard to know what was going on. The time I put to elaborate my findings in this issue was precisely to alert everyone. You seem to imply that just because nobody (and when I say nobody, I don't mean just contributors) thought to include this in the README is for some other reason than the fact our time is very limited.

The people who have created and worked on Carthage have done that to serve the community. They don't owe anything to anyone. Even less to users who can't appreciate how much work has been put here and in other projects like CocoaPods, precisely to make our jobs easier.

I'm very sorry you didn't find Carthage to your liking. You can help make it better by clicking on the "fork" button. Otherwise, sending messages like that is not helping anyone.

@ratkins

This comment has been minimized.

Show comment
Hide comment
@ratkins

ratkins Dec 12, 2015

Contributor

You're right, I apologise for the blame-y tone鈥擨 do realise nobody owes anyone anything, and I did interpret it as the problem having been known about well before the date on this issue. Penance: #989 (rdar://23551273 duped also.)

Contributor

ratkins commented Dec 12, 2015

You're right, I apologise for the blame-y tone鈥擨 do realise nobody owes anyone anything, and I did interpret it as the problem having been known about well before the date on this issue. Penance: #989 (rdar://23551273 duped also.)

@NachoSoto

This comment has been minimized.

Show comment
Hide comment
@NachoSoto

NachoSoto Jan 27, 2016

Contributor

What do you mean by fixed? There's no mention about this in the release notes. Do you mean that debugging actually works, or that it doesn't crash?
The crash was fixed in beta 1.

Contributor

NachoSoto commented Jan 27, 2016

What do you mean by fixed? There's no mention about this in the release notes. Do you mean that debugging actually works, or that it doesn't crash?
The crash was fixed in beta 1.

@drewcrawford

This comment has been minimized.

Show comment
Hide comment
@drewcrawford

drewcrawford Jan 27, 2016

I'm able to use the debugger in b2, and I think I deleted all the build products.

I'm still waiting for confirmation from a coworker, as there may be a build product on my computer somewhere that I don't know how to delete.

drewcrawford commented Jan 27, 2016

I'm able to use the debugger in b2, and I think I deleted all the build products.

I'm still waiting for confirmation from a coworker, as there may be a build product on my computer somewhere that I don't know how to delete.

@NachoSoto

This comment has been minimized.

Show comment
Hide comment
@NachoSoto

NachoSoto Jan 27, 2016

Contributor

Can you reference symbols from pre-built modules, as well as calling methods?

Contributor

NachoSoto commented Jan 27, 2016

Can you reference symbols from pre-built modules, as well as calling methods?

@drewcrawford

This comment has been minimized.

Show comment
Hide comment
@drewcrawford

drewcrawford Jan 27, 2016

I'm able to call the constructor on a class defined inside a pre-built Swift module at the lldb prompt. I also get autocomplete now in lldb, so that's fun.

Again, it's possible there's still some build product squirreled away on my system, I've made a test case for a coworker and waiting to hear back before I declare the problem solved.

drewcrawford commented Jan 27, 2016

I'm able to call the constructor on a class defined inside a pre-built Swift module at the lldb prompt. I also get autocomplete now in lldb, so that's fun.

Again, it's possible there's still some build product squirreled away on my system, I've made a test case for a coworker and waiting to hear back before I declare the problem solved.

@NachoSoto

This comment has been minimized.

Show comment
Hide comment
@NachoSoto

NachoSoto Jan 27, 2016

Contributor

That's exciting! Looking forward to hearing about your coworker.

Contributor

NachoSoto commented Jan 27, 2016

That's exciting! Looking forward to hearing about your coworker.

@kcharwood

This comment has been minimized.

Show comment
Hide comment
@kcharwood

kcharwood Jan 28, 2016

Waiting with anticipation...

kcharwood commented Jan 28, 2016

Waiting with anticipation...

@mathiasnagler

This comment has been minimized.

Show comment
Hide comment
@mathiasnagler

mathiasnagler commented Jan 28, 2016

Hopefully...

@drewcrawford

This comment has been minimized.

Show comment
Hide comment
@drewcrawford

drewcrawford Jan 29, 2016

Coworker can debug my sample project too.

He doesn't have the sourcecode for the library, so he has never built it on his system.

Sent from my iPhone

On Jan 28, 2016, at 2:40 PM, Mathias Nagler notifications@github.com wrote:

Hopefully...


Reply to this email directly or view it on GitHub.

drewcrawford commented Jan 29, 2016

Coworker can debug my sample project too.

He doesn't have the sourcecode for the library, so he has never built it on his system.

Sent from my iPhone

On Jan 28, 2016, at 2:40 PM, Mathias Nagler notifications@github.com wrote:

Hopefully...


Reply to this email directly or view it on GitHub.

@mdarnall

This comment has been minimized.

Show comment
Hide comment
@mdarnall

mdarnall Feb 3, 2016

Xcode 7.2.1 Released today.

Resolved a debugger crash that could occur in code depending on a binary Swift library or framework

嗖燺嗖

mdarnall commented Feb 3, 2016

Xcode 7.2.1 Released today.

Resolved a debugger crash that could occur in code depending on a binary Swift library or framework

嗖燺嗖

@NachoSoto

This comment has been minimized.

Show comment
Hide comment
@NachoSoto

NachoSoto Feb 3, 2016

Contributor

They probably backported the crash fix to 7.2.x.

Contributor

NachoSoto commented Feb 3, 2016

They probably backported the crash fix to 7.2.x.

@NachoSoto

This comment has been minimized.

Show comment
Hide comment
@NachoSoto

NachoSoto Feb 3, 2016

Contributor

I can confirm: LLDB works MUCH better now thanks to this new "not crashing" feature they've added. Black magic!

I think we should leave this open until we can confirm that all symbols and everything is available for debugging, but I think this is a small victory already :) I hadn't been able to print things in the debugger or step through frames in the stacktrace in over a year...

Contributor

NachoSoto commented Feb 3, 2016

I can confirm: LLDB works MUCH better now thanks to this new "not crashing" feature they've added. Black magic!

I think we should leave this open until we can confirm that all symbols and everything is available for debugging, but I think this is a small victory already :) I hadn't been able to print things in the debugger or step through frames in the stacktrace in over a year...

@kcharwood

This comment has been minimized.

Show comment
Hide comment
@kcharwood

kcharwood Feb 3, 2016

@NachoSoto Does this mean I can have another team member build the swift binary, and I have debug access? Have you validated the paths look good in the DWARFs?

kcharwood commented Feb 3, 2016

@NachoSoto Does this mean I can have another team member build the swift binary, and I have debug access? Have you validated the paths look good in the DWARFs?

@NachoSoto

This comment has been minimized.

Show comment
Hide comment
@NachoSoto

NachoSoto Feb 3, 2016

Contributor

I haven't looked at the DWARFs now, but I doubt anything has changed because .frameworks don't contain anything new. I meant that we can now debug symbols that we have information about without it crashing completely due to other modules being missing. When stepping through a RAC stack trace for example all I could see was assembly, which is 99% times than immediately crashing...

Contributor

NachoSoto commented Feb 3, 2016

I haven't looked at the DWARFs now, but I doubt anything has changed because .frameworks don't contain anything new. I meant that we can now debug symbols that we have information about without it crashing completely due to other modules being missing. When stepping through a RAC stack trace for example all I could see was assembly, which is 99% times than immediately crashing...

@carlossless

This comment has been minimized.

Show comment
Hide comment
@carlossless

carlossless Feb 10, 2016

Resolved a debugger crash that could occur in code depending on a binary Swift library or framework

馃帄 馃檭 馃帄

When stepping through a RAC stack trace for example all I could see was assembly, which is 99% times than immediately crashing...

@NachoSoto I don't quite get this, isn't it the normal behaviour given your using framework binaries?

carlossless commented Feb 10, 2016

Resolved a debugger crash that could occur in code depending on a binary Swift library or framework

馃帄 馃檭 馃帄

When stepping through a RAC stack trace for example all I could see was assembly, which is 99% times than immediately crashing...

@NachoSoto I don't quite get this, isn't it the normal behaviour given your using framework binaries?

@NachoSoto

This comment has been minimized.

Show comment
Hide comment
@NachoSoto

NachoSoto Feb 10, 2016

Contributor

@NachoSoto I don't quite get this, isn't it the normal behaviour given your using framework binaries?

But frameworks can contain debug symbols (DSYMs, etc) so it doesn't have to be that way. In practice, however, this debug information is only stored in derived data so it's completely volatile.

Contributor

NachoSoto commented Feb 10, 2016

@NachoSoto I don't quite get this, isn't it the normal behaviour given your using framework binaries?

But frameworks can contain debug symbols (DSYMs, etc) so it doesn't have to be that way. In practice, however, this debug information is only stored in derived data so it's completely volatile.

@ColdLogical

This comment has been minimized.

Show comment
Hide comment
@ColdLogical

ColdLogical Feb 29, 2016

So does release of 7.2.1 mean we can distribute the .framework binaries to clients without fear of instantly crashing the application when they try to debug?

ColdLogical commented Feb 29, 2016

So does release of 7.2.1 mean we can distribute the .framework binaries to clients without fear of instantly crashing the application when they try to debug?

@maxdesiatov

This comment has been minimized.

Show comment
Hide comment
@maxdesiatov

maxdesiatov Apr 11, 2016

was anyone able to reproduce this with Xcode 7.3?

maxdesiatov commented Apr 11, 2016

was anyone able to reproduce this with Xcode 7.3?

@ikesyo

This comment has been minimized.

Show comment
Hide comment
@ikesyo

ikesyo Aug 3, 2016

Member

This should have been fixed in Xcode 7.3, let's close this.

Member

ikesyo commented Aug 3, 2016

This should have been fixed in Xcode 7.3, let's close this.

@AndrewSB

This comment has been minimized.

Show comment
Hide comment
@AndrewSB

AndrewSB Oct 25, 2016

If I understand this correctly, it looks like crashing on pre-built binaries is no longer an issue. If so, can it be removed from the README https://github.com/Carthage/Carthage#known-issues?

AndrewSB commented Oct 25, 2016

If I understand this correctly, it looks like crashing on pre-built binaries is no longer an issue. If so, can it be removed from the README https://github.com/Carthage/Carthage#known-issues?

@NachoSoto

This comment has been minimized.

Show comment
Hide comment
@NachoSoto

NachoSoto Oct 26, 2016

Contributor

馃憤

Contributor

NachoSoto commented Oct 26, 2016

馃憤

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment