Skip to content
This repository has been archived by the owner on Apr 14, 2021. It is now read-only.

WIP: add command to check dynamic library linkage #4765

Merged
merged 5 commits into from Aug 4, 2016

Conversation

mistydemeo
Copy link
Contributor

This new command, linkage, checks for broken dynamic library links in C extensions.

I'd heard there was some interest in adding this functionality, so I thought I'd submit a preliminary PR for discussion. In my experience, broken dylib linkage is a common issue with C extensions, so I think having a good way to diagnose it would be valuable.

This command scans all of the specifications in the bundle for .bundle files in C extensions. If any of the dynamic libraries linked against no longer exist, bundler will report a message to the console and exit non-0.

TODOs:

  • Add support for non-Darwin OSs
  • Improve tests

A few questions:

  • Is there a good way to mock functionality in the tests? Doing it in the standard rspec way isn't working since bundle :command runs in a subprocess. I'd like to be able to stub out stuff that actually checks dylibs and the like.
  • Is making this a new command the right approach? I assumed this wouldn't be ideal to include in, say, check because it would slow it down.

@indirect
Copy link
Member

indirect commented Jul 7, 2016

Whoa, awesome! There is definitely interest in having something like this.

@indirect
Copy link
Member

indirect commented Jul 7, 2016

  1. I would suggest writing a CLI command class that contains all the code, and then unit-test invoking that class. The bundle! :command helper is really only for integration tests, and this command probably only needs a single one to make sure the entry in cli.rb is wired up to the command class.
  2. Maybe the command could be named doctor, and we could try to diagnose other common problems? "You're running Spring, so changes to your Gemfile might not apply until you restart Spring.", that sort of thing.

@mistydemeo
Copy link
Contributor Author

Maybe the command could be named doctor, and we could try to diagnose other common problems?

I think this is a great idea! Pushed a commit to rename it.

Glad to hear there's interest! I'll beef up the tests (thanks for the suggestion), and add support for ldd.

@mistydemeo
Copy link
Contributor Author

Looks like the one failure on Travis is unrelated to my PR? It succeeded on the other jobs.

@segiddins
Copy link
Member

@mistydemeo I restarted that build and it failed again. I'm happy to take a look at that Ruby/RubyGems combination locally if you'd like?

@mistydemeo
Copy link
Contributor Author

Looks like the test is flaky; my File.exist? stub is overaggressive and stomping on other invocations. May be order-dependent. Will fix!

This new command, doctor, checks for common problems.
Currently, it looks for broken dynamic library links in C
extensions.

It scans all of the specifications in the bundle for .bundle files in C
extensions. If any of the dynamic libraries linked against no longer
exist, bundler will report a message to the console.
This will eventually support other tests, so the dylib test needs to be
able to to meaningfully return an empty result
@mistydemeo
Copy link
Contributor Author

Rebased on master, and made some extra changes:

  • Since this will eventually have more checks, not being able to scan deps due to them not being installed is no longer fatal and exits 0.
  • The dylib check now checks for the presence of a working otool or ldd as appropriate, and returns a stub result if they aren't present. Unfortunately, due to limitations of recent OS X versions, this may pop up a GUI window for users who a) have gems with C extensions installed, and b) do not have the CLT or Xcode installed. This should be niche, but could be irritating.
  • Added doctor to the manpage.

Any other changes you'd like to see?

@indirect
Copy link
Member

indirect commented Aug 2, 2016

Hooray! 🎉 🏆 I'm super excited about this, thank you so much!

@homu r+

@homu
Copy link
Contributor

homu commented Aug 2, 2016

📌 Commit 8fa722c has been approved by indirect

@homu
Copy link
Contributor

homu commented Aug 2, 2016

⌛ Testing commit 8fa722c with merge f8c3bde...

homu added a commit that referenced this pull request Aug 2, 2016
WIP: add command to check dynamic library linkage

This new command, linkage, checks for broken dynamic library links in C extensions.

I'd heard there was some interest in adding this functionality, so I thought I'd submit a preliminary PR for discussion. In my experience, broken dylib linkage is a common issue with C extensions, so I think having a good way to diagnose it would be valuable.

This command scans all of the specifications in the bundle for .bundle files in C extensions. If any of the dynamic libraries linked against no longer exist, bundler will report a message to the console and exit non-0.

TODOs:
* Add support for non-Darwin OSs
* Improve tests

A few questions:

* Is there a good way to mock functionality in the tests? Doing it in the standard rspec way isn't working since `bundle :command` runs in a subprocess. I'd like to be able to stub out stuff that actually checks dylibs and the like.
* Is making this a new command the right approach? I assumed this wouldn't be ideal to include in, say, `check` because it would slow it down.
Otherwise, Bundler prints a success message and exits with a status of 0.

The bundle's gem dependencies must all be installed to run this command; if
they are not, Bundler prints an error message and exits with a status of 2.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is no longer correct, I should update these docs. Will pushing a new commit mess with the approval you just granted?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that I have just cancelled the imminent merge of this PR. Go ahead and push another commit, and I'll kick off another build to get a final merge with green tests. Thanks for following up!

@indirect
Copy link
Member

indirect commented Aug 2, 2016

@homu r-

end

def otool_available?
system("otool --version 2>&1 >/dev/null")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can't use /dev/null and need to use Bundler::NULL instead because of Windows

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure thing, though this line is only run on Darwin. Same with ldd_available? below.

@mistydemeo
Copy link
Contributor Author

Updated.

@mistydemeo
Copy link
Contributor Author

mistydemeo commented Aug 4, 2016

This only failed on one job, and it seems unrelated to my PR.

@indirect
Copy link
Member

indirect commented Aug 4, 2016

Sometimes the suite is kind of flaky on old Rubies. :/ hopefully it will pass for merging!

@homu r+

@homu
Copy link
Contributor

homu commented Aug 4, 2016

📌 Commit 6117de9 has been approved by indirect

@homu
Copy link
Contributor

homu commented Aug 4, 2016

⌛ Testing commit 6117de9 with merge e511f9f...

homu added a commit that referenced this pull request Aug 4, 2016
WIP: add command to check dynamic library linkage

This new command, linkage, checks for broken dynamic library links in C extensions.

I'd heard there was some interest in adding this functionality, so I thought I'd submit a preliminary PR for discussion. In my experience, broken dylib linkage is a common issue with C extensions, so I think having a good way to diagnose it would be valuable.

This command scans all of the specifications in the bundle for .bundle files in C extensions. If any of the dynamic libraries linked against no longer exist, bundler will report a message to the console and exit non-0.

TODOs:
* Add support for non-Darwin OSs
* Improve tests

A few questions:

* Is there a good way to mock functionality in the tests? Doing it in the standard rspec way isn't working since `bundle :command` runs in a subprocess. I'd like to be able to stub out stuff that actually checks dylibs and the like.
* Is making this a new command the right approach? I assumed this wouldn't be ideal to include in, say, `check` because it would slow it down.
@homu
Copy link
Contributor

homu commented Aug 4, 2016

💔 Test failed - status

@homu
Copy link
Contributor

homu commented Aug 4, 2016

☀️ Test successful - status

@homu homu merged commit 6117de9 into rubygems:master Aug 4, 2016
@homu homu mentioned this pull request Aug 4, 2016
10 tasks
@mistydemeo mistydemeo deleted the linkage branch August 4, 2016 07:40
@coilysiren coilysiren modified the milestone: Release Archive Sep 22, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants