Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Please deprecate (and/or rename) `add_development_dependency` #1104
The main reasons:
Summary: gem development and testing is not the responsibility of RubyGems - and should never be, because otherwise it's just wasting precious developer hours.
I've never heard this complaint before. Can you link me to similar complaints?
If you are developing for gem a you do not need development dependencies for gem b. I don't understand what --development-all would fix here. I guess this ties into complaint number 1 or is a bug in RubyGems. Can you clarify?
Bundler's groups don't get exported as metadata on gem servers so they can't supersede the Gem::Specification.
I see this as an argument for adding more dependency types, not reducing the existing number of types.
Do you mean "development dependencies should be listed in the Gemfile, runtime dependencies should be listed in the specification"?
If so this violates the DRY principle and I am highly unlikely to make such a change to RubyGems.
I haven't seen any complains about this. Can you clarify?
I don't see this as compelling, it is not the only context that RubyGems must consider. (See also my response about DRY?)
What do you mean by "too much stuff"?
Note that RubyGems relatively recently added a generic metadata hash to allow authors to be more descriptive with the content in their specifications, so the trend is to add more features to a specification, not less.
RubyGems does not download the full specification when determining which gem to install and instead downloads only a subset of the information available. The full specification is compressed inside the gem and typically consumes 512KB (a single tar block).
I don't understand how this applies to development dependencies. See also the following
How is #gemspec insufficient? This seems to be the best way for using bundler along with a Gemfile:
You have not yet convinced me it is the right direction.
This would be a breaking API change that could not be introduced until RubyGems 3. There is no schedule for such a release.
You'll find many gems with incorrectly/incompletely specified dev deps:
$ gem install cucumber --dev Fetching: builder-3.2.2.gem (100%) (...) Fetching: cucumber-1.3.17.gem (100%) ERROR: Error installing cucumber: cucumber requires diff-lcs (>= 1.1.3)
gem install --dev nenv Fetching: diff-lcs-1.2.5.gem (100%) Successfully installed diff-lcs-1.2.5 Fetching: rspec-3.1.0.gem (100%) ERROR: Error installing nenv: rspec requires rspec-core (~> 3.1.0)
Note how ridiculous it is for a developer to include all dependencies of rspec in their gemspec (or be expected to already have them installed).
$ gem install --dev aruba Fetching: builder-3.2.2.gem (100%) (...) Fetching: cucumber-1.3.17.gem (100%) ERROR: Error installing aruba: cucumber requires gherkin (~> 2.12)
$ gem install --development-all rspec ERROR: While executing gem ... (Gem::DependencyResolutionError) conflicting dependencies minitest (~> 5.3, development) and minitest (~> 5.2, development) Activated minitest-5.2.0 which does not match conflicting dependency (~> 5.3) Conflicting dependency chains: rspec (>= 0), 3.1.0 activated, depends on rspec-mocks (~> 3.1.0), 3.1.0 activated, depends on minitest (~> 5.2, development), 5.2.0 activated versus: rspec (>= 0), 3.1.0 activated, depends on rspec-core (~> 3.1.0), 3.1.0 activated, depends on minitest (~> 5.3, development)
(This one actually baffles me to no end)
All 4 examples suggest incorrect usage of the 'development gem' in the gemspec.
Can you point me to a gem that uses
Of course, the above gems rely on bundler resolving the dependencies.
No one is probably using
Some people define
So the usage is obviously incorrect - but the problem is mitigated by using Bundler.
Exactly. My point being: there are only two options for dealing with dev deps in
I don't even understand what the use case is for that's still relevant nowadays.
It's the only other command (AFAIK) that makes uses of the 'add_development_dependency' entries.
You're right - and while it would do what
Yes. As far as I understand,
So it's a problem either way:
What's the point of exporting development deps as metadata? I mean practically?
Runtime deps and everything else in the specification - no argument there.
How is that information relevant? It isn't complete.
Take RSpec for example: https://rubygems.org/gems/rspec
No deps listed, but even with --development-all can't be installed.
So we have 4 types of dependencies:
Nowadays, tests have become integral to development so much, that they are often more important than developer tools.
Test dependencies and dev dependencies are semantically different (ask any dev op).
It doesn't make sense to install Pry on a CI (where there's no terminal anyway), while doing so may break the build for unecessary reasons (I won't even mention platform deps). With JRuby and Rubinius you likely even won't be able to install the dev tools - no matter how precisely they are defined in the gemspec.
If the gemspec had "add_deployment_dependency" and "add_test_dependency" I wouldn't argue - because even as metadata it could be used.
I may be able to install test deps on CI on every platform, but maybe only one platform supports deployment.
I agree the minimalism is useful - which is precisely why the name "development" defined on RubyGems page as "typically test frameworks and build systems" is confusing. It's not uncommon to see people adding Pry as a "development" dependency in their gemspec.
It's so confusing, deprecation is the only thing I can think of.
Personally, I'd love to be able to do
Technically, to match the def on RubyGems, for my needs the method should be
gemspec development_group: :test gem 'pry', group: :development
And on CI do
The problem is naming: I need to map
Because for me "development" and "test" are 2 different types of gems (like Pry vs Rspec).
That would be perfect, yes. I can use the
I can make
It's the opposite - this actually helps people make things DRYer, because often they put in a
This is a clean separation of responsibilities - there's nothing "duplicated" to break the DRY principle.
This may just be a conflict of definitions for the word "development". For me, this is sufficient to deprecate the method.
For me, the gemspec dev deps are VERY useful if I use them to define gems for the CI builds (because I prefer them in the gemspec, while Gemfile is much better suited for "real" development deps).
The problem with things like
Bundler is better suited for this, because it's state is not version specific (that's what the
In development mode, there's no version. (and so, e.g. it doesn't make sense to build a gem in order to run unit tests on it).
I haven't seen any complains about this. Can you clarify?
I've made the mistake of adding development deps to runtime deps and vice versa.
Somehow the lines:
are just too easy to confuse. For me, the gemspec is much more important than the Gemspec because it affects users. As for the Gemspec I can experiment without breaking anything - which makes it perfect for trying out all sorts of development "conveniences".
I don't see this as compelling, it is not the only context that RubyGems must consider. (See also my response about DRY?)
Yes - and that's why I'm suggesting this context can be dropped in favor of putting focus on more practical issues.
Compared to bundler:
I'm not against adding features at all. Far from it. In fact, adding features usually means needs are changing. So if needs are changing, it makes sense that some thing are no longer relevant, necessary, or even useful.
So deprecations are not only unsurprising, they are to be expected.
And the deprecation I'm suggesting reflects a change of times - where gems have more dependencies than ever (through breaking up responsibilities into smaller gems), development tools are getting more advanced (Pry, rspec) and the productivity of the bleeding edge is prefered over stability (so pulling from git makes more sense than from rubygems), CI builds are more frequend and more demanding (so separating dev gemsets from test gemsets becomes more important).
As a sidenote I recently had a problem installing an older gem. A tiny gem installed for minutes and the
Turned out the gems used the
I wish that was deprecated and I had seen a message sooner.
By "too much" I mean visually and for editing. As a developer I care more about adding gem deps in the right place - I don't edit the metadata that often.
There's a much higher cognitive load while working with gemspec files. I usually just want to "fill it out" perfectly the first time and never work on the file ever again - that's probably also why I'd prefer to put the development deps in the Gemfile.
It's not uncommon to switch to non-rubygems sources for dependencies (
Also, it depends on how the word "development" is defined. The default Rails distinction of
That's why I'm suggesting the deprecation. Rails has changed how people think about the terms since the original specification field was added.
Yes, unfortunately the typical Gemfile with gemspec statement includes "gemspec dev deps" into Bundler's "development" group, and also typically, people expect to install on the CI with the
Symptom: if you add
The workaround people resort to: adding
I don't know how to 'grep' the most popular gems, but I'm pretty sure there are lots of popular gems getting this wrong because of the confusion between the terms "development" and "development".
I hope I've shed some light and shared my views clearly enough.
If not, feel free to ask further question or for more clarity.
I see 3 "solutions", but none of them are "obvious" (both are actually counter-intuitive in some way not fully documented):
You could blame it on incorrect/bad Travis docs, incorrect/bad Bundle defaults, but I think you'd agree that RubyGems was never designed to perfectly fit into this kind of workflow.
According to SemVer deprecating is a minor version number change, which SHOULD happen before a major number change so that people can gradually migrate (check semver.org references to "deprecated"). So if this was to be introduced, it would HAVE to be either in 2.x or no earlier than 3.1.
So if this would be deprecated in 2.x, it could be removed completely in 3.0
With respect to cucumber, etc., it seems that is a bug in RubyGems which should be fixed, not a reason to remove development dependencies. I thought the tests covered such cases but obviously they are inadequate.
For the cucumber case, diff-lcs is a regular dependency, not a development dependency and is not installed, thus the failure. I suspect the others are the same.
With respect to deployment I was unclear. Users do not want development dependencies installed along with a library they need in a deployed application. This is why multiple types of dependencies were added in the first place.
So far there has only been a handful of requests for multiple dependency types besides "runtime" and "development" which in RubyGems covers all development tasks including test-only environments and handy developer tools. RubyGems makes no distinction here.
You misunderstand the DRY principle which is “Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.” (See The Pragmatic Programmer). For a library the gem specification is the single unambiguous authoritative representation of library metadata which includes dependencies.
Note that this is specific to ruby software released as gems and used as a library. For software that is used as an application there is no need for a gem specification thus the Gemfile.lock is where dependency metadata is stored. (Not in the Gemfile as a Gemfile alone does not allow consistent deployments. See http://yehudakatz.com/2010/12/16/clarifying-the-roles-of-the-gemspec-and-gemfile/)
Bundler cannot be used to exclude dependencies of gems it installs. If a gem depends on ronn as a runtime dependency then the platform your are installing on must support it. RubyGems does not support optional or platform-specific dependencies in this manner.
I see in your response that we could do a better job at documentation and could make some improvements around declaring dependencies but I don't see a compelling reason to remove development dependencies.
Overall it seems you have a clear distinction between what RubyGems considers a runtime and development dependency.
That you sometimes remove a dependency from the wrong place is not something that I see removing development dependencies solving. You don't seem to mind the idea of adding additional dependency types which is also something that would prevent you from removing a dependency from the wrong set.
As a gem author it is up to you to take care to ensure that the software you release is fully correct. RubyGems has continually added checks to help authors release gems that have minimal problems, but there is a limit to what RubyGems can check. Ensuring that your dependencies are properly specified in your gem specification is one of those things that is difficult for RubyGems to check statically.
PS: I find bundler entirely inadequate as a tool for developing gems. It requires you to rebuild test suites, documentation generation, release process, correctness checking, etc. for every library you write. I use hoe which has built-in tasks or reusable plugins for all these features. For application deployment it is an acceptable tool.
My point being: no one actually cares it's even broken.
Seriously, no one cares! You're time is EXTREMELY precious given the role you are playing in the Ruby world. Instead of working of fixing something no one cares about, I believe it would be better to show a deprecation with a url, and have a clever description protecting you of any "accusations".
And CI's are also better off also not having those installed. The problem is dev and test share more deps than runtime and dev do. Since CIs have replaced testers, CI don't require "human tools" (like PRy). And CIs have become "first class citizens" in typical development workflow. Obviously, they can't cry out and fight for their rights to "better working conditions" with "less effort for needless installs of human-only dependencies".
Neither does bundler - to the point that bundler resolves and installs EVERYTHING - except when the
Also, many dev tools "essential" for human being may not even be gems in the first place (external repos, tarballs). I don't see a reason why RubyGems should even care.
First, thanks for the quote - it's much better defined than what I had in mind.
And it also just solidifies my point. Simply because runtime dependencies are obviously needed by RubyGems, there's no need to force developers to "log" which tools they used in RubyGems as well (because that's what defining development dependencies is - just developers declaring what they've used for development).
In short: development dependencies and runtime dependencies together are NOT - by any stretch - "a single piece of knowledge". They are two unrealated pieces artificially "mashed" together inside a single gemspec. And test deps included with development deps don't represent a "single piece of knowledge" either. (Any reasonable dev op would confirm).
I believe it would be better to even have a separate file for development dependencies - and (if at all) just have a reference to it in the Gemfile. Because it would be rarely needed by RubyGems (if at all).
Currently, the existence of the add_development_dependencies method "inspires" people to unnecessarily fill it out - "just because it's there". Bundler has a rake task for installing dependencies (
I am constantly aware of this - I'm actually trying hard not to let conventions from the "Rails world" cloud how I'm thinking about this subject (because the rationale there is very different).
Then again gems and apps are getting a lot more similar in terms of the complexity of the development workflow (deploying to hosting provider is not much different from deploying a release to RubyGems.
I see this as yet another reason to delegate the definitions to the Gemfile. The only reason I can think of to not put them in the Gemfile is: if Gemfiles (and Gemfile.locks) are ultimately going to be deprecated in place of gemspecs. Because why let dev dependencies be specified BOTH in the Gemfile and gemspec - especially since now RubyGems uses Gemfile.lock for dependencies
Actually, that's the first decent article I came across when I was searching for "good practices" about defining dev deps in gemspecs. And partially the reason I brought this up. Development deps for gems weren't expanded on in the article - probably because they are useless or irrelevant anyway (except when things fail when combined with
I was talking about development dependencies - ronn is used for generating man pages for gems, but it can't be installed on JRuby (I think it was because of rdiscont). So it's a good example of a development dependency essential for updating docs, but not e.g. for fixing bugs. So if a developer is fixing JRuby-specific bugs, it doesn't make sense for installing dev gems to fial simply because it's listed as a dev dependency.
It's unclear if dev deps in the gemspec are meant to be "optional", "suggested" or "essential". So if development deps were meant to only be in Gemfiles, I'd simply have a
If dev deps are meant to be correct, they should be checked (during
I don't see a practical reason for development dependencies to exist. I mean if the field became a no-op today, what exactly would that "break" that isn't broken already?
I can't find a single use case as to why dev deps should be in the gemspec, and not a Gemfile (especially since only Bundler can reliably install dev deps right now).
Maybe informational metadata is useful - but then there's no reason to supply version constraints, because enforcing them would be more painful than not (ronn example on JRuby).
If dev deps in the gemspec were "deprecated", I wouldn't be able to put dev deps there without feedback telling me not to (I'd have to put them in the Gemfile, obviously). This would make a clear distinction: runtime stuff -> in the gemspec. dev stuff -> in the Gemfile it goes.
With dev gems allowed in the gemspec I have TWO places I can add dev deps to. That violates DRY for me. (and the development_group option is kind of proof of DRY being violated - because it means Bundler is mapping gemspec devs into it's own group).
It's an "I can live with it" thing - if metadata is so important and the gemspec provides the distinctions tools need, then I don't have to bother with a Gemfile anymore. Because I actually need multiple types defined (
(It would still be "strange" to define "gem-version agnostic" dev deps in the gemspec, but if that would be recommended over putting them in the Gemfile for gems, I'd live with it).
But having "just" a "development group" (aside from
So if the dev deps aren't clearly defined what they should have (as opposed to the
I'm open to hearing a "right way to do things" or being pointed to docs showing the "the right way" - and Yehuda's article is the best thing out there. And from what I understand, the idea of putting dev deps into the gemspec is wrong.
Just to switch to apps/Rails for a second: the distinctions there are MUCH clearer and more practical. The only issues is Bundler resolves all dependencies anyway (just skips the installs if told to), and many people just put all their gems in both the
Rubygems doesn't allow me to optimize those, because I have no idea what
The thing is: most people learn through "herd mentality" - if something is done in a gemspec in one project, they mindlessly "copy" it thinking it's a good practice. So deprecations seems to be the only way to communicate the intention behind the gemspec dev deps.
To me it seems
After all the investigation and research about how to use "gemspec dev deps" correctly, I still don't understand how specifying them is even useful from even a metadata perspective. It either increases bandwidth unnecessarily, or, if it's optimized not to be served - that just proves it's not important for anything.
Yes, that's why I have no objection to defining runtime dependencies in the gemspec (or install time dependencies). I expect installing a gem to work. And I do not expect development dependencies to have ANY impact on installing a gem. If they do, it's a bug or I've screwed up the runtime dependencies.
For some gems I may need Rails for development and testing (not unusual). But who really cares if I do? Only the person who is working on the area of the gem that actually needs it. Maybe it's only good for debugging. Maybe it's used for generating docs (not a strange idea).
And that's why I'd leave dealing with development dependencies to Bundler. It's because I don't want RubyGems to be checking stuff - especially if I'm using other gems in the form of potentially checked out master branches. I don't care about the version number at all, because on a master branch of a gem can have the same version number - but a totally different behavior (which I'm actually relying on). Or if the gem is no longer maintained, I may need to switch to a fork (which obviously isn't released as a gem).
I don't see a reason for RubyGems to have anything to do with my workflow in regards to development, testing, generating docs, etc. It's only once I've finished development and I want to release - when I even care about version constraints or metadata.
I only expect RubyGems to come into play when the user types in
But otherwise RubyGems only makes development harder if I have specify the deps manually and then manually install them (or add a special rake task to do that). With Bundler I can just put in
I would have agreed not that long ago. Nowadays I find that if a gem needs that kind of "management", it probably should be split up into smaller gems. It's as if when then documentation doesn't fit in the README - or the code isn't sufficiently simple and descriptive to replace API documentation - then I'm overdoing things.
Ironically, RubyGems makes it so easy to deal with "tiny" gems ... and Bundler takes care of dependencies so well ... releases can happen faster with more confidence, APIs of components can mature faster and independently and each component can have it's own branches, so migration becomes fluid and unobtrusive - yet still any change can be backported without forcing people into major versions they otherwise don't care about, etc.
I've used hoe in the past and it somehow overall discouraged me from creating gems. I was based on great ideas (extracting the specification data), and the plugins were nice - it "seemed" to be more optimized for "managing" gems intended for users who are developers. (The coolest thing about hoe (that I miss) is that I actually can avoid seeing the gemspec altogether.)
And I wish I could "hoeize" existing gems in a less intrusive way that existing gem authors would accept in their project.
To me "adding" stuff to a gem "from the outside" (as opposed to editing existing files as configuration) just seems better - probably because I prefer to add functionality through "revertible commits" and not "config".
I also do things very differently nowadays, which probably explains why I don't see the point of involving RubyGems in development:
And those commands are pretty project-agnostic, so it would be hard to justify migrating projects to use
On the otherhand, if
Also, since docs usually represent the API for SemVer, if docs change without a major release it's more likely a sign that compatibility was broken.
I don't see a reason for even Hoe to actually bother with the development dependencies. The hoe
To summarize, here are options I'd be happy with:
I believe I understand this issue well enough to explain to other developers how to correctly get RubyGems, Bundler and Travis working as expected. And given the amount of people out there - I can't pass on this knowledge by contacting every person on the planet. I'm not really interested in sending thousands of PRs to fix other people's dev deps (I'd die under email notifications). And it's annoying for me to break away from the PR/fix I'm working on to create a separate PR or commit just fixing the deps.
So it would be really nice if other people just had a way of knowing how to define dev/test/build/doc dependencies to get the results they want both for them and other contributors to their projects.
As a contributor, I'm a "nobody", so people will be suspicious if I submit a PR fixing "something" in their gemspec.
But if RubyGems points them to an article with explanations - I can just point them to that official doc even in the commit. And the patch is more likely to be accepted.
If I have trouble convincing even one of the biggest Ruby heroes out there that this is confusing...
... there's a problem either in my understanding or my communication.
I'm not asking to touch anything that would affect users installing gems (that would obviously be a nightmare for the Rubygems team). This is no "worse" than the "gem dependency defined multiple times" Bundler warning when people have e.g. rake in both the Gemfile and gemspec. In fact the deprecation would help guide people towards correctly avoiding the warning.
According to SemVer it would be perfectly normal to include this in the next minor release - so it will only affect developers who've upgraded RubyGems.
Let me know if I can do anything else to help resolve this (or if there is anything this deprecation could actually "brake").
I've fixed at least two bugs in development dependency installation (as fallout from the switch to the dependency resolver) in the past six months. There is one or two more that isn't explicitly mentioned in History, so I know the feature gets used.
With respect to time invested, maintaining the feature is something I will continue to have to do until the feature can be removed. Since there is no timeline on removal and, relative to the rest of my workload, the maintenance burden is small I do not find this compelling. Just because I am removing a feature doesn't mean I'm allowed to stop fixing bugs in it as that is equivalent to removing the feature.
There are many different types of users of RubyGems, not just "dev ops" people. I can't make a decision based on the ideals of one group of users unless it is absolutely overwhelming.
There are many gems that are written and released without using Bundler in any way. Forcing users to edit two files to build their gems does not follow the DRY principle regardless of how you wish to define dependencies. You still force users to put one type of gem metadata in one place and another type in a separate place.
It is almost always more work for me to remove a feature from RubyGems than it is to maintain it. Removing a feature involves many carefully orchestrated, documented and announced steps over many months and years. In this particular case it involves writing new code to deal with existing gems with existing development dependencies in a sane manner, something I will need to do for the rest of RubyGems history.
I've read and skimmed about a fifth of what you've written, I don't have the time or energy to read it all, can you summarize down to a three or four paragraphs?
Sorry if I suggested that. To me the feature feels "abandoned" by users (given it doesn't work for popular gems).
Yes, and those users are likely manually using
BUILD GEMS??? So that's what I was confused about!!!
It's not "development dependencies", it's "BUILD dependencies" the gemspec defines!
So I can assume:
is REALLY something like:
This makes LOTS of sense to me now!
I wish there was at least an alias to
An in Gemfiles I could use e.g.:
gemspec development_group: :build gem 'foo', group: :development # (etc.)
I'm sorry I was so confused about this and it took me this long to make the distinction I needed.
I understand completely.
My goal here is to reduce confusion without creating extra work for you.
Would aliasing 'add_build_dependency
I understand those - that's why I though deprecation would be a good way of "testing the waters" without risk or maintenance costs.
Thank you sincerly for your time and insights!
Thanks and have a warm, happy Christmas!
+1 to mark
Agree that it might be useful in pre-bundler era. However things changed and any ruby projects should use bundler - including gem development. I don't know what is the merit of having dev dependency in gem spec itself and in gem source.
If it's extraneous feature, then it would be better to deprecate and then remove it to focus on other key features.
I appreciate contributors' efforts on this feature, but I don't think we need to consider such sunken cost. Of course we have to get more feedback and then take phaseout process when decided.
I get your point but I'd argue that using a gem and developing a gem is different use case. In bundler era all ruby developers are expected to use bundler in their environment. dev dependencies can be in such environment by Gemfile. It doesn't need to be in gemspec.
I think all *users use just runtime dependency. dev dependency is for developer of each gem.
I think the main point is how seriously we consider bundler as a "standard tool"...
@evanphx - how so? I've always believed that, but now I can't find a single "good" reason where it isn't using the "wrong tool for a given job".
The point I 100% agree with is that given the "age" and scale of RubyGems, deprecation is simply not worth the actual costs.
TL;DR - I agree deprecation is costly, so I think a "new" tool (alongside Bundler and RubyGems) would be a better place for "new features" and "deprecations".
@chulkilee - thanks, you totally understand me!
And without Bundler, there's no guarantee you're actually using the right versions of build tools to build the gem anyway (while
I understand that just keeping RubyGems working as it is requires a ton of effort few people are even capable of appreciating. Also, there are at least thousands of people "relying" on the feature (even for strange purposes), so any "feedback" will be enormous in quantity (i.e. tons of "complaints").
I think a better option would be a "new" tool that would integrate Bundler and RubyGems functionality in an "integrated" and "recommended" way. Package managers are a tough problem - especially when it comes to backward compatibilty. While it would be confusion to have another tool (given there's Bundler and RubyGems already), but it does give millions of people a choice (backward compatibility vs new features).
Also, a lot of gems out there are "ancient" - adding a deprecation to them would only cause maintainers and users to waste focus.
I can't agree more.
Gems are for "use", while repositories are for development. (In a perfect world, we'd all be using the master branches of everything in production).
Not to mention - almost every development dependency can be optional (contributing/developing does not necessarily imply you have to be able to build a gem yourself on your own workstation). And so if dev/build deps are optional - why should they be expected to be part of a "released specification" (gemspec)?
I'd disagree that it's even about bundler - you can have a Rakefile so that e.g.
A gemspec is just too restrictive for dev deps - you can't lock to a specific version of Golang for example (e.g.
@e2 What features are you thinking a new tool needs? We've been adding new things people need to Rubygems fine.
Because dev_deps is unable to articulate the universe of development needs doesn't mean it's a bad option. It's not uncommon when working on a gem for the first time to install the development dependencies and the best way they're articulated is via dev_deps.
TL;DR - Bundler Rakefile shows that gemspec devdeps don't help with a real-world problems
@evanphx - a new tool would "absorb" paradigm changes faster than the heavily relied on RubyGems which already has a massive legacy or expensive (backward-compatibility) features.
As for dev_deps in a gemspec - I believe the fact that no extremely knowledgeable and experienced Ruby "guru" (e.g. you or @drbrain) can articulate a single "good" use case for it is ... case in point.
It's not a bad option technically (I'd just say it's plain inferior given other options) - it's just "cargo cult". It's like a configuration option that no one uses, but everyone specifies because "other people do it".
Rake and Bundler are much better for setting up the dev environment - whether those deps are in the gemspec or not is just an implementation detail. There's just no advantage to having those in a gemspec - only disadvantages.
This is especially true for platform-specific dev tools (Rubinius, JRuby, etc.).
It's pretty much the same story as with specifying test files in a gemspec. People do it but ... what's the point?
Not true - the best way to install deps is by what's recommended in the docs. E.g. even Bundler (which can't use itself to install deps) has custom code (for
And because it uses the gemspec (a mistake in my opinion), it breaks the DRY principle (that @drbrain talked about), because it has to remove
An important feature would be "weak dependencies", especially "conflicts", meaning that gem X at version Y causes problems if used - but is not necessary as a dependency (I don't remember if I searched the issues for this, but I'm sure it's popped up). This would allow to have "optional" gems, but also lock to correct versions if they are actually installed.
Another would be "features" - so that gems could depend on a "feature" of a gem, and not on a whole gem - "features" could have conflicts, etc. (you could install multiple "conflicting" gems, but that would be fine if the needed "features" are not in conflict). The alternative would be to have one gem per feature - but that would duplicate a lot of code (breaking DRY), be a maintenance nightmare (lots of gems to track, even though most of the functionality overlaps) and make it hard to "discover" related/available gems.
But I'm discouraged from even suggesting ideas like that if there's so much debate and trouble (though justified) deprecating a single existing method.
P.S. Bundler is a fantastic example of how this worked out (new tool solving existing problems with RubyGems)
@e2 Bundler was designed to solve a very real and specific problem that Rubygems did not: namely a single place to specify all gems for an app that was resolved as an atomic unit.
As for the other features you're discussing, they're interesting and have been brought up over the years. Either they've got unsolved ramifications (if gem A requests feature B that 2 gems provide, which one is activated?) or haven't been explained to provide any tangible value (optional features).
Perhaps I'm not explaining the case for dev_deps enough if you claim that I can't come up with a use. But even if I'm not explaining it well enough, I feel it still provides more value than an optional gems feature.
I hope this doesn't come across as hostile, that's not my intention. We're open to changes within Rubygems but it's important the changes be well considered.
TL;DR - "optional runtime dep info in gemspec" = bad but "dev deps in gemspec" = good? I don't get it.
Yes - and this was so useful in practice, some of this functionality is being merged back into RubyGems (reading a Gemfile, etc.), while Bundler outgrew it's purpose (e.g. you can quickly setup a new gem project using Bundler's features). It's now more of a development tool than RubyGems will ever be - and thus my point.
I'm sure they've been brought up - the point I was making, is that I don't believe any of those feature would get implemented in the next 3-5 years anyway (which is a long time in the IT industry) - no matter how well specified they get. And at that point, the paradigm will be so different again, that there may be a "new" different tool (like Bundler) which will superseed the needed functionality.
I understood the example - I just don't see how that's superior e.g. to having dev deps in a Gemfile and using Bundler. Especially since "development" practically means you have the source tree checked out from a repo anyway. I mean - if everyone suddenly moved their dev deps from gemspecs to Gemfiles - I don't see how the world would break or how that would negatively impact productivity.
Optional gems are useful if you have a plugin architecture - and there are dependencies between plugins and/or if the plugins can be used as standalone gems. The only other option is to pull in all plugins (makes no sense if they are unrelated) or users will get runtime errors (due to missing conditional deps) - or - if you use complex sets of "microgems" and "metagems" (which the user has to more carefully select in their project = more work). Not to mention that you'd get a circular dependency (plugin depends on app, app depends on plugin - to maintain semver on both sides).
But the value in storing effectively a development configuration in a "product specification"? And where the dependencies are used only for installation and not activation? I don't see the benefit.
Since Ruby is an interpreter (and not a compiler), activation is more important than any "listed" installation dependencies (as you pointed out with the example about activation). And dev deps in a gemspec are not about activation (unless there's a Gemfile, etc.).
I don't get the difference: somehow adding development dependencies (not even used when building a gem) in a gem's metadata is more useful than putting optional runtime dependencies in a gem (which would allow a lib to do checks at runtime).
Having "optional runtime deps" would be extremely useful, since an outdated gem could work out that it's outdated (in the given context) - even with no RubyGems support for the actual checking.
Why would it be ok to hardcode runtime version checking inside a lib (for supporting optional features/gems), but somehow putting development dep checking in the gem's metadata is "useful"?
Having it both ways is double standards to me.
No, I apologize if I seem hostile or "trolling". It's baffling to me. To over-exaggerate: it's like a conspiracy - "everyone" acts like it's a well-thought-out super-useful feature, but no one can explain why (or I'm too dumb to understand).
Putting extra runtime-related metadata in a gemspec seems "evil", except for development deps, which seem "too sacred to even touch".
It's like Franz Kafka's "The Trial".
Given everyone's time, I feel guilty for even starting the topic.
I expected one of the following resolutions:
I'm closing this issue, because I believe it will sooner become irrelevant (by how technology advances) than coming to a resolution.
I see zero upside for defining development dependencies inside gemspec. It's by all measurements easier to define them inside Gemfile, and the result is more flexible, easier to change, etc. So, if we're invoking DRY, that's where the functionality should live.
I published my very first gem recently, and in my search for guidance, stumbled upon this thread. I learned a lot from reading through it, and I thought you might be interested to know that it formed the basis for a blog post I just wrote about effective communication.
I don't point any fingers or take any sides, and I hope you guys find it valuable.