Free Adapters #83

Closed
rking opened this Issue Feb 2, 2013 · 33 comments

Comments

Projects
None yet
6 participants

rking commented Feb 2, 2013

Now that wdm can be installed without error (thanks @maher4ever!), What prevents listen from depending on these gems directly?

There are very many projects that do their adapters deps wrong (either they have none, and fall back to polling, or they write Ruby conditionals in the Gemfile and make the Gemfile.lock useless on other platforms).

Depending on rb-fsevent, rb-inotify, and wdm would mean all three of those systems would get OS-accelerated listen with as little as gem 'guard' in their Gemfile.

Sure, it downloads unreferenced code, but who cares about that? If someone wants to be platform-specific they can use the other gems directly. If there is a thundering crowd of users requesting the listen API be available to people who want platform-specific Gemfile.locks, we could perhaps reactor listen_base gem out and listen itself merely a metagem downloading it and all other adapters.

Also, we can work with the adapters' authors to push even lighter gems for the mismatch platforms (e.g., a win32 gem of rb-fsevent that had nothing but a check that wdm it's installed an a small warning otherwise).

What do you think?

rking commented Feb 2, 2013

Oh, and rb-fchange is ready to go as well. I think the only platform missing is the Casio wristwatch, but that thing says it's water resistant to 25 meters but will flood if you take a shower while wearing it, so it's users are accustomed to disappointment.

Owner

thibaudgg commented Feb 4, 2013

Hi @rking what do you think of the solution proposed by @justinforce on #81 ?

Member

Maher4Ever commented Feb 11, 2013

The reason behind allowing WDM to be installed on non-Windows systems is to allow Listen (and other projects that use it) to specify their dependencies in the gemspec file.

Because we now let the user install the appropriate adapter depending on their system, some things got a bit complicated for Listen users, and consequently Guard users as well. For example, developers using multiple development systems and working on one project need to have multiple if/else statements in the project's Gemfile to install the write adapter for the used system. This is exactly what's being done in the Gemfile of Listen!

Considering this fact, one can come to the conclusion that what's actually happening right now is that the developers themselves are the ones managing the dependency for Listen in their Gemfiles. The current dependency manager was previously needed for WDM. Since WDM is not more a problem, I agree with @rking's proposal to list Listen's dependencies in the gemspec.

Owner

thibaudgg commented Feb 12, 2013

As remember, it was implemented that way at first but @voxik opened this issue #54 .
I'm personally for re-adding all the gem dependencies in the gemspec as well.

@voxik is it still an issue for you? Do you see better alternative?

Solution proposed in #81 (comment) is maybe still a good idea? @justinforce is it working well with bundler?

voxik commented Feb 12, 2013

Adding dependencies which you don't need does not make sense to me. Especially in era of Bundler, they'll get baked in in the Gemfile.lock and you cannot get rid of them. We would need to remove (patch out) such dependencies in Fedora and user experience would be even more broken then using polling.

This comment [1] is still very valid, although @Maher4Ever already agrees with inclusion.

[1] #54 (comment)

Adding all platform-specific dependencies makes good sense to me. Here's my rationale:

If you use RVM, you get bundler and the rubygems-bundler gem as dependencies. Have you used rubygems-bundler yet? It's really cool! You don't have to type bundle exec foo anymore. However, If a gem is not defined in your Gemfile, it is not provided to your environment. This is super handy because it means my development environment only has what I specify, and I know that I'm not going to push a build where I forgot to add some gem I already had (like pry or something) because it was available in my local environment but not my app.

The downside would be that in order to use rb-inotify with listen and guard, I have to manage the dependency myself (add it to my Gemfile), even though it's not really my dependency—it's listen's. See, this doesn't sound to me like it's my responsibility or rubygems-bundler's responsibility. This sounds like listen's responsibility.

I would much rather install a few no-op gems as dependencies of a dependency in my Gemfile (I include guard, guard includes listen, listen includes adapters for Windows, Linux, and Mac.) Even though I only use Linux, I don't care. If I wanted peak efficiency, I'd be writing in C. I want rapid, expressive development with minimal fuss. This is why I use Ruby.

I'm going to put some time into pursuing #81 as soon as I can (work has been demanding the past few weeks. 😄)

rking commented Feb 12, 2013

@voxik - What is the harm if they do get baked into the Gemfile.lock?

I can tell you what is the harm if they don't -- Guard ends up using polling, which makes it prohibitively slow on large projects.

voxik commented Feb 13, 2013

@justinforce
What you said is wrong. It is like if you install RoR, it would install RSpec and test-unit, Sass and Less, Sqlite3, Postgres and MySQL. But may be you noticed that it is not true. RoR install just basic subset of tools you need and than you can extend. Actually even RoR install too much crap, since you can use RoR without ActiveRecord for example. So it is always up to you to manage your dependencies and it is the best way how to keep them on reasonable level.

@rking
Speaking of Fedora, there are several problems:

  1. You have to get the additional gems into Fedora and someone has to maintain them
  2. Once the additional gems are in Fedora, you cannot update them, because somebody has them in their Gemfile.lock file. Every even tiny update of whatever gets into Gemfile.lock will break your application. This might be fine when you are developer on your own machine, but when I do update in Fedora, I am responsible to not break possibly millions of applications which are developed on Fedora.

voxik commented Feb 13, 2013

And I forgot:

  1. First of all, it does not make any sense to package some packages which dose not run on Fedora, although they are installable, so this is no go for review.
  2. Therefore, we would have to remove the dependency and we would be different from other platforms
  3. Now, if I would like to cooperate with some mac developer, I would give him my application with Gemfile.lock, this would not be usefull on mac, since the dependencies would be different and this is the situation I'd like to avoid.

rking commented Feb 13, 2013

Ok, so if there is this back-pressure against extra gems, then what about merging wdm + rb-fsevent + rb-inotify ... etc into one codebase that has the same API but different compilation, depending on OS?

It could be the exact same gems as we have now, but unified under one tree.

voxik commented Feb 13, 2013

If you are thinking something about vendor directory, then that is no-go as well:

https://fedoraproject.org/wiki/Packaging:Guidelines#Bundling_of_multiple_projects

@voxik I still disagree with that assessment. I do not think that the comparison is apt. The impact of installing test-unit and rspec or mysql and postgres or sass and less alongside each other is very different than installing wdm, rb-inotify, and rb-fsevent together. These are adapters, not engines. Their functionality is mutually-exclusive because you can only use one per platform. The gems which do not support your platform are completely inert once installed, and they do not add appreciable bloat.

So is there some drawback that I'm missing related to the Gemfile or Gemfile.lock? So far, I'm not convinced.

voxik commented Feb 13, 2013

@justinforce "you can only use one per platform." - this is valid only as long as I will not implement something better then wdm, rb-inotify or rb-fsevent. If something is engine or adapter does not really matter.

You are missing that gems are distributed and installable not only by rubygems, but also by other means, such as yum on Fedora or apt-get on Ubuntu/Debian. I packaged listen gem for Fedora and as soon as you add the other dependencies, you will force me to package every of them and non of them has any meaning on my platform, so they will not pass review and will not be included in distribution.

@voxik Of course! I actually never considered installing gems without RubyGems. That makes perfect sense.

rking commented Feb 15, 2013

@voxik Why can't you patch the .gemspec to remove the wdm/rb-fsevent/etc deps?

voxik commented Feb 15, 2013

@rking Because if you develop on Fedora, you would have Gemfile.lock without the deps. So if you want to cooperate with lets say mac developers, they couldn't use Gemfile.lock, since their gem has different dependencies.

rking commented Feb 18, 2013

@voxik - I'm so confused about the use case you're describing.

If someone's packaging the listen gem for Fedora, then they can have their build script snip out the wdm/etc gems.

If someone's collaborating with someone on OS X, then their Gemfile.lock has to have rb-fsevent.

Owner

thibaudgg commented Feb 19, 2013

@rking @voxik yeah I'm quite confused too.

voxik commented Feb 19, 2013

Simply, the platform specific libraries are just extension.

  1. Nothing will break if they will not be available.
  2. Installing on my system something what I don't need does not make sense.
  3. Gems such as wdm will not go to Fedora, therefore if there will be hard dependency on it, which will be removed in Fedora from obvious reasons, applications using bundler will be broken.
Member

Maher4Ever commented Feb 23, 2013

Since this issue has sparked such a lengthy discussion, let me break the situation down and look at it from two perspectives. First of all, let's look at it from the perspective of direct users of Listen (i.e., people working directly with the Listen library). These users include software engineers using Listen as a lib in their application, maintainers of Listen like @thibaudgg and me, and packagers like @voxik.

One might expect that most of these users are familiar with what Listen does and know about platform-specific adapters (even if this knowledge was gained after seeing the polling warning on the first run). I think that this group of users will be in favor of an elegant design that decouples Listen from the platform-specific gems. Moreover, this group will not appreciate adding "dead" code because it a waste of bandwidth and storage. The dependency manager we added to Listen allows us to implement that elegant design. But as I already said in my previous comment, it shifted managing the dependencies to the users instead of letting rubygems do its job. Nevertheless, the directs users of Listen might have no problem with managing the dependencies themselves.

Now lets talk about the indirect users of Listen like the users of applications and other libs using Listen as a dependency. These users are not interested in Listen, its design or its dependencies. In fact, they shouldn't even be bothered because they might not have proper knowledge to understand or judge these things. Still, these users are now the ones managing the platform-specific dependency, for example by listing the adapter gem in the Gemfile of a project. If we make Listen depend on all platform-specific gems, these users would be relieved from that task.

Guard is a popular gem depending on Listen. Every Guard user is an indirect user of Listen. You can imagine that a user of Guard-plugin like guard-sass (e.g., a designer) might not even be familiar with ruby, let alone Bundler or rubygems. Forcing him to learn about Listen adapters (even via the warning message Listen shows) seems illogical to me.

Basically one must choose either elegant design or ease of use. I must say that I would choose ease of use for the users over elegant design. Concentrating on the design of a library might make Listen a perfect jewel, but it can also steer potential users away.

I would also like to address the specific case of @voxik. In one comment you say:

if I would like to cooperate with some mac developer, I would give him my application with Gemfile.lock, this would not be useful on mac, since the dependencies would be different and this is the situation I'd like to avoid.

This is actually true for the current version of Listen, because the developer needs to list his platform-specific adapter gem in the Gemfile which gets recorded in Gemfile.lock. I would argue that having all dependencies of Listen in the Gemfile.lock is actually better for collaboration between developers across systems. If all the dependencies are in the Gemfile.lock, then the other developer doesn't need to add the adapter of his system to the Gemfile when working on a new system; it's already in the Gemfile.lock!

Nothing will break if they will not be available.

True, but the polling adapter should only be used as a last resort when nothing else works. Polling should be used when the platform-specific adapter can't be used, not because it's not available, but when there is an issue with for example rights.

I packaged listen gem for Fedora and as soon as you add the other dependencies, you will force me to package every of them and non of them has any meaning on my platform, so they will not pass review and will not be included in distribution.

I'm not sure I understand what you mean by "you will force me to package every of them"? We plan to add all the adapters to the gemspec file, but yum doesn't rely on it. If i'm not wrong, packages in Fedora have their own spec file. As the maintainer of the Fedora Listen package, you are free to not depend on wdm and rb-fsevent.

Owner

thibaudgg commented Feb 27, 2013

@voxik It would be interesting to have your point on the fact that the gemspec isn't used by yum. In that case, I think it would be perfectly fine to have all theses dependencies included.

voxik commented Feb 27, 2013

Sorry for me being late.

I packaged listen gem for Fedora and as soon as you add the other dependencies, you will force me to package every of them and non of them has any meaning on my platform, so they will not pass review and will not be included in distribution.

I'm not sure I understand what you mean by "you will force me to package every of them"? We plan to add all the adapters to the gemspec file, but yum doesn't rely on it. If i'm not wrong, packages in Fedora have their own spec file. As the maintainer of the Fedora Listen package, you are free to not depend on wdm and rb-fsevent.

You are right, YUM/RPM doesn't care about .gemspec. But still, the .gemspec is used later by RubyGems to load the gems. I.e. if you do require 'listen', RubyGems checks Listens runtime dependencies specified in .gemspec file and tries to add the dependencies on Ruby's $LOAD_PATH alongside with Listen. If the dependency specified in .gemspec is not satisfied, the require fails. Therefore, the RPM's metadata and gem's metadata must be aligned.

Owner

rymai commented Apr 6, 2013

But still, the .gemspec is used later by RubyGems to load the gems. I.e. if you do require 'listen', RubyGems checks Listens runtime dependencies specified in .gemspec file and tries to add the dependencies on Ruby's $LOAD_PATH alongside with Listen. If the dependency specified in .gemspec is not satisfied, the require fails.

I have a naive question: Why don't Fedora users install and require Listen via RubyGems (instead of installing it via YUM/RPM and loading it via RubyGems)?

That all said, I think the discussion reaches its end, we've heard the rationales of everyone and IMHO the final decision will be taken by @thibaudgg and @Maher4Ever.

Owner

thibaudgg commented Apr 7, 2013

Yeah, I'm personally don't want to see anymore that kind of things: https://github.com/jonleighton/spring#filesystem-polling
For the rubygems community and indirect Listen users I think it'll be a lot better to include all these gems dependencies, Fedora users (and other RPM users) will need to use rubygems like everyone else or use another workaround.

👍
On Apr 7, 2013 12:46 AM, "Thibaud Guillaume-Gentil" <
notifications@github.com> wrote:

Yeah, I'm personally don't want to see anymore that kind of things:
https://github.com/jonleighton/spring#filesystem-polling
For the rubygems community and indirect Listen users I think it'll be a
lot better to include all these gems dependencies, Fedora users (and other
RPM users) will need to use rubygems like everyone else or use another
workaround.


Reply to this email directly or view it on GitHubhttps://github.com/guard/listen/issues/83#issuecomment-16011097
.

voxik commented Apr 7, 2013

@rymai Fedora users can of course install listen as a gem. But it is inconvenient. Imagine RPM/YUM can install all dependencies needed, not just RubyGems. I.e. if you installing some application, you can install it with proper DB, install service files to run it after system start etc. Using RubyGems, you can use just README and human labor to achieve this.

And actually, this is not about Fedora, not about packaging system, etc. This is about unnecessary code on my system. I don't want to have libraries I don't/can't use on my Fedora, Windows nor any other system.

It seems that the only objection thus far to just including all libraries
and simplifying this problem is that it causes negligible bloat to people
using listen in Fedora without RubyGems.

I think that a Ruby library developer should develop for the Ruby ecosystem
(RubyGems/Bundler) without heed for the specific requirements of any
particular platform. It is up to the maintainers of the platform to ensure
that useful libraries work on their platform. It is not the job of a gem
developer to ensure that every conceivable platform can utilize the gem
with zero friction. If a shim is required just for Fedora, the rpm
maintainer can see to the shim.

listen is bigger than that, and specifically useful to the Ruby dev
community. It should target the Ruby dev ecosystem, and that means working
with RubyGems.
On Apr 7, 2013 1:37 AM, "Vít Ondruch" notifications@github.com wrote:

@rymai https://github.com/rymai Fedora users can of course install
listen as a gem. But it is inconvenient. Imagine RPM/YUM can install all
dependencies needed, not just RubyGems. I.e. if you installing some
application, you can install it with proper DB, install service files to
run it after system start etc. Using RubyGems, you can use just README and
human labor to achieve this.

And actually, this is not about Fedora, not about packaging system, etc.
This is about unnecessary code on my system. I don't want to have libraries
I don't/can't use on my Fedora, Windows nor any other system.


Reply to this email directly or view it on GitHubhttps://github.com/guard/listen/issues/83#issuecomment-16011571
.

Owner

thibaudgg commented Apr 7, 2013

@justinforce I completely agree with you! I'll go ahead with that.

voxik commented Apr 7, 2013

"listen is bigger than that" ... oh my ... You might not hear about many of Fedora users/developers, because they rely on my work. Claims who is bigger are completely irrelevant. We should collaborate and do not separate Fedora and Ruby communities.

Anyway, I used all ammo I had. Do what you need to do.

And as I told you, I would have the same comments to be Windows user.

Owner

thibaudgg commented Apr 7, 2013

@voxik I really want to found the best solution for everybody, but I'm scared that currently it isn't possible because Bundler is broken. Before you said:

  1. Now, if I would like to cooperate with some mac developer, I would give him my application with Gemfile.lock,
    this would not be usefull on mac, since the dependencies would be different and this is the situation I'd like to avoid.

But currently Gemfile.lock made on a specific platform (say Fedora) will not be compatible/usefull on a mac because rb-fsevent wouldn't be included in that Gemfile.lock. So, right now, the only solution to have a Gemfile.lock compatible with all platforms is to include all gems needed for each platform. Once Bundler will be fixed, I see no problem at all to update the Listen gemspec with the ext/mkrf_conf.rb trick found by @justinforce

So for me the problem is more on the Bundler side than the Listen one. Don't you think?

voxik commented Apr 7, 2013

I agree that Bundler is the problem.

Owner

thibaudgg commented Apr 8, 2013

@voxik nice, thanks for your time on this issue.

Owner

thibaudgg commented Apr 12, 2013

=> #97 :)

thibaudgg closed this Apr 12, 2013

@tiredpixel tiredpixel added a commit to tiredpixel/curses that referenced this issue Feb 23, 2014

@tiredpixel tiredpixel Make pre-2.1.0 installation a successful no-op.
curses is part of ruby-core pre-2.1.0, so this gem is not required for
such versions. Previously, installation failed on Ruby 1.9.3 (but was
successful on Ruby 2.0.0).

    ruby#1

Subsequently, `required_ruby_version` was set in the `gemspec` to
clarify that only 2.1.0 and later are supported.

    ruby@4e55ed5

But this still means that gems attempting to offer multi-version Ruby
support fail, because of not being able to specifify `gem 'curses'`
within the `gemspec`. Instead, use the referenced resolution within

    guard/listen#83

as inspiration, and make pre-2.1.0 installation result in a no-op. In such
a case, `require 'curses'` is expected to work as usual.
643f17c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment