Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

Renaming presentation to Minimizing Library Dependencies. Structuring…

… a basic short presentation with a lot of notes. A few slides are incomplete.
  • Loading branch information...
commit de133a81e600b57447140cddc8c4523c12b927e4 1 parent 3592d66
Peter Marklund authored
View
298 README
@@ -1,285 +1,97 @@
############################################################################################
-### Rails Library Creep
+### MINIMIZING LIBRARY DEPENDENCIES
############################################################################################
-TODO: Rename to "Minimizing Library Dependencies"
-Using ~>: http://yehudakatz.com/2010/08/21/using-considered-harmful-or-whats-wrong-with
-
-http://semver.org/
-
-"In the world of software management there exists a dread place called "dependency hell." The bigger your system grows and the more packages you integrate into your software, the more likely you are to find yourself, one day, in this pit of despair."
-
+These are my slides for a presentation held at the Rails Meetup in Stockholm on October 25:th
+2010, see: http://rails.se/rails/show/SHRUG+25+Oktober+2010
############################################################################################
-### TABLE OF CONTENTS
+### NOTES
############################################################################################
-* Library Creep
-
-TODO: Rename to minimizing library dependencies
-Russian dolls. Libraries that wrap other libraries (TinyMCE, Sunspot, etc.)
-60 libraries in one project!
-The magplus story - brought number of libraries down to 0, then upgraded to Rails 3
-The Ruby 1.9.2 test.
-The s3_swf_upload story. Forked first to get rid of aws-s3 dependency. Then ripped out only the one file needed from the library. Mostly a generator.
-Types of libraries - generators, etc.
-FasterCSV example - not needed in Ruby 1.9.2!
-Fundamental problem - conflicting dependencies
-Patches of Ruby classes in the standard library (example 32 bit Integer shift in s3_swf_upload gem)
-Uses global variables (s3_swf_upload, logworm_amqp)
-How many different XML, JSON, etc. parsers do you end up with?
-Review your libraries and understand them! Libraries cannot be treated as a black box. They are too tightly coupled with the rest of your system.
-
-Example: carrierwave
-peter@flygarn ~/service-plus]$ sudo gem install carrierwave
-Password:
-Successfully installed carrierwave-0.5.0
-1 gem installed
-Installing ri documentation for carrierwave-0.5.0...
-Installing RDoc documentation for carrierwave-0.5.0...
-peter@flygarn ~/service-plus]$
-peter@flygarn ~/service-plus]$
-peter@flygarn ~/service-plus]$
-peter@flygarn ~/service-plus]$ sudo gem install fog
-Building native extensions. This could take a while...
-Successfully installed excon-0.2.4
-Successfully installed formatador-0.0.15
-Successfully installed net-ssh-2.0.23
-Successfully installed nokogiri-1.4.3.1
-Successfully installed fog-0.3.8
-
-Alternative to carrierwave:
-@brand.bucket.put("test-thumb.png", params[:thumb_image].open)
-
-MiniMagick is a light weight alternative to RMagick. Here is a light weight alternative to MiniMagick:
-identify mag-plus/popular-photo/thumb_com.bonnier.magplus.PPHiTune001D.PPH1110D.20101014-20.png
-mag-plus/popular-photo/thumb_com.bonnier.magplus.PPHiTune001D.PPH1110D.20101014-20.png PNG 80x80 80x80+0+0 8-bit DirectClass 13.5kb
-
-Start by learning the underlying technology rather than the wrapping technology, the fundamentals. Lives longer.
-
-Ruby developers over optimize for pretty DSLs. You can have pretty DSLs without libraries. Underrated: writing your own application DSL.
-
+h1. Weighing In At
Ruby developers are in the mindset of minimizing lines of code. For some reason they tend to only apply this to their own code, not to the code of the libraries they install.
-
-Advantage: you run the tests. You understand the code. You have less code because you only write code you need.
-
-Look at all the libs I'm not installing
-
-Pretty dsls overrated
-
-Why?
-
-
-############################################################################################
-### LIBRARY CREEP
-############################################################################################
-
Rails Library Creep. A cousin of Feature Creep and Software Bloat. Ironic. Aren't we supposed to be lean?
-LOC
-
-Vendor Lockin
-
-############################################################################################
-### REFERENCES
-############################################################################################
-
-* "Feature creep":http://en.wikipedia.org/wiki/Feature_creep
-* "Software bloat":http://en.wikipedia.org/wiki/Software_bloat
-
-+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-++++++++++++++++++++++++++++++++ temporary notes below, unstructured ++++++++++++++++++++++++++++++++++++
-+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-
-*** Rails Library Dependencies (Det finns en röd tråd, jag lovar!)
-- I am going to be honest with you upfront: "I HATE RUBY ON RAILS PLUGIN AND GEM DEPENDENCIES!" PITA
-"warning: HTTParty depends on version 0.1.7 of crack, not 0.1.6"
-"can't activate , already activated gherkin-2.2.0"
-
-- A lot of Ruby on Rails libraries are driven for the desire for beautiful DSL wrappers. Examples: httparty, whenever.
-- The other type of library delivers gives you application level functionality (engines). Example: devise.
-- Third type is infrastructure/API level outside the scope of Rails. Most legitimate IMHO.
-
-Libraries with functionality that you don't really need, that are overkill. Example: capistrano-ext
-
-Example LOC from Newsdesk with library counts and LOC
-The httparty/clicky story
-Rational Version policy
-
-Managing complexity.
-
We Ruby developers are supposed to be lean and simple. Less is more, right?
+Libraries Accumulate and Quickly Become a Liability Over Time
-- Intertwining, when everybody talks to everybody.
-
-academic earth!
-
-<!-- * "Promoting Code Quality":promoting-quality.html
-* "Measuring Code Quality":measuring-quality.html
-* "Rails Library Dependencies":library-dependencies.html -->
-
-
-Choose Your Dependencies Wisely
-
-Not very Rails-esque. Choices, choices, choices. This library or that library? Lack of conventions and consensus. Rapidly changing landscape. The rug being swept away under your feet.
-
+h1. Library Smells
How do you decide if to use a library? How do you evaluate libraries? Like other code, since after all, libraries are code. How do you evaluate code?
-
-
-Software Health - chapter from my Newsdesk presentation
-
-Metrics
-
-Model layer independent of the framework (Rails)? Independent of the persistence layer? Why should your business logic depend on Rails? The fewer dependencies your code has, the longer it can live.
-
-Plugins that are intertwined with Rails. restful-authentication example. Code generation with controllers and views. authlogic, now devise.
-
-Libraries that are coupled and have a lot of dependencies don’t age well.
-
-A discussion of when to bring in libraries, and what to require from a library. Pros and cons.
-
-We are lured by the magic of the libraries and their pretty DSLs. and by short term productivity boosts.
-
-Version hell. Picking the right version of rails and the right version of the library.
-
-Prevent you from Ruby upgrades
-Rails upgrades
-Database upgrades.
+How many different XML, JSON, etc. parsers do you end up with?
+Review your libraries and understand them! Libraries cannot be treated as a black box. They are too tightly coupled with the rest of your system.
+Version incompatibility is a message like: "can't activate , already activated gherkin-2.2.0"
+"warning: HTTParty depends on version 0.1.7 of crack, not 0.1.6"
+Intertwining, when everybody talks to everybody.
+Convetions, Idioms, and Consistency: Always make HTTP requests the same way, Always parse JSON the same way, Always implement state machines the same way, Always do pagination the same way
+We are lured by the magic of the libraries and their pretty DSLs. and by short term productivity boosts. Model macros - has_amazing_feature.
Search engine upgrades
-
-libraries can give you flying start, quick win
-
-Questions to ask about the library:
-
-- How well does it functionality map your needs? Does it do a lot of stuff that you don’t need? Is there some stuff you need to add or change?
-- Does it have good test coverage
-- Is it being maintained?
-- Is it mature and stable?
-- What does it depend on?
-- How big is the code base? Can you read the code and readily understand how it works?
-- What is the interface of the library?
-- How high is the cohesion of the library?
-- How high is the coupling of the library?
-- What are the code metrics of the library?
-
Treating libraries as black boxes doesn’t really work so well in Rails land.
-
A typical Rails library is developed quickly (over night) by a single Rails developer in his free time, then further developed and maintained for a couple of months, then mostly abandoned, sometimes forked if it’s popular. A typical library is highly coupled to Rails. The library landscape is messy.
+Example: globalize2 - globalize2_hacked
+Can you maintain it?
The Ruby web development landscape has gotten a lot more diversified over the last five years. There used to be only two popular books (AWR and the Pickaxe). Now there are hundreds. There used to be one Ruby interpreter, now there are several. There used to be only one major Rails version, now there are three. There used to be few alternatives to Rails, now there are many other popular frameworks. Everybody used to use RDBMS databases, now NoSQL databases are all the rage. A typical Rails system used to be a single simple app, now service oriented architectures with several apps are popular.
-application code does exactly what you need, no more no less
-you wrote it, so you probably understand it
-you have tests for it that you run frequently
+h1. Advantages to Rolling Your Own
+You have less code because you only write code you need.
+Model layer independent of the framework (Rails)? Independent of the persistence layer? Why should your business logic depend on Rails?
+Libraries that are coupled and have a lot of dependencies don’t age well.
+The fewer dependencies your code has, the longer it can live.
+Can you install your app anywhere? A dependency is a sort of binding.
+Making it more difficult for a new Rails developer to take over and maintain your application.
+h1. 1. Application Level Libraries (Engines)
+I personally don't like having to look for application level stuff (controllers and views) outside the app.
Which levels in the stack belong in a library and which belong in your application? The engine discussion.
-Using libraries give you a bunch of stuff you don’t need (your needs and the needs of the plugin author are seldom identical).
+h1. 2. Libraries To Make Your App More DRY
-Libraries Accumulate and Quickly Become a Liability Over Time
-Do you understand exactly what the library does and how it’s implemented?
-Can you maintain it?
-Is it tested?
-Does it work with future versions of Ruby and Rails?
-Which dependencies does it have?
-Does it patch Rails or Ruby in bad ways?
-Who wrote it?
-Sexy DSLs (httparty, sunspot are examples)
-How many lines of Ruby code is it?
-When was it last updated?
-What is the current version?
-How stable is it?
-Do you need to hack it or monkey patch it to do what you need?
-
-Example: rsolr an sunspot gems:
-Code LOC: 16672
-find vendor/gems/rsolr-0.12.1/lib -name '*.rb'|xargs wc -l # => 492
-find vendor/gems/sunspot-1.0.3/lib -name '*.rb'|xargs wc -l # => 6889
-find vendor/gems/sunspot_rails-1.0.3/lib -name '*.rb'|xargs wc -l # => 1237
-
-Example: httparty
-find vendor/gems/httparty-0.6.0/lib -name '*.rb'|xargs wc -l # => 903
-find vendor/gems/crack-0.1.7 -name '*.rb'|xargs wc -l # => 1073
-
-When you upgrade, beware of backwards incompatibilities. Rational versioning policy of RubyGems, not always followed: http://docs.rubygems.org/read/chapter/7
-
-major.minor.build - i.e. 1.2.4
+h1. 3. Wrapper Libraries (Ruby DSLs)
+Ruby gems, especially Rails gems, tend to come and go like fashion.
+Learning the underlying API has more lasting value.
+Lack of conventions and consensus when it comes to libraries. Rapidly changing landscape. The rug being swept away under your feet.
+Start by learning the underlying technology rather than the wrapping technology, the fundamentals. Lives longer.
+Ruby developers over optimize for pretty DSLs. You can have pretty DSLs without libraries. Underrated: writing your own application DSL.
+h1. Specifying Version Dependency
A category 1 change (implementation detail) will increment the build number.
A category 2 change (backwards compatible) will increment the minor version number and reset the build number.
A category 3 change (incompatible) will increment the major build number and reset the minor and build numbers.
-
Example: httparty going from 0.4.3 to 0.6.0
Commit message: “Rolling back httparty to version 0.4.3 for compatibility with Clicky API (don't ask)”
-Example: globalize2
-globalize2_hacked
-
-Maybe in practice only category 1 changes (build number) can be relied on to be backwards
-compatible??????
-
-35 plugins and 35 gems!
-
-Libraries tend to depend on other libraries and “drag these along” into your up. You tend to end up with several libraries that do the same thing (XML, JSON parsing etc.).
-
-Ideally you want to have one consistent way that you always do pagination, file uploads, state machines, JSON/XML/CSV parsing/generation, authentication etc. in your app.
-
-Wikipedia: “A liability can mean something that is a hindrance or puts an individual or group at a disadvantage”
-
-Multiple libraries that do the same thing
-Over focus on pretty dsls so blindly that we sacrifice simplicity and maintainability
-The ruby interpreter
-Rails
-The database
-The os?
-
-Can you install your app anywhere
+h1. Look at all the libs I'm Not Installing
+The s3_swf_upload story. Forked first to get rid of aws-s3 dependency. Then ripped out only the one file needed from the library. Mostly a generator. Had Ruby patch example 32 bit Integer shift. Was not Ruby 1.9.2 compatible. Then replaced hundreds of lines
+with a one-liner that was Ruby 1.9.2 compatible and got rid of the gem.
+FasterCSV example - not needed in Ruby 1.9.2!
+Examples of use of global variables (s3_swf_upload, logworm_amqp)
+The fog library in turn depends on: excon, formatador, net-ssh, and nokogiri.
+Alternative to carrierwave: @brand.bucket.put("test-thumb.png", params[:thumb_image].open)
+MiniMagick is a light weight alternative to RMagick. Here is a light weight alternative to MiniMagick:
+identify mag-plus/popular-photo/thumb_com.bonnier.magplus.PPHiTune001D.PPH1110D.20101014-20.png
+=> mag-plus/popular-photo/thumb_com.bonnier.magplus.PPHiTune001D.PPH1110D.20101014-20.png PNG 80x80 80x80+0+0 8-bit DirectClass 13.5kb
+rsolr (492 LOC) - sunspot (6889 LOC) - sunspot-rails (1237 LOC)
+httparty (903 LOC) - crack (1073 LOC)
-Staying lean and simple
-Isolationist testing
+############################################################################################
+### REFERENCES
+############################################################################################
-Integration testing of APIs
+* "Feature creep":http://en.wikipedia.org/wiki/Feature_creep
+* "Software bloat":http://en.wikipedia.org/wiki/Software_bloat
+* "
+Using >= Considered Harmful (or, What’s Wrong With >=)":http://yehudakatz.com/2010/08/21/using-considered-harmful-or-whats-wrong-with
+* "Semantic Versioning":http://semver.org
+* "RubyGems Versioning Policies":http://docs.rubygems.org/read/chapter/7
-References:
-Newsdes Product Quality
12 hours to rate: http://www.slideshare.net/ehuard/12-hours-to-rate-a-rails-application
metric fu presentations
-Multiple libraries that do the same thing
-Over focus on pretty dsls so blindly that we sacrifice simplicity and maintainability
-
-The ruby interpreter
-Rails
-The database
-The os?
-
-Can you install your app anywhere
-
-Staying lean and simple
-
-like russian dolls
-code weight
-boom, sunspot, 7000 lines of code extra
-weighing you down, changing quickly, conflicting version changes,
-compatibility of versions, ruby upgrades, rails upgrades
-
-Quantity
-Libraries
-Moving parts
-Hacked libs
-
-Structure of files
-Lib dir
-Vendor
-Concerns
-
-Integration testing apis
-
Making it harder to understand an application. Example: friendly_id. 1500 lines, with datamapper and sequel support, AR 2 and AR 3.
Described as a swiss army knife. Great, now that's cohesion for you! The friendly_id library depends on the babosa gem which is 700 lines
of Ruby that does the actual slugging. Now, IMHO, it's better to use the babosa gem directly.
-Making it more difficult for a new Rails developer to take over and maintain your application.
############################################################################################
### REFERENCES FOR CONSIDERATION
@@ -293,5 +105,3 @@ Making it more difficult for a new Rails developer to take over and maintain you
* "Jake Scruggs: Metric Fu Presentation":http://www.channels.com/episodes/show/4013322/Using-metric-fu-to-Make-Your-Rails-Code-Better-Jake-Scruggs
* "Ruby on Rails Code Quality Checklist":http://www.matthewpaulmoore.com/ruby-on-rails-code-quality-checklist
* "Peter Marklund: Test Driven Development with Ruby":http://marklunds.com/test-driven-development-with-ruby.html
-
-The complexity cost of functionality is bigger than you think
View
34 code/cachable_model.rb
@@ -1,34 +0,0 @@
-#START:before
-def find_by_sql(sql)
- if model_cache.enabled? && cachable = model_cache.parse_sql(sanitize_sql(sql))
- if cachable[:column] == "id"
- return model_cache.fetch(cachable[:value]) { super(sql) }
- else
- if id = model_cache.read_lookup_id(cachable[:column], cachable[:value])
- return [find(id)]
- else
- objects = super(sql)
- # What if record doesn't exist? Well, we can't cache that since that introduces a dependency on all records.
- id = objects.first.try(:id)
- model_cache.store_id_lookup(cachable[:column], cachable[:value], id) if id && objects.size == 1
- return objects
- end
- end
- else
- return super(sql)
- end
-end
-#END:before
-
-#START:after
-def find_by_sql(sql)
- finder = CachableModel::Finder.new(self, sanitize_sql(sql))
- if finder.is_cached?
- finder.cached_objects
- else
- objects = super(sql)
- finder.store_in_cache(objects) if finder.can_cache?(objects)
- objects
- end
-end
-#END:after
View
6 config/metadata.yml
@@ -1,5 +1,5 @@
-title: Rails Library Creep
+title: Minimizing Library Dependencies
author: Peter Marklund
-company: Elabs
+company: Rails Mentor
copyright: 2010 Peter Marklund
-date: August 4:th, 2010
+date: October 25:th, 2010
View
3  content/library_creep.textile
@@ -1,3 +0,0 @@
-h1. Library Creep
-
-h1. Library Overdose
View
85 content/library_dependencies.textile
@@ -0,0 +1,85 @@
+h1. Minimizing Library Dependencies
+
+h1. Weighing In At
+
+<pre>
+Application LOC: 16008
+
+28 vendor/gems, LOC: 36000
+
+37 vendor/plugins, LOC: 28000
+
+vendor/rails, LOC: 96000
+</pre>
+
+h1. Library Smells
+
+* The library does too much (low cohesion) or too little
+* You need to patch the library to fit your application
+* A lot of gem dependencies
+* The gem dependencies duplicate or conflict with your current gems
+* High coupling through global variables, Rails or Ruby patches
+* Lack of test coverage
+* Not mature/stable. Unclear interface.
+* No active maintenance or community
+* Hard to read and understand. A lot of code.
+
+h1. Advantages to Rolling Your Own
+
+* You understand your own code and know how to maintain it
+* You run regression tests for your code when you make changes
+* The code does exactly what your app needs, no more, no less
+* You control the quality and style of the code
+* Ruby and Rails upgrades are easier
+
+h1. 1. Application Level Libraries (Engines)
+
+restful_authentication/device
+paperclip/carrierwave
+
+h1. 2. Libraries To Make Your App More DRY
+
+inherited_resource
+
+h1. 3. Wrapper Libraries (Ruby DSLs)
+
+mongoid
+right_aws
+rsolr - sunspot - sunspot-rails (russian dolls)
+whenever
+httparty
+TinyMCE
+
+h1. Specifying Versions
+
+<pre>
+ gem 'foobar', '>= 4.3.2' # probably not backwards compatible
+ gem 'foobar', '~> 4.3' # should be backwards compatible
+ gem 'foobar', '~> 4.3.2' # most likely backwards compatible
+ gem 'foobar', '4.3.2' # version lock
+</pre>
+
+h1. Version Lock/Promiscuity (Hell)
+
+"In systems with many dependencies, releasing new package versions can quickly become a nightmare. If the dependency specifications are too tight, you are in danger of version lock (the inability to upgrade a package without having to release new versions of every dependent package). If dependencies are specified too loosely, you will inevitably be bitten by version promiscuity (assuming compatibility with more future versions than is reasonable). Dependency hell is where you are when version lock and/or version promiscuity prevent you from easily and safely moving your project forward."
+
+h1. Look at all the libs I'm Not Depending On
+
+* carrierwave and fog
+* minimagick/rmagick
+* s3_swf_upload and aws/s3
+* httparty
+* restful_acl
+* whenever
+* restful_authentication
+* ssl_requirement
+* friendly_id - babosa
+
+h1. Libraries I Still Depend On
+
+* rails
+* delayed_job
+* right_aws (Github fork)
+* logworm_amqp
+
+h1. Resources
View
2  content/table_of_contents.textile
@@ -1,3 +1,3 @@
h1. Table of Contents
-* "Library Creep":library_creep.html
+* "Minimizing Library Dependencies":library_dependencies.html
Please sign in to comment.
Something went wrong with that request. Please try again.