atom.xml

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">

  <title><![CDATA[Bear Metal]]></title>
  <link href="http://bearmetal.eu/theden/atom.xml" rel="self"/>
  <link href="http://bearmetal.eu/"/>
  <updated>2015-01-18T09:07:53+02:00</updated>
  <id>http://bearmetal.eu/</id>
  <author>
    <name><![CDATA[Bear Metal OÜ]]></name>
    
  </author>
  <generator uri="http://octopress.org/">Octopress</generator>

  
  <entry>
    <title type="html"><![CDATA[Help! My Rails App Is Melting under the Launch Day Load]]></title>
    <link href="http://bearmetal.eu//theden/help-my-rails-app-is-melting-under-the-launch-day-load/"/>
    <updated>2015-01-12T13:39:00+02:00</updated>
    <id>http://bearmetal.eu//theden/help-my-rails-app-is-melting-under-the-launch-day-load</id>
    <content type="html"><![CDATA[<figure markdown="1">
  <a href="https://www.flickr.com/photos/pasukaru76/4484360302">
    <img src="https://farm5.staticflickr.com/4065/4484360302_70ac4b6a8d_o_d.jpg">
  </a>

  <figcaption>
    <p>
      Photo by <a href="https://www.flickr.com/photos/pasukaru76/4484360302">Pascal</a>
    </p>
  </figcaption>
</figure>


<p>It was to be our day. The Finnish championship relay in orienteering was about to be held close to us, in a terrain type we knew inside and out. We had a great team, with both young guns and old champions. My friend Samuli had been fourth in the individual middle distance championships the day before. My only job was to handle the first leg securely and pass the baton to the more experienced and tougher guys who would then take care of our success. And I failed miserably.</p>

<p>My legs were like dough from the beginning. I was supposed to be in good shape, but I couldn&rsquo;t keep up with anyone. I was supposed to take it easy and orienteer cleanly, but I ran like a headless chicken, making a mistake after another. Although I wouldn&rsquo;t learn the term until years later, this was my crash course to <a href="http://en.wikipedia.org/wiki/Ego_depletion">ego depletion</a>.</p>

<hr />

<p>The day before the relay we organized the middle distance Finnish champs in my old hometown Parainen. For obvious reasons, I was the de facto webmaster of our club pages, which also hosted the result service. The site was running on OpenACS, a system running on TCL I had a year or so of work experience with. I was supposed to know it.</p>

<p>After the race was over, I headed back to my friend&rsquo;s place, opened up my laptop… only to find out that the local orienteering forums were ablaze with complaints about our results page being down. Crap.</p>

<p>After hours or hunting down the issue, getting help on the OpenACS IRC channel, serving the results from a static page meanwhile, I finally managed to fix the issue. The app wasn&rsquo;t running enough server processes to keep up with the load. And the most embarrassing thing was that <em>the load wasn&rsquo;t even that high</em> – from high dozens to hundreds of simultaneous users. I headed to bed with my head spinning, hoping to scramble up my self confidence for the next day&rsquo;s race (with well-known results).</p>

<p>What does this have to do with Ruby or Rails? Nothing, really. And yet everything. The point is that most of us have a similar story to share. It&rsquo;s much more common to have a meltdown story with a reasonably low number of users than actually have a slashdotting/hackernewsing/daring fireball hit your app. If you aren&rsquo;t old enough to have gone through something like this, you probably will. But you don&rsquo;t have to.</p>

<hr />

<p>During the dozen or so years since the aforementioned episode, I&rsquo;ve gone through some serious loads. Some of them we have handled badly, but most – including the <a href="http://wildfireapp.blogspot.fi/2009/04/wildfire-runs-facebook-site-governance.html">Facebook terms of service vote campaign</a> – with at least reasonable grace. This series of articles about Rails performance builds upon those war stories.</p>

<p>We have already posted a couple of articles to start off the series.</p>

<ol>
<li><a href="http://bearmetal.eu/theden/rails-garbage-collection-naive-defaults/">Rails Garbage Collection: Naive Defaults</a></li>
<li><a href="http://bearmetal.eu/theden/does-rails-scale/">Does Rails Scale?</a></li>
<li><a href="http://bearmetal.eu/theden/rails-garbage-collection-age-matters/">Rails Garbage Collection: Age Matters</a></li>
</ol>


<p>This article will serve as kind of a belated intro to the series, introducing our high level principles regarding the subject but without going more to the details.</p>

<h2>The Bear Metal ironclad rules of Rails performance</h2>

<h3>Scalability is not the same as performance</h3>

<p>As I already noted in <a href="http://bearmetal.eu/theden/does-rails-scale/">Does Rails Scale?</a>, it&rsquo;s worth pointing out that performance is not the same thing as scalability. They are related for sure. But you can perform similarly poorly from your first to your millionth user and be “scalable”. There is also the difference that performance is right here and now. If your app scales well, you can just throw more hardware (virtual or real) at the problem and solve it by that.</p>

<p>The good news is that Rails scales quite well out of the box for the vast majority of real-world needs. You probably won&rsquo;t be the next Twitter or even Basecamp. In your dreams and VC prospectus maybe, but let&rsquo;s be honest, the odds are stacked against you. So don&rsquo;t sweat about that too much.</p>

<p>Meanwhile, you do want your app to perform well enough for your initial wave of users.</p>

<h3>Perceived performance is what matters</h3>

<p>There are basically three different layers of performance for any web app: the server level (I&rsquo;m bundling everything from the data store to the frontend server here), browser rendering and the performance perceived by the user. The two latter ones are very close to each other but not exactly the same. You can tweak the perceived performance with tricks on the UI level, something that often isn&rsquo;t even considered performance.</p>

<p>The most important lesson here is that the perceived performance is what matters. It makes no difference what your synthetic performance tests on your server say if the UI of the app feels sluggish. There is no panacea to solve this, but make no mistake, it is what matters when the chicken come home to roost.</p>

<h3>Start with quick, large, and obvious wins</h3>

<p>The fact is that even a modest amount of users can make your app painfully slow. The good thing is that you can probably fix that just with hitting the low-hanging fruit. You won&rsquo;t believe how many Rails apps reveal that they&rsquo;re running in the development mode even in production by – when an error occurs – leaking the whole error trace out to the end user.</p>

<p>Other examples of issues that are fairly easy to spot and fix are N+1 query issues with ActiveRecord associations, missing database indeces, and running a single, non-concurrent app server instance, where any longer-running action will block the whole app from other users.</p>

<h3>YAGNI</h3>

<p>Once you have squashed all the low-hanging fruit with your metal-reinforced bat, relax. Tuning app performance shouldn&rsquo;t be your top priority at the moment – unless it is, but in that case you will know for sure. What you should be focusing on is how to get paying customers and how you can make them kick ass. If you have clear performance issues, by all means fix them. However…</p>

<h3>Don&rsquo;t assume, measure</h3>

<p>You probably don&rsquo;t have any idea how many users your app needs to support from the get go. That&rsquo;s fine. The reality will teach you. As long as you don&rsquo;t royally fuck up the share nothing (and 12 factor if you&rsquo;re on a cloud platform such as Heroku) architecture, you should be able to react to issues quickly.</p>

<p>That said, you probably do want to do some baseline load testing with your app if you&rsquo;re opening for a much larger private group or the public. The good news is that it is very cheap to spin up a virtual server instance just for a couple of hours and hit your app hard with it. Heck, you can handle the baseline from your laptop if needed. With that you should be able to get over the initial, frightening launch.</p>

<p>Once your app is up and running under load from real users, your tuning work starts for real. Only now will you be able to measure where the real hot paths and bottlenecks in your app are, based on real usage data, not just assumptions. At this point you&rsquo;ll have a plethora of tools at your disposal, from the good old <a href="https://github.com/wvanbergen/request-log-analyzer">request log analyzer</a> to commercial offerings such as <a href="https://www.skylight.io">Skylight</a>, and <a href="http://newrelic.com">New Relic</a>.</p>

<p>On the frontend most browsers have nowadays developer tools to optimize end-user performance, from Chrome and Safari&rsquo;s built-in developer tools to <a href="http://getfirebug.com">Firebug</a> for Firefox.</p>

<h2>Wrap-up</h2>

<p>In this introductory article to building performant Rails (or any, for that matter) web apps, we took a look at five basic rules of performance optimization:</p>

<ol>
<li>Scalability is not the same as performance.</li>
<li>Perceived performance is what matters.</li>
<li>Start with the low-hanging fruit.</li>
<li>YAGNI</li>
<li>Don&rsquo;t assume, measure.</li>
</ol>


<p>We will get (much) more in the details of Rails performance optimization in later articles. At that point we&rsquo;ll enter a territory where one size does not fit all anymore. However, whatever your particular performance problem is, you should keep the five rules above at the top your mind.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[How to install Ruby 2.2.0 on OS X Yosemite with Homebrew and rbenv]]></title>
    <link href="http://bearmetal.eu//theden/install-ruby-2-dot-2-0-yosemite-openssl-error/"/>
    <updated>2015-01-06T16:15:00+02:00</updated>
    <id>http://bearmetal.eu//theden/install-ruby-2-dot-2-0-yosemite-openssl-error</id>
    <content type="html"><![CDATA[<p>When trying to install Ruby 2.2.0 on Yosemite with <a href="https://github.com/sstephenson/rbenv">rbenv</a> and <a href="http://brew.sh">Homebrew</a>, I got a weird error:</p>

<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<span class='line-number'>12</span>
<span class='line-number'>13</span>
<span class='line-number'>14</span>
<span class='line-number'>15</span>
<span class='line-number'>16</span>
<span class='line-number'>17</span>
<span class='line-number'>18</span>
<span class='line-number'>19</span>
<span class='line-number'>20</span>
<span class='line-number'>21</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>➜  ~ ✗ rbenv install 2.2.0
</span><span class='line'>Downloading ruby-2.2.0.tar.gz...
</span><span class='line'>-&gt; http://dqw8nmjcqpjn7.cloudfront.net/7671e394abfb5d262fbcd3b27a71bf78737c7e9347fa21c39e58b0bb9c4840fc
</span><span class='line'>Installing ruby-2.2.0...
</span><span class='line'>
</span><span class='line'>BUILD FAILED (OS X 10.10.1 using ruby-build 20141225)
</span><span class='line'>
</span><span class='line'>Inspect or clean up the working tree at /var/folders/s_/_zh3skrd0qz2933nns5y425r0000gn/T/ruby-build.20150106151200.81466
</span><span class='line'>Results logged to /var/folders/s_/_zh3skrd0qz2933nns5y425r0000gn/T/ruby-build.20150106151200.81466.log
</span><span class='line'>
</span><span class='line'>Last 10 log lines:
</span><span class='line'>make[2]: *** Waiting for unfinished jobs....
</span><span class='line'>compiling raddrinfo.c
</span><span class='line'>compiling ifaddr.c
</span><span class='line'>make[1]: *** [ext/openssl/all] Error 2
</span><span class='line'>make[1]: *** Waiting for unfinished jobs....
</span><span class='line'>linking shared-object zlib.bundle
</span><span class='line'>linking shared-object socket.bundle
</span><span class='line'>linking shared-object date_core.bundle
</span><span class='line'>linking shared-object ripper.bundle
</span><span class='line'>make: *** [build-ext] Error 2</span></code></pre></td></tr></table></div></figure>


<p>When I opened up the log file mentioned in the output above, I could see the actual cause of the error:</p>

<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<span class='line-number'>12</span>
<span class='line-number'>13</span>
<span class='line-number'>14</span>
<span class='line-number'>15</span>
<span class='line-number'>16</span>
<span class='line-number'>17</span>
<span class='line-number'>18</span>
<span class='line-number'>19</span>
<span class='line-number'>20</span>
<span class='line-number'>21</span>
<span class='line-number'>22</span>
<span class='line-number'>23</span>
<span class='line-number'>24</span>
<span class='line-number'>25</span>
<span class='line-number'>26</span>
<span class='line-number'>27</span>
<span class='line-number'>28</span>
<span class='line-number'>29</span>
<span class='line-number'>30</span>
<span class='line-number'>31</span>
<span class='line-number'>32</span>
<span class='line-number'>33</span>
<span class='line-number'>34</span>
<span class='line-number'>35</span>
<span class='line-number'>36</span>
<span class='line-number'>37</span>
<span class='line-number'>38</span>
<span class='line-number'>39</span>
<span class='line-number'>40</span>
<span class='line-number'>41</span>
<span class='line-number'>42</span>
<span class='line-number'>43</span>
<span class='line-number'>44</span>
<span class='line-number'>45</span>
<span class='line-number'>46</span>
<span class='line-number'>47</span>
<span class='line-number'>48</span>
<span class='line-number'>49</span>
<span class='line-number'>50</span>
<span class='line-number'>51</span>
<span class='line-number'>52</span>
<span class='line-number'>53</span>
<span class='line-number'>54</span>
<span class='line-number'>55</span>
<span class='line-number'>56</span>
<span class='line-number'>57</span>
<span class='line-number'>58</span>
<span class='line-number'>59</span>
<span class='line-number'>60</span>
<span class='line-number'>61</span>
<span class='line-number'>62</span>
<span class='line-number'>63</span>
<span class='line-number'>64</span>
<span class='line-number'>65</span>
<span class='line-number'>66</span>
<span class='line-number'>67</span>
<span class='line-number'>68</span>
<span class='line-number'>69</span>
<span class='line-number'>70</span>
<span class='line-number'>71</span>
<span class='line-number'>72</span>
<span class='line-number'>73</span>
<span class='line-number'>74</span>
<span class='line-number'>75</span>
<span class='line-number'>76</span>
<span class='line-number'>77</span>
<span class='line-number'>78</span>
<span class='line-number'>79</span>
<span class='line-number'>80</span>
<span class='line-number'>81</span>
<span class='line-number'>82</span>
<span class='line-number'>83</span>
<span class='line-number'>84</span>
<span class='line-number'>85</span>
<span class='line-number'>86</span>
<span class='line-number'>87</span>
<span class='line-number'>88</span>
<span class='line-number'>89</span>
<span class='line-number'>90</span>
<span class='line-number'>91</span>
<span class='line-number'>92</span>
<span class='line-number'>93</span>
<span class='line-number'>94</span>
<span class='line-number'>95</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>ossl_ssl.c:125:5: error: use of undeclared identifier 'TLSv1_2_method'
</span><span class='line'>    OSSL_SSL_METHOD_ENTRY(TLSv1_2),
</span><span class='line'>    ^
</span><span class='line'>ossl_ssl.c:119:69: note: expanded from macro 'OSSL_SSL_METHOD_ENTRY'
</span><span class='line'>#define OSSL_SSL_METHOD_ENTRY(name) { #name, (SSL_METHOD *(*)(void))name##_method }
</span><span class='line'>                                                                    ^
</span><span class='line'>&lt;scratch space&gt;:148:1: note: expanded from here
</span><span class='line'>TLSv1_2_method
</span><span class='line'>^
</span><span class='line'>ossl_ssl.c:126:5: error: use of undeclared identifier 'TLSv1_2_server_method'
</span><span class='line'>    OSSL_SSL_METHOD_ENTRY(TLSv1_2_server),
</span><span class='line'>    ^
</span><span class='line'>ossl_ssl.c:119:69: note: expanded from macro 'OSSL_SSL_METHOD_ENTRY'
</span><span class='line'>#define OSSL_SSL_METHOD_ENTRY(name) { #name, (SSL_METHOD *(*)(void))name##_method }
</span><span class='line'>                                                                    ^
</span><span class='line'>&lt;scratch space&gt;:148:1: note: expanded from here
</span><span class='line'>TLSv1_2_server_method
</span><span class='line'>^
</span><span class='line'>ossl_ssl.c:127:5: error: use of undeclared identifier 'TLSv1_2_client_method'
</span><span class='line'>    OSSL_SSL_METHOD_ENTRY(TLSv1_2_client),
</span><span class='line'>    ^
</span><span class='line'>ossl_ssl.c:119:69: note: expanded from macro 'OSSL_SSL_METHOD_ENTRY'
</span><span class='line'>#define OSSL_SSL_METHOD_ENTRY(name) { #name, (SSL_METHOD *(*)(void))name##_method }
</span><span class='line'>                                                                    ^
</span><span class='line'>&lt;scratch space&gt;:148:1: note: expanded from here
</span><span class='line'>TLSv1_2_client_method
</span><span class='line'>^
</span><span class='line'>ossl_ssl.c:131:5: error: use of undeclared identifier 'TLSv1_1_method'
</span><span class='line'>    OSSL_SSL_METHOD_ENTRY(TLSv1_1),
</span><span class='line'>    ^
</span><span class='line'>ossl_ssl.c:119:69: note: expanded from macro 'OSSL_SSL_METHOD_ENTRY'
</span><span class='line'>#define OSSL_SSL_METHOD_ENTRY(name) { #name, (SSL_METHOD *(*)(void))name##_method }
</span><span class='line'>                                                                    ^
</span><span class='line'>&lt;scratch space&gt;:148:1: note: expanded from here
</span><span class='line'>TLSv1_1_method
</span><span class='line'>^
</span><span class='line'>ossl_ssl.c:132:5: error: use of undeclared identifier 'TLSv1_1_server_method'
</span><span class='line'>    OSSL_SSL_METHOD_ENTRY(TLSv1_1_server),
</span><span class='line'>    ^
</span><span class='line'>ossl_ssl.c:119:69: note: expanded from macro 'OSSL_SSL_METHOD_ENTRY'
</span><span class='line'>#define OSSL_SSL_METHOD_ENTRY(name) { #name, (SSL_METHOD *(*)(void))name##_method }
</span><span class='line'>                                                                    ^
</span><span class='line'>&lt;scratch space&gt;:148:1: note: expanded from here
</span><span class='line'>TLSv1_1_server_method
</span><span class='line'>^
</span><span class='line'>ossl_ssl.c:133:5: error: use of undeclared identifier 'TLSv1_1_client_method'
</span><span class='line'>    OSSL_SSL_METHOD_ENTRY(TLSv1_1_client),
</span><span class='line'>    ^
</span><span class='line'>ossl_ssl.c:119:69: note: expanded from macro 'OSSL_SSL_METHOD_ENTRY'
</span><span class='line'>#define OSSL_SSL_METHOD_ENTRY(name) { #name, (SSL_METHOD *(*)(void))name##_method }
</span><span class='line'>                                                                    ^
</span><span class='line'>&lt;scratch space&gt;:148:1: note: expanded from here
</span><span class='line'>TLSv1_1_client_method
</span><span class='line'>^
</span><span class='line'>ossl_ssl.c:210:21: error: invalid application of 'sizeof' to an incomplete type 'const struct &lt;anonymous struct at ossl_ssl.c:115:14&gt; []'
</span><span class='line'>    for (i = 0; i &lt; numberof(ossl_ssl_method_tab); i++) {
</span><span class='line'>                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
</span><span class='line'>ossl_ssl.c:19:35: note: expanded from macro 'numberof'
</span><span class='line'>#define numberof(ary) (int)(sizeof(ary)/sizeof((ary)[0]))
</span><span class='line'>                                  ^~~~~
</span><span class='line'>ossl_ssl.c:1127:13: warning: using the result of an assignment as a condition without parentheses [-Wparentheses]
</span><span class='line'>            if (rc = SSL_shutdown(ssl))
</span><span class='line'>                ~~~^~~~~~~~~~~~~~~~~~~
</span><span class='line'>ossl_ssl.c:1127:13: note: place parentheses around the assignment to silence this warning
</span><span class='line'>            if (rc = SSL_shutdown(ssl))
</span><span class='line'>                   ^
</span><span class='line'>                (                     )
</span><span class='line'>ossl_ssl.c:1127:13: note: use '==' to turn this assignment into an equality comparison
</span><span class='line'>            if (rc = SSL_shutdown(ssl))
</span><span class='line'>                   ^
</span><span class='line'>                   ==
</span><span class='line'>ossl_ssl.c:2194:23: error: invalid application of 'sizeof' to an incomplete type 'const struct &lt;anonymous struct at ossl_ssl.c:115:14&gt; []'
</span><span class='line'>    ary = rb_ary_new2(numberof(ossl_ssl_method_tab));
</span><span class='line'>                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
</span><span class='line'>ossl_ssl.c:19:35: note: expanded from macro 'numberof'
</span><span class='line'>#define numberof(ary) (int)(sizeof(ary)/sizeof((ary)[0]))
</span><span class='line'>                                  ^~~~~
</span><span class='line'>ossl_ssl.c:2195:21: error: invalid application of 'sizeof' to an incomplete type 'const struct &lt;anonymous struct at ossl_ssl.c:115:14&gt; []'
</span><span class='line'>    for (i = 0; i &lt; numberof(ossl_ssl_method_tab); i++) {
</span><span class='line'>                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
</span><span class='line'>ossl_ssl.c:19:35: note: expanded from macro 'numberof'
</span><span class='line'>#define numberof(ary) (int)(sizeof(ary)/sizeof((ary)[0]))
</span><span class='line'>                                  ^~~~~
</span><span class='line'>1 warning and 9 errors generated.
</span><span class='line'>make[2]: *** [ossl_ssl.o] Error 1
</span><span class='line'>make[2]: *** Waiting for unfinished jobs....
</span><span class='line'>compiling raddrinfo.c
</span><span class='line'>compiling ifaddr.c
</span><span class='line'>make[1]: *** [ext/openssl/all] Error 2
</span><span class='line'>make[1]: *** Waiting for unfinished jobs....
</span><span class='line'>linking shared-object zlib.bundle
</span><span class='line'>linking shared-object socket.bundle
</span><span class='line'>linking shared-object date_core.bundle
</span><span class='line'>linking shared-object ripper.bundle
</span><span class='line'>make: *** [build-ext] Error 2</span></code></pre></td></tr></table></div></figure>


<p>The weird thing here is that I did not get the usual &lsquo;Missing the OpenSSL lib?&rsquo; warning. The lib <em>was</em> found but somehow the headers were fucked up. It also did not happen with older rbenv Rubies.</p>

<p>Thanks to <a href="https://bearmetal.eu/team/tarmo/">Tarmo</a> I found the solution <a href="https://issues.apache.org/jira/browse/THRIFT-2515?focusedCommentId=14012758&amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14012758">here</a>.</p>

<p>What I had to do was this:</p>

<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>➜  ~  brew update
</span><span class='line'>➜  ~  brew install openssl
</span><span class='line'>➜  ~  /usr/local/opt/openssl/bin/c_rehash</span></code></pre></td></tr></table></div></figure>


<p>Now make sure that your new binary is in your PATH before the system one.</p>

<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>➜  ~  ln -s /usr/local/opt/openssl/bin/openssl /usr/local/bin/openssl
</span><span class='line'>➜  ~ which openssl
</span><span class='line'>/usr/bin/openssl
</span><span class='line'>➜  ~ echo $PATH
</span><span class='line'>/usr/local/heroku/bin:/Users/jarkko/.rbenv/shims:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin</span></code></pre></td></tr></table></div></figure>


<p>That&rsquo;s no good. Let&rsquo;s fix our PATH. I&rsquo;m using zsh, so for me it&rsquo;s set in <code>~/.zshrc</code>. Your particular file depends on the shell you&rsquo;re using (for bash it would be <code>~/.bashrc</code> or <code>~/.bash_profile</code>, but see the caveat <a href="http://www.joshstaiger.org/archives/2005/07/bash_profile_vs.html">here</a>).</p>

<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>➜  ~ vim ~/.zshrc
</span><span class='line'># Change the line that sets PATH so that /usr/local/bin
</span><span class='line'># comes BEFORE /usr/bin. For me, it looks like this:
</span><span class='line'># export PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</span></code></pre></td></tr></table></div></figure>


<p>Open up a new terminal window and check that the PATH is correct:</p>

<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>➜  ~  echo $PATH
</span><span class='line'>/usr/local/heroku/bin:/Users/jarkko/.rbenv/shims:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
</span><span class='line'>➜  ~  which openssl
</span><span class='line'>/usr/local/bin/openssl</span></code></pre></td></tr></table></div></figure>


<p>Better. Now, let&rsquo;s make sure that homebrew libs symlink to the newer openssl.</p>

<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>➜  ~  brew unlink openssl
</span><span class='line'>➜  ~  brew link --overwrite --force openssl
</span><span class='line'>➜  ~  openssl version -a
</span><span class='line'>
</span><span class='line'>OpenSSL 1.0.1j 15 Oct 2014
</span><span class='line'>built on: Sun Dec  7 02:14:31 GMT 2014
</span><span class='line'>platform: darwin64-x86_64-cc
</span><span class='line'>options:  bn(64,64) rc4(ptr,char) des(idx,cisc,16,int) idea(int) blowfish(idx) 
</span><span class='line'>compiler: clang -fPIC -fno-common -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -arch x86_64 -O3 -DL_ENDIAN -Wall -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
</span><span class='line'>OPENSSLDIR: "/usr/local/etc/openssl"</span></code></pre></td></tr></table></div></figure>


<p>Splendid.</p>

<p>After that, Ruby 2.2.0 installed cleanly without any specific parameters needed:</p>

<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>➜  ~  rbenv install 2.2.0
</span><span class='line'>Downloading ruby-2.2.0.tar.gz...
</span><span class='line'>-&gt; http://dqw8nmjcqpjn7.cloudfront.net/7671e394abfb5d262fbcd3b27a71bf78737c7e9347fa21c39e58b0bb9c4840fc
</span><span class='line'>Installing ruby-2.2.0...
</span><span class='line'>Installed ruby-2.2.0 to /Users/jarkko/.rbenv/versions/2.2.0</span></code></pre></td></tr></table></div></figure>


<p><strong>[UPDATE 1, Jan 7]</strong> The original version of this post told you to <code>rm /usr/bin/openssl</code>, based on the link above. As James Tucker pointed out, this is a horrible idea. I fixed the article so that we now fix the <code>$PATH</code> instead.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Rails Garbage Collection: age matters]]></title>
    <link href="http://bearmetal.eu//theden/rails-garbage-collection-age-matters/"/>
    <updated>2014-12-22T20:26:00+02:00</updated>
    <id>http://bearmetal.eu//theden/rails-garbage-collection-age-matters</id>
    <content type="html"><![CDATA[<p>In a <a href="https://bearmetal.eu/theden/rails-garbage-collection-naive-defaults/">previous post</a> in the <a href="http://bearmetal.eu/theden/categories/rails-performance">Rails Performance</a> series we stated that the default garbage collection settings for <a href="http://www.rubyonrails.org">Ruby on Rails</a> applications are not optimal. In this post we&rsquo;ll explore the basics of object age in RGenGC, Ruby 2.1&rsquo;s new <em>restricted generational garbage collector</em>.</p>

<p>As a prerequisite of this and subsequent posts, basic understanding of a <em>mark and sweep</em><sup id="fnref:1"><a href="#fn:1" rel="footnote">1</a></sup> collector is assumed.</p>

<p><img src="http://bearmetal.eu/images/gc_mark_sweep.png" alt="" /></p>

<p>A somewhat simplified mark and sweep cycle goes like this:</p>

<ol>
<li>A mark and sweep collector traverses the object graph.</li>
<li>It checks which objects are in use (referenced) and which ones are not.</li>
<li>This is called object marking, aka. the <strong>MARK PHASE</strong>.</li>
<li>All unused objects are freed, making their memory available.</li>
<li>This is called sweeping, aka. the <strong>SWEEP PHASE</strong>.</li>
<li>Nothing changes for used objects.</li>
</ol>


<p>A GC cycle prior to Ruby 2.1 works like that. A typical Rails app boots with 300 000 live objects of which all need to be scanned during the <strong>MARK</strong> phase. That usually yields a smaller set to <strong>SWEEP</strong>.</p>

<p>A large percentage of the graph is going to be traversed over and over again but will never be reclaimed. This is not only CPU intensive during GC cycles, but also incurs memory overhead for accounting and anticipation for future growth.</p>

<h2>Old and young objects</h2>

<p><strong>What generally makes an object old?</strong></p>

<ul>
<li><em>All new objects are considered to be young</em>.</li>
<li><em>Old objects survived at least one GC cycle (major or minor)</em> The collector thus reasons that the object will stick around and not become garbage quickly.</li>
</ul>


<p>The idea behind the new generational garbage collector is this:</p>

<blockquote><p><strong>MOST OBJECTS DIE YOUNG.</strong></p></blockquote>

<p>To take advantage of this fact, the new GC classifies objects on the Ruby heap as either <strong>OLD</strong> or <strong>YOUNG</strong>. This segregation now allows the garbage collector to work with two distinct generations, with the <strong>OLD</strong> generation much less likely to yield much improvement towards recovering memory.</p>

<p>For a typical Rails request, some examples of old and new objects would be:</p>

<ul>
<li><strong>Old:</strong> compiled routes, templates, ActiveRecord connections, cached DB column info, classes, modules etc.</li>
<li><strong>New:</strong> short lived strings within a partial, a string column value from an ActiveRecord result, a coerced DateTime instance etc.</li>
</ul>


<p>Young objects are more likely to reference old objects than old objects referencing young objects. Old objects also frequently reference other old objects.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'>  <span class="n">u</span> <span class="o">=</span> <span class="no">User</span><span class="o">.</span><span class="n">first</span>
</span><span class='line'>  <span class="c1">#&lt;User id: 1, email: &quot;lourens@something.com&quot;, encrypted_password: &quot;blahblah...&quot;, reset_password_token: nil, reset_password_sent_at: nil, remember_created_at: nil, sign_in_count: 2, current_sign_in_at: &quot;2014-10-31 11:52:30&quot;, last_sign_in_at: &quot;2014-10-29 10:04:01&quot;, current_sign_in_ip: &quot;127.0.0.1&quot;, last_sign_in_ip: &quot;127.0.0.1&quot;, created_at: &quot;2014-10-29 10:04:01&quot;, updated_at: &quot;2014-11-30 14:07:15&quot;, provider: nil, uid: nil, first_name: &quot;dfdsfds&quot;, last_name: &quot;dfdsfds&quot;, confirmation_token: nil, confirmed_at: &quot;2014-10-30 10:11:42&quot;, confirmation_sent_at: nil, unconfirmed_email: nil, onboarded_at: nil&gt;</span>
</span></code></pre></td></tr></table></div></figure>


<p>Notice how the transient attribute keys and names reference the long lived columns here:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<span class='line-number'>12</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="p">{</span><span class="s2">&quot;id&quot;</span><span class="o">=&gt;</span>
</span><span class='line'>   <span class="c1">#&lt;ActiveRecord::ConnectionAdapters::PostgreSQLAdapter::OID::Integer:0x007fbe756d1d30&gt;,</span>
</span><span class='line'>  <span class="s2">&quot;email&quot;</span><span class="o">=&gt;</span>
</span><span class='line'>   <span class="c1">#&lt;ActiveRecord::ConnectionAdapters::PostgreSQLAdapter::OID::Identity:0x007fbe756d1718&gt;,</span>
</span><span class='line'>  <span class="s2">&quot;encrypted_password&quot;</span><span class="o">=&gt;</span>
</span><span class='line'>   <span class="c1">#&lt;ActiveRecord::ConnectionAdapters::PostgreSQLAdapter::OID::Identity:0x007fbe756d1718&gt;,</span>
</span><span class='line'>  <span class="s2">&quot;reset_password_token&quot;</span><span class="o">=&gt;</span>
</span><span class='line'>   <span class="c1">#&lt;ActiveRecord::ConnectionAdapters::PostgreSQLAdapter::OID::Identity:0x007fbe756d1718&gt;,</span>
</span><span class='line'>  <span class="s2">&quot;reset_password_sent_at&quot;</span><span class="o">=&gt;</span>
</span><span class='line'>   <span class="c1">#&lt;ActiveRecord::AttributeMethods::TimeZoneConversion::Type:0x007fbe741f63c0</span>
</span><span class='line'>    <span class="vi">@column</span><span class="o">=</span>
</span><span class='line'>     <span class="c1">#&lt;ActiveRecord::ConnectionAdapters::PostgreSQLAdapter::OID::Timestamp:0x007fbe756d0e58&gt;&gt;,</span>
</span></code></pre></td></tr></table></div></figure>


<p>Age segregation is also just a classification &ndash; old and young objects aren&rsquo;t stored in distinct memory spaces &ndash; they&rsquo;re just conceptional buckets. The generation of an object refers to the amount of GC cycles it survived:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">00</span><span class="mi">9</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="no">ObjectSpace</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="o">[]</span><span class="p">)</span>
</span><span class='line'><span class="o">=&gt;</span> <span class="s2">&quot;{</span><span class="se">\&quot;</span><span class="s2">address</span><span class="se">\&quot;</span><span class="s2">:</span><span class="se">\&quot;</span><span class="s2">0x007fd24c007668</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">type</span><span class="se">\&quot;</span><span class="s2">:</span><span class="se">\&quot;</span><span class="s2">ARRAY</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">class</span><span class="se">\&quot;</span><span class="s2">:</span><span class="se">\&quot;</span><span class="s2">0x007fd24b872038</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">length</span><span class="se">\&quot;</span><span class="s2">:0, </span><span class="se">\&quot;</span><span class="s2">file</span><span class="se">\&quot;</span><span class="s2">:</span><span class="se">\&quot;</span><span class="s2">(irb)</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">line</span><span class="se">\&quot;</span><span class="s2">:9, </span><span class="se">\&quot;</span><span class="s2">method</span><span class="se">\&quot;</span><span class="s2">:</span><span class="se">\&quot;</span><span class="s2">irb_binding</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">generation</span><span class="se">\&quot;</span><span class="s2">:8, </span><span class="se">\&quot;</span><span class="s2">flags</span><span class="se">\&quot;</span><span class="s2">:{</span><span class="se">\&quot;</span><span class="s2">wb_protected</span><span class="se">\&quot;</span><span class="s2">:true}}</span><span class="se">\n</span><span class="s2">&quot;</span>
</span><span class='line'><span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">010</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="no">GC</span><span class="o">.</span><span class="n">count</span>
</span><span class='line'><span class="o">=&gt;</span> <span class="mi">8</span>
</span></code></pre></td></tr></table></div></figure>


<h2>What the heck is major and minor GC?</h2>

<p>You may have heard, read about or noticed in GC.stat output the terms &ldquo;minor&rdquo; and &ldquo;major&rdquo; GC.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<span class='line-number'>12</span>
<span class='line-number'>13</span>
<span class='line-number'>14</span>
<span class='line-number'>15</span>
<span class='line-number'>16</span>
<span class='line-number'>17</span>
<span class='line-number'>18</span>
<span class='line-number'>19</span>
<span class='line-number'>20</span>
<span class='line-number'>21</span>
<span class='line-number'>22</span>
<span class='line-number'>23</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">003</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">pp</span> <span class="no">GC</span><span class="o">.</span><span class="n">stat</span>
</span><span class='line'><span class="p">{</span><span class="ss">:count</span><span class="o">=&gt;</span><span class="mi">32</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:heap_used</span><span class="o">=&gt;</span><span class="mi">1181</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:heap_length</span><span class="o">=&gt;</span><span class="mi">1233</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:heap_increment</span><span class="o">=&gt;</span><span class="mi">50</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:heap_live_slot</span><span class="o">=&gt;</span><span class="mi">325148</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:heap_free_slot</span><span class="o">=&gt;</span><span class="mi">156231</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:heap_final_slot</span><span class="o">=&gt;</span><span class="mi">0</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:heap_swept_slot</span><span class="o">=&gt;</span><span class="mi">163121</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:heap_eden_page_length</span><span class="o">=&gt;</span><span class="mi">1171</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:heap_tomb_page_length</span><span class="o">=&gt;</span><span class="mi">10</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:total_allocated_object</span><span class="o">=&gt;</span><span class="mi">2050551</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:total_freed_object</span><span class="o">=&gt;</span><span class="mi">1725403</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:malloc_increase</span><span class="o">=&gt;</span><span class="mi">1462784</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:malloc_limit</span><span class="o">=&gt;</span><span class="mi">24750208</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:minor_gc_count</span><span class="o">=&gt;</span><span class="mi">26</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:major_gc_count</span><span class="o">=&gt;</span><span class="mi">6</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:remembered_shady_object</span><span class="o">=&gt;</span><span class="mi">3877</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:remembered_shady_object_limit</span><span class="o">=&gt;</span><span class="mi">4042</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:old_object</span><span class="o">=&gt;</span><span class="mi">304270</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:old_object_limit</span><span class="o">=&gt;</span><span class="mi">259974</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:oldmalloc_increase</span><span class="o">=&gt;</span><span class="mi">23639792</span><span class="p">,</span>
</span><span class='line'> <span class="ss">:oldmalloc_limit</span><span class="o">=&gt;</span><span class="mi">24159190</span><span class="p">}</span>
</span></code></pre></td></tr></table></div></figure>


<p><strong>Minor GC (or &ldquo;partial marking&rdquo;):</strong> This cycle only traverses the young generation and is very fast. Based on the hypothesis that most objects die young, this GC cycle is thus the most effective at reclaiming back a large ratio of memory in proportion to objects traversed.</p>

<p>It runs quite often - 26 times for the GC dump of a booted Rails app above.</p>

<p><strong>Major GC:</strong> Triggered by out-of-memory conditions - Ruby heap space needs to be expanded (not OOM killer! :-)) Both old and young objects are traversed and it&rsquo;s thus significantly slower than a minor GC round. Generally when there&rsquo;s a significant increase in old objects, a major GC would be triggered. Every major GC cycle that an object survived bumps its current generation.</p>

<p>It runs much less frequently - six times for the stats dump above.</p>

<p>The following diagram represents a minor GC cycle (<strong>MARK</strong> phase completed, <strong>SWEEP</strong> still pending) that identifies and promotes some objects to old.</p>

<p><img src="http://bearmetal.eu/images/gc_first_minor.png" alt="" /></p>

<p>A subsequent minor GC cycle (<strong>MARK</strong> phase completed, <strong>SWEEP</strong> still pending) ignores old objects during the mark phase.</p>

<p><img src="http://bearmetal.eu/images/gc_second_minor.png" alt="" /></p>

<p>Most of the reclaiming efforts are thus focussed on the young generation (new objects). Generally 95% of objects are dead by the first GC. The current generation of an object is the number of major GC cycles it has survived.</p>

<h2>RGenGC</h2>

<p>At a very high level C Ruby 2.1&rsquo;s collector has the following properties:</p>

<ul>
<li>High throughput - it can sustain a high rate of allocations / collections due to faster minor GC cycles and very rare major GC cycles.</li>
<li>GC pauses are still long (&ldquo;stop the world&rdquo;) for major GC cycles.</li>
<li>Generational collectors have much shorter mark cycles as they traverse only the young generation, most of the time.</li>
</ul>


<p>This is a marked improvement to the C Ruby GC and serves as a base for implementing other advanced features moving forward. Ruby 2.2 supports incremental GC and object ages beyond just old and new definitions. A major GC cycle in 2.1 still runs in a &ldquo;stop the world&rdquo; manner, whereas a more involved incremental implementation (Ruby 2.2) interleaves short steps of mark and sweep cycles between other VM operations.</p>

<h2>Object references</h2>

<p>In this simple example below we create a String array with three elements.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">001</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="nb">require</span> <span class="s1">&#39;objspace&#39;</span>
</span><span class='line'><span class="o">=&gt;</span> <span class="kp">true</span>
</span><span class='line'><span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">002</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="no">ObjectSpace</span><span class="o">.</span><span class="n">trace_object_allocations_start</span>
</span><span class='line'><span class="o">=&gt;</span> <span class="kp">nil</span>
</span><span class='line'><span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">003</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">ary</span> <span class="o">=</span> <span class="sx">%w(a b c)</span>
</span><span class='line'><span class="o">=&gt;</span> <span class="o">[</span><span class="s2">&quot;a&quot;</span><span class="p">,</span> <span class="s2">&quot;b&quot;</span><span class="p">,</span> <span class="s2">&quot;c&quot;</span><span class="o">]</span>
</span></code></pre></td></tr></table></div></figure>


<p>Very much like a river flowing downstream, the array has knowledge of (a reference to) each of its String elements. On the contrary, the strings don&rsquo;t have an awareness of (or references back to) the array container.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">004</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="no">ObjectSpace</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">ary</span><span class="p">)</span>
</span><span class='line'><span class="o">=&gt;</span> <span class="s2">&quot;{</span><span class="se">\&quot;</span><span class="s2">address</span><span class="se">\&quot;</span><span class="s2">:</span><span class="se">\&quot;</span><span class="s2">0x007fd24b890fd8</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">type</span><span class="se">\&quot;</span><span class="s2">:</span><span class="se">\&quot;</span><span class="s2">ARRAY</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">class</span><span class="se">\&quot;</span><span class="s2">:</span><span class="se">\&quot;</span><span class="s2">0x007fd24b872038</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">length</span><span class="se">\&quot;</span><span class="s2">:3, </span><span class="se">\&quot;</span><span class="s2">embedded</span><span class="se">\&quot;</span><span class="s2">:true, </span><span class="se">\&quot;</span><span class="s2">references</span><span class="se">\&quot;</span><span class="s2">:[</span><span class="se">\&quot;</span><span class="s2">0x007fd24b891050</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">0x007fd24b891028</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">0x007fd24b891000</span><span class="se">\&quot;</span><span class="s2">], </span><span class="se">\&quot;</span><span class="s2">file</span><span class="se">\&quot;</span><span class="s2">:</span><span class="se">\&quot;</span><span class="s2">(irb)</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">line</span><span class="se">\&quot;</span><span class="s2">:3, </span><span class="se">\&quot;</span><span class="s2">method</span><span class="se">\&quot;</span><span class="s2">:</span><span class="se">\&quot;</span><span class="s2">irb_binding</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">generation</span><span class="se">\&quot;</span><span class="s2">:7, </span><span class="se">\&quot;</span><span class="s2">flags</span><span class="se">\&quot;</span><span class="s2">:{</span><span class="se">\&quot;</span><span class="s2">wb_protected</span><span class="se">\&quot;</span><span class="s2">:true}}</span><span class="se">\n</span><span class="s2">&quot;</span>
</span><span class='line'><span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">004</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="no">ObjectSpace</span><span class="o">.</span><span class="n">reachable_objects_from</span><span class="p">(</span><span class="n">ary</span><span class="p">)</span>
</span><span class='line'><span class="o">=&gt;</span> <span class="o">[</span><span class="nb">Array</span><span class="p">,</span> <span class="s2">&quot;a&quot;</span><span class="p">,</span> <span class="s2">&quot;b&quot;</span><span class="p">,</span> <span class="s2">&quot;c&quot;</span><span class="o">]</span>
</span><span class='line'><span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">006</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="no">ObjectSpace</span><span class="o">.</span><span class="n">reachable_objects_from</span><span class="p">(</span><span class="n">ary</span><span class="o">[</span><span class="mi">1</span><span class="o">]</span><span class="p">)</span>
</span><span class='line'><span class="o">=&gt;</span> <span class="o">[</span><span class="nb">String</span><span class="o">]</span>
</span><span class='line'><span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">007</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="no">ObjectSpace</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">ary</span><span class="o">[</span><span class="mi">1</span><span class="o">]</span><span class="p">)</span>
</span><span class='line'><span class="o">=&gt;</span> <span class="s2">&quot;{</span><span class="se">\&quot;</span><span class="s2">address</span><span class="se">\&quot;</span><span class="s2">:</span><span class="se">\&quot;</span><span class="s2">0x007fd24b891028</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">type</span><span class="se">\&quot;</span><span class="s2">:</span><span class="se">\&quot;</span><span class="s2">STRING</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">class</span><span class="se">\&quot;</span><span class="s2">:</span><span class="se">\&quot;</span><span class="s2">0x007fd24b829658</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">embedded</span><span class="se">\&quot;</span><span class="s2">:true, </span><span class="se">\&quot;</span><span class="s2">bytesize</span><span class="se">\&quot;</span><span class="s2">:1, </span><span class="se">\&quot;</span><span class="s2">value</span><span class="se">\&quot;</span><span class="s2">:</span><span class="se">\&quot;</span><span class="s2">b</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">encoding</span><span class="se">\&quot;</span><span class="s2">:</span><span class="se">\&quot;</span><span class="s2">UTF-8</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">file</span><span class="se">\&quot;</span><span class="s2">:</span><span class="se">\&quot;</span><span class="s2">(irb)</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">line</span><span class="se">\&quot;</span><span class="s2">:3, </span><span class="se">\&quot;</span><span class="s2">method</span><span class="se">\&quot;</span><span class="s2">:</span><span class="se">\&quot;</span><span class="s2">irb_binding</span><span class="se">\&quot;</span><span class="s2">, </span><span class="se">\&quot;</span><span class="s2">generation</span><span class="se">\&quot;</span><span class="s2">:7, </span><span class="se">\&quot;</span><span class="s2">flags</span><span class="se">\&quot;</span><span class="s2">:{</span><span class="se">\&quot;</span><span class="s2">wb_protected</span><span class="se">\&quot;</span><span class="s2">:true, </span><span class="se">\&quot;</span><span class="s2">old</span><span class="se">\&quot;</span><span class="s2">:true, </span><span class="se">\&quot;</span><span class="s2">marked</span><span class="se">\&quot;</span><span class="s2">:true}}</span><span class="se">\n</span><span class="s2">&quot;</span>
</span></code></pre></td></tr></table></div></figure>


<p>We stated earlier that:</p>

<p><strong>Young objects are more likely to reference old objects, than old objects referencing young objects. Old objects also frequently reference other old objects.</strong></p>

<p><em>However</em> it&rsquo;s possible for old objects to reference new objects. What happens when old objects reference new ones?</p>

<p>Old objects with references to new objects are stored in a &ldquo;remembered set&rdquo;. The remembered set is a container of references from old objects to new objects and is a shortcut for preventing heap scans for finding such references.</p>

<h2>Implications for Rails</h2>

<p>As our friend Ezra used to say, &ldquo;no code is faster than no code.&rdquo; The same applies to automatic memory management. Every object allocation also has a variable recycle cost. Allocation generally is low overhead as it happens once, except for the use case where there are no free object slots on the Ruby heap and a major GC is triggered as a result.</p>

<p>A major drawback of this limited segregation of OLD vs YOUNG is that <strong>many transient objects are in fact promoted to old during large contexts such as a Rails request</strong>. These long lived objects eventually become unexpected &ldquo;memory leaks&rdquo;. These transient objects can be conceptually classified as of &ldquo;medium lifetime&rdquo; as they need to stick around for the duration of a request. There&rsquo;s however a large probability that a minor GC would run during request lifetime, promoting young objects to old, effectively increasing their lifetime to well beyond the end of a request. This situation can only be revisited during a major GC which runs infrequently and sweeps both old and young objects.</p>

<p><strong>Each generation can be specifically tweaked, with the older generation being particularly important for balancing total process memory use with maintaining a minimal transient object set (young ones) per request. And subsequent too fast promotion from young to old generation.</strong></p>

<p><em>In our next post we will explore how you&rsquo;d approach tuning the Ruby GC for Rails applications, balancing tradeoffs of speed and memory. Leave your email address below and we&rsquo;ll let you know as soon as it&rsquo;s posted.</em></p>
<div class="footnotes">
<hr/>
<ol>
<li id="fn:1">
<p>See the Ruby Hacking Guide&rsquo;s <a href="https://ruby-hacking-guide.github.io/gc.html">GC chapter</a> for further context and nitty gritty details. I&rsquo;d recommended scanning the content below the first few headings, until turned off by C.<a href="#fnref:1" rev="footnote">&#8617;</a></p></li>
</ol>
</div>

]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Does Rails Scale?]]></title>
    <link href="http://bearmetal.eu//theden/does-rails-scale/"/>
    <updated>2014-12-19T15:22:00+02:00</updated>
    <id>http://bearmetal.eu//theden/does-rails-scale</id>
    <content type="html"><![CDATA[<figure markdown="1">
  <a href="https://www.flickr.com/photos/soctech/43279549/">
    <img src="https://farm1.staticflickr.com/24/43279549_465d50976e_b_d.jpg">
  </a>

  <figcaption>
    <p>
      Photo by <a href="https://www.flickr.com/photos/soctech/43279549/">Soctech</a>
    </p>
  </figcaption>
</figure>


<p>Back when Rails was still not mainstream, a common dismissal by developers using other – more established – technologies was that Rails is cool and stuff, but it will never scale<sup id="fnref:1"><a href="#fn:1" rel="footnote">1</a></sup>. While the question isn&rsquo;t (compared to Rails&#8217; success) as common these days, it still appears in one form or another every once in a while.</p>

<p>Last week on the Ruby on Rails Facebook group, someone asked <a href="https://www.facebook.com/groups/rubyandrails/permalink/10153620823655752/">this question</a>:</p>

<blockquote><p>Can Rails stand up to making a social platform like FB with millions of users using it at the same time?</p>

<p>If so what are the pro&rsquo;s and the cons?</p></blockquote>

<p>So in other words, can Rails scale <em>a lot</em>?</p>

<p>Just as is customary for a Facebook group, the question got a lot of clueless answers. There were a couple of gems like this:</p>

<blockquote><p>Tony if you want to build some thing like FB, you need to learn deeper mountable engine and SOLID anti pattern.</p></blockquote>

<p>The worst however are answers from people who <em>don&rsquo;t know</em> they don&rsquo;t know shit but insist on giving advice that is only bound to either confuse the original poster or lead them astray – and probably both:</p>

<blockquote><p>Twitter is not a good example. They stopped using Rails because it couldn&rsquo;t handle millions of request per second. They began using Scala.</p></blockquote>

<p>This is of course mostly BS with a hint of truth in it, but we&rsquo;ll get back to that in a bit.</p>

<p>The issue with the question itself, is that <em>it&rsquo;s the wrong question to ask</em>, and this has nothing to do with Ruby or Rails per se.</p>

<p>Why is it the wrong question? Let&rsquo;s have a look.</p>

<p>Sure, Ruby is slow in raw performance. It has gotten a lot faster during the past decade, but it is still a highly dynamic interpreted scripting language. Its main shtick has always been programmer happiness, and its primary way to attain that goal has definitely not been to return from that test run as fast as possible. The same goes for Rails.</p>

<p>That said, there are two reasons bare Ruby performance doesn&rsquo;t matter <em>that</em> much. First, it&rsquo;s only a tiny part of the <strong>perceived app performance</strong> for the user. Rails has gone out of its way to automatically make the frontend of your web app performant. This includes frontend caching, asset pipeline, and more opinionated things like Turbolinks. You can of course screw all that up, but you would be amazed how much actual end-user performance you&rsquo;d miss if you&rsquo;d write the same app from scratch – not to mention the time you&rsquo;d waste building it.</p>

<p>Second, and most important for this discussion: <strong>scaling is not the same thing as performance</strong>. Rails has always been built on the <a href="http://en.wikipedia.org/wiki/Shared_nothing_architecture">shared nothing architecture</a>, where in theory the only thing you need to do to scale out your app is to throw more hardware at it – the app should scale linearly. Of course there are limits to this, but they are by no means specific to Rails or Ruby.</p>

<p>Scaling and performance are two separate things. They are related as terms, but not strictly connected. Your app can be very fast for a couple users but awful for a thousand (didn&rsquo;t scale). Or it can scale at O(1) to a million users but loading a page for even a single concurrent user can take 10 seconds (scales but doesn&rsquo;t perform).</p>

<p>Like stated above, a traditional crud-style app on Rails can be made to scale very well by just adding app server instances, cores, and finally physical app servers serving the app. This is what is meant by scaling out<sup id="fnref:2"><a href="#fn:2" rel="footnote">2</a></sup>. Most often the limiting factor here is not Rails but the datastore, which is still often the one shared component in the equation. Scaling the database out is still harder than the appservers, but nevertheless possible. That is way outside the scope of this article, however.</p>

<h2>Is this the right tool for the job?</h2>

<p>It&rsquo;s clear in hindsight that a Rails app wasn&rsquo;t the right tool for what Twitter became – a juggernaut where millions of people were basically realtime chatting with the whole world.</p>

<p>That doesn&rsquo;t mean that Rails wasn&rsquo;t a valid choice for the original app. Maybe it wasn&rsquo;t the best option even then from the technical perspective, but it for sure made developing Twitter a whole lot faster in its initial stages. You know, twitter wasn&rsquo;t the only contender in the microblogging stage in the late naughties. We finns fondly remember Jaiku. Then there was that other San Fransisco startup using Django that I can&rsquo;t even name anymore.</p>

<p>Anyway, the point is that <em>reaching a scale where you have to think harder about scalability is a very, very nice problem to have</em>. Either you built a real business and are making money hand over fist, or you are playing – and winning – the eyeball lotto and have VCs knocking on your door (or, more realistically, have taken on several millions already). The vast majority of businesses <em>never</em> reach this stage.</p>

<p>More likely you just fail in the hockeystick game (the VC option), or perhaps build a sustainable business (the old-fashioned <em>people pay me for helping them kick ass</em> kind). In any case, you won&rsquo;t have to worry about scaling to millions of concurrent users.</p>

<p>Even at the very profitable, high scale SaaS market there are hoards of examples of apps running on Rails. Kissmetrics runs its frontend on Rails, as does GitHub, not to mention Groupon, Livingsocial<sup id="fnref:3"><a href="#fn:3" rel="footnote">3</a></sup>, and many others.</p>

<p>However, at certain scale you have to go for a more modular architecture, SOA if I may. You can use a message queue for message passing, a noSQL db for non-relational and ephemeral data, node.js for realtime apps, and so on. <em>A good tool for every particular sub-task of your app</em>.</p>

<p>That said, you need to keep in mind what I said above. It is pretty unlikely you will ever reach a state where you really need to scale. Thus, thinking about the architecture at the initial stage too much is a form of premature optimization. As long as you don&rsquo;t do anything extra stupid, you can probably get away with a simple Rails app. Because splitting up your app to lots of components early on makes several things harder and more expensive:</p>

<ul>
<li>Complexity of development.</li>
<li>Operating and deploying a bunch of different apps.</li>
<li>Keeping track that all apps are up and running.</li>
<li>Hunting bugs.</li>
<li>Making changes in a lean development environment where things change rapidly</li>
<li>Cognitive cost of understanding and learning how the app works. This is especially true when you&rsquo;re expanding your team.</li>
</ul>


<p>This doesn&rsquo;t mean that at some point you shouldn&rsquo;t do the split. There might be a time where the scale for the points above tips, and a monorail app becomes a burden. But then again, <em>there might not</em>. So do what makes sense now, not what makes sense in your imaginary future.</p>

<p>Of <em>course</em> Rails alone won&rsquo;t scale to a gazillion users for an app it wasn&rsquo;t really meant for to begin with. Neither is it supposed to. However, it is amazing how far you can get with it, just the same way that the old boring PostgreSQL still beats the shit out of its more &ldquo;modern&rdquo; competitors in most common usecases<sup id="fnref:4"><a href="#fn:4" rel="footnote">4</a></sup>.</p>

<h2>Questions you should be asking</h2>

<p>When making a technology decision, instead of “Does is scale?”, here&rsquo;s what you should be asking instead:</p>

<ul>
<li>What is the right tool for the jobs of my app?</li>
<li>How far can I likely get away with a single Rails app?</li>
<li>Will we ever really reach the scale we claim in our investor prospectus? No need to lie to yourself here.</li>
<li>What is more important: getting the app up and running and in front of real users fast, or making it scalable in an imaginary future that may never come?</li>
</ul>


<p>Only after answering those are you equipped to make a decision.</p>

<p>P.S. Reached the point where optimizing Rails and Ruby performance <em>does</em> make a difference? We&rsquo;re writing <a href="https://bearmetal.eu/theden/categories/rails-performance/">a series of articles</a> about just that. Pop your details in the <a href="#mc_embed_signup">form ☟ down there</a> and we&rsquo;ll keep you posted.</p>
<div class="footnotes">
<hr/>
<ol>
<li id="fn:1">
<p>Another good one I heard in EuroOSCON 2005 was that the only thing good about Rails is its marketing.<a href="#fnref:1" rev="footnote">&#8617;</a></p></li>
<li id="fn:2">
<p>Versus scaling up, which means making the single core or thread faster.<a href="#fnref:2" rev="footnote">&#8617;</a></p></li>
<li id="fn:3">
<p>OK, the last two might not pass the profitable bit.<a href="#fnref:3" rev="footnote">&#8617;</a></p></li>
<li id="fn:4">
<p>There are obviously special cases where a single Rails app doesn&rsquo;t cut it even from the beginning. E.g. computationally intensive apps such as Kissmetrics or Skylight.io obviously won&rsquo;t run their stats aggregation processes on Rails.<a href="#fnref:4" rev="footnote">&#8617;</a></p></li>
</ol>
</div>

]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Rails Garbage Collection: naive defaults]]></title>
    <link href="http://bearmetal.eu//theden/rails-garbage-collection-naive-defaults/"/>
    <updated>2014-12-04T17:27:00+02:00</updated>
    <id>http://bearmetal.eu//theden/rails-garbage-collection-naive-defaults</id>
    <content type="html"><![CDATA[<p><a href="https://www.flickr.com/photos/x1klima/13349077375/in/photolist-mkBvCt-9F5bop-psoHyh-6pkzNo-9uDMLx-85EMnZ-ibSsrK-iog9vf-JtxCJ-iohdxP-ibS242-7RtfVT-k1H87W-jNAG6M-oxaFaw-cR3ow7-gEqUsd-6z6KY5-e1m1pQ-diRWXG-i5md69-iogg32-ibSVHi-ibStrn-ibSVUy-n8CpB1-67QKGw-p3qtEX-4THpny-ebLNCE-nycgpC-6U69md-4yXv5b-pTDf3R-861fmQ-6zABJu-3FKVM-nwzafz-6pgrY2-9ejbm6-QuSM-hvn32M-aomUMi-9eebae-b15Lpi-8tBhZj-6o1Xmn-6z3YKz-5s868-61WvU1"><img src="https://farm8.staticflickr.com/7036/13349077375_36fc92ecce_k_d.jpg" alt="" /></a></p>

<p><small>Photo by <a href="https://www.flickr.com/photos/x1klima/13349077375/in/photolist-mkBvCt-9F5bop-psoHyh-6pkzNo-9uDMLx-85EMnZ-ibSsrK-iog9vf-JtxCJ-iohdxP-ibS242-7RtfVT-k1H87W-jNAG6M-oxaFaw-cR3ow7-gEqUsd-6z6KY5-e1m1pQ-diRWXG-i5md69-iogg32-ibSVHi-ibStrn-ibSVUy-n8CpB1-67QKGw-p3qtEX-4THpny-ebLNCE-nycgpC-6U69md-4yXv5b-pTDf3R-861fmQ-6zABJu-3FKVM-nwzafz-6pgrY2-9ejbm6-QuSM-hvn32M-aomUMi-9eebae-b15Lpi-8tBhZj-6o1Xmn-6z3YKz-5s868-61WvU1">martin</a>, used under the Creative Commons license.</small></p>

<p>The vast majority of <a href="http://www.rubyonrails.org">Ruby on Rails</a> applications deploy to production with the vanilla Ruby GC configuration. A conservative combination of growth factors and accounting that &ldquo;works&rdquo; for a demographic from IRB sessions (still my preferred calculator) to massive monolithic Rails apps (the fate of most successful ones). In practice this doesn&rsquo;t work very well, however. It produces:</p>

<ul>
<li>Too aggressive growth of Ruby heap slots and pages when thresholds are reached.</li>
<li>A large ratio of short and medium lived objects in relation to long lived ones for Rails applications.</li>
<li>Too many intermittent major GC cycles during the request / response cycle.</li>
<li>Heap fragmentation.</li>
</ul>


<p>Let&rsquo;s use a metaphor most of us can better relate to: <em>dreaded household chores.</em> Your ability and frequency of hosting dinners at home are limited by four things (takeaways and paper plates aside):</p>

<ul>
<li>How many seats and tables you have</li>
<li>How many sets of clean cutlery, plates and glasses are available</li>
<li>Overhead preparing a particular choice of cuisine</li>
<li>Willingness to clean up and do dishes after</li>
</ul>


<p>This is what you have to work with at home:</p>

<ul>
<li>4 chairs and a table</li>
<li>12 plates and equivalent utensils</li>
<li>83 friends (60 from Facebook, 20 at work, your 2 brothers and then there&rsquo;s Jim)</li>
<li>3 wine glasses and a beer mug</li>
<li>1 bottle of wine and 24 beers<sup id="fnref:1"><a href="#fn:1" rel="footnote">1</a></sup></li>
<li>3 awesome steaks and a piece of tofu</li>
<li>Fresh local produce</li>
</ul>


<p>Some of your friends are also vegetarian.</p>

<p>Let&rsquo;s have a look at two different scenarios.</p>

<h4>IRB scenario</h4>

<p>You&rsquo;ve invited and subsequently prepared dinner and the table—seats, plates and cutlery sets—for four, popped open your bottle of wine and fired up the grill. However, only one friend arrives, quite late. You&rsquo;re grilling steak number three, yet he&rsquo;s the vegetarian…and only drinks beer. And even then doesn&rsquo;t talk very much.</p>

<p>In the end, you down the whole bottle of wine and the three steaks. Life&rsquo;s good again. There&rsquo;s plenty to clean up and pack away, still.</p>

<h4>Rails scenario</h4>

<p>17 guests show up at your door. Half of them are heavily intoxicated because Dylan invited the rest of his wine tasting group, too. Only one eats any of your food, yet breaks four plates. Beer disappeared in three minutes. The group members reveal seven new bottles of wine, make your dog drink one and he kernel panics as a result.</p>

<p>You were not f*cking prepared. At all. Marinated steak&rsquo;s now ruined, there&rsquo;s less inventory and 30+ bottles to recycle. You&rsquo;re hungry and now there are no plates left!</p>

<p>In both of these scenarios, from the perspective of your friends it mostly worked out just fine. It wasn&rsquo;t optimal for you or your environment, though. What&rsquo;s important is that you learned a few things:</p>

<ul>
<li>Next time it&rsquo;s easier to execute optimally, but there may still be a party and some broken plates.</li>
<li>A barbeque for 17 in your one bedroom flat with a George Foreman grill doesn&rsquo;t scale well.</li>
</ul>


<h2>Cooking with Ruby</h2>

<p>In the same manner, different use cases for the Ruby runtime require different preparations. Let&rsquo;s tie the dinner metaphor back to Ruby land and its memory model.</p>

<h4>Home environment</h4>

<p>The Ruby runtime, with everything else inside. Pages, objects and auxilary object data.</p>

<h4>Guest count</h4>

<p>The number of major features and facets you need to support. Gems and engines are good candidates along with individual models, controllers, views etc. These &ldquo;seats&rdquo; are also connected - different guests mingle together.</p>

<h4>Guest distribution</h4>

<p>Rails provides a framework for building applications, thus should be considered as part of the guest list too. Like some family members that make their way to gettogethers. First and second tier cousins you may hear of once a year and never talk with - they&rsquo;re present (consume memory), yet don&rsquo;t always add much value to the ambient.</p>

<h4>Food and drink</h4>

<p>The amount and distribution of objects required to make a feature or facet work. A mix bag of small entrees (embedded objects like 2-char strings), main dishes (a Rails request and all its context) to cocktails and tequila shots (threads!).</p>

<h4>Plates and glasses</h4>

<p>An object slot on the Ruby heap. One String, Array, Hash or any other object. Keep in mind that they can overflow and be recycled too - a wine glass is good for multiple servings. For buffets, a single plate can go very far too :-)</p>

<h4>Tables</h4>

<p>Ruby pages - containers for objects. All of the plates and glasses on a given table. They&rsquo;re mostly prepared in advance, but you can &ldquo;construct&rdquo; and improvise as needed to.</p>

<h4>Type of cuisine</h4>

<p>Some dishes incur a lot of work to prepare <em>and</em> to clean up. Cooked basmati rice will leave a very very different footprint in your kitchen than a paella or salmon option would.</p>

<p>The GC defaults for most Rails applications assume a reasonable sized home environment, a well defined guest list and just enough food and drinks for each. Everyone can sit at the same table, wine and dine on fine dishes, all with a minimal cleanup burden.</p>

<p><em>In reality, it&rsquo;s a frat party. Gone seriously wrong.</em></p>

<p><em>In the next part of this series, we&rsquo;re going to take a look at how the Ruby runtime can better host Rails applications. And what you can optimize for.</em></p>
<div class="footnotes">
<hr/>
<ol>
<li id="fn:1">
<p>Because if there&rsquo;s a promotion, you buy.<a href="#fnref:1" rev="footnote">&#8617;</a></p></li>
</ol>
</div>

]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Metaprogramming Ruby for Greater Good]]></title>
    <link href="http://bearmetal.eu//theden/metaprogramming-ruby-for-greater-good/"/>
    <updated>2014-09-29T11:43:00+03:00</updated>
    <id>http://bearmetal.eu//theden/metaprogramming-ruby-for-greater-good</id>
    <content type="html"><![CDATA[<p><em>This is the transcript of the talk I gave in Reaktor Dev Day in Helsinki, September 26, 2014.</em></p>

<script async class="speakerdeck-embed" data-id="89eb608027ae01321e4c624b84330d5d" data-ratio="1.77777777777778" src="//speakerdeck.com/assets/embed.js"></script>


<p>Thanks, and hi, everyone! It&rsquo;s a real honor to be here. I&rsquo;ve been a big fan of Reaktor for a long time, that is, UNTIL ALL MY GEEK FRIENDS DEFECTED THERE. There&rsquo;s been lots of talk about Rosatom building a new nuclear plant here in Finland. I say fuck that, we already have enough nuclear knowledge locally. But I digress.</p>

<p>I&rsquo;d like to be one of the cool kids and <em>Start with Why</em> just like Simon Sinek told us. However, before that it&rsquo;s worth defining the term metaprogramming in the context of this talk.</p>

<p>What do we mean by metaprogramming? <em>In its simplest form, metaprogramming means code that writes code</em>.</p>

<p>A-ha! So, code generators are metaprogramming, too? Not really. I go with the definition where the code is generated on the fly in the runtime. We could perhaps call it dynamic metaprogramming. This means that most of what I&rsquo;m going to talk about is not possible in a static language.</p>

<p>So a more appropriate definition might be, to quote Paolo Perrotta,</p>

<blockquote><p>Writing code that manipulates language constructs at runtime.</p></blockquote>

<h2>Why?</h2>

<p>But why, I hear you ask. What&rsquo;s in it for me? Well, first of all, because metaprogramming is…</p>

<p>…magic. And magic is good, right? Right? RIGHT? Well, it <em>can</em> be. At least it&rsquo;s cool.</p>

<p>It&rsquo;s also a fun topic for a conference like this, because it&rsquo;s frankly, quite often, mind-boggling. Think of it like this. You take your brain out of your head. You put it in your backpocket. Then you sit on it. <em>Does it bend?</em> If it does, you&rsquo;re talking about metaprogramming.</p>

<p>I also like things that make me scratch my head. I mean, scratching your head is a form of exercise. Just try it yourself. Scratch your head vigorously and your Fitbit will tell you you worked out like crazy today. That&rsquo;s healthy.</p>

<p>But all joking aside, we don&rsquo;t use metaprogramming to be clever, we use it to be flexible. And with Ruby – and any other sufficiently dynamic language – in the end of the day, <em>metaprogramming is just a fancy word for normal, advanced programming</em>.</p>

<h2>Why Ruby?</h2>

<p>So why Ruby? First of all, Ruby is the language I know by far the best. Second, Ruby combines Lisp-like dynamism and flexibility to a syntax that humans can actually decipher.</p>

<p>Like said, in Ruby there&rsquo;s really no distinction between metaprogramming and advanced OO programming in general. Thus, before we go to things that are more literally metaprogramming, let&rsquo;s have a look at Ruby&rsquo;s object model and constructs that lay the groundwork for metaprogramming.</p>

<p>Thus, in a way, this talk can be reduced to <em>advanced OO concepts in Ruby</em>.</p>

<h2>How?</h2>

<p>Before we delve more deeply into the Ruby object model, let&rsquo;s take a step back and have a look at what we mean by object-orientation.</p>

<p>Generally, there are two main approaches to object-oriented programming. By far the most popular is class-based OO, used in languages such as C++ and Java. The other one is prototype-based OO, which is most commonly seen in Javascript. So which of the two does Ruby use?</p>

<p>Class-based? Well, let&rsquo;s have a look.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'>  <span class="n">speaker</span> <span class="o">=</span> <span class="no">Object</span><span class="o">.</span><span class="n">new</span>
</span><span class='line'>  
</span><span class='line'>  <span class="k">def</span> <span class="nc">speaker</span><span class="o">.</span><span class="nf">talk_length</span>
</span><span class='line'>    <span class="vi">@talk_length</span> <span class="o">||=</span> <span class="mi">30</span>
</span><span class='line'>  <span class="k">end</span>
</span><span class='line'>
</span><span class='line'>  <span class="k">def</span> <span class="nc">speaker</span><span class="o">.</span><span class="nf">talk_length</span><span class="o">=</span><span class="p">(</span><span class="n">length</span><span class="p">)</span>
</span><span class='line'>    <span class="vi">@talk_length</span> <span class="o">=</span> <span class="n">length</span>
</span><span class='line'>  <span class="k">end</span>
</span></code></pre></td></tr></table></div></figure>




<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'>  <span class="n">gary</span> <span class="o">=</span> <span class="n">speaker</span><span class="o">.</span><span class="n">clone</span>
</span><span class='line'>  <span class="n">gary</span><span class="o">.</span><span class="n">talk_length</span> <span class="c1"># =&gt; 30</span>
</span><span class='line'>
</span><span class='line'>  <span class="n">gary</span><span class="o">.</span><span class="n">talk_length</span> <span class="o">=</span> <span class="mi">60</span>
</span><span class='line'>  <span class="n">gary</span><span class="o">.</span><span class="n">talk_length</span> <span class="c1"># =&gt; 60</span>
</span><span class='line'>
</span><span class='line'>  <span class="n">scott</span> <span class="o">=</span> <span class="n">gary</span><span class="o">.</span><span class="n">clone</span>
</span><span class='line'>  <span class="n">scott</span><span class="o">.</span><span class="n">talk_length</span> <span class="c1"># =&gt; 60</span>
</span><span class='line'>
</span><span class='line'>  <span class="n">scott</span><span class="o">.</span><span class="n">talk_length</span> <span class="o">=</span> <span class="mi">45</span>
</span><span class='line'>  <span class="n">scott</span><span class="o">.</span><span class="n">talk_length</span> <span class="c1"># =&gt; 45</span>
</span></code></pre></td></tr></table></div></figure>


<p>How&rsquo;s that for prototype-oriented OO in Ruby? But, noone does anything like this with Ruby. <em>No?</em> Just ask the DCI guys. Or, well, ask <a href="https://twitter.com/garybernhardt/status/514413978777563136">Gary about DCI (and Snuggies)</a>.</p>

<p>But I get your point, mainly when you do Ruby programming, you use something that resembles more the good ole class-based OO model. However, in Ruby it comes with a twist – or a dozen.</p>

<h3>Everything is executable</h3>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="k">class</span> <span class="nc">Conference</span>
</span><span class='line'>  <span class="nb">puts</span> <span class="s2">&quot;Hello world (open)&quot;</span>
</span><span class='line'>
</span><span class='line'>  <span class="k">def</span> <span class="nf">venue</span>
</span><span class='line'>  <span class="k">end</span>
</span><span class='line'>
</span><span class='line'>  <span class="c1"># …</span>
</span><span class='line'><span class="k">end</span>
</span><span class='line'>
</span><span class='line'><span class="c1"># Hello world (open)</span>
</span><span class='line'><span class="c1"># =&gt; nil</span>
</span></code></pre></td></tr></table></div></figure>


<p>In Ruby, everything is executable, even the class definitions. But it doesn&rsquo;t end there. What does the following produce?</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="k">class</span> <span class="nc">Conference</span> <span class="o">&lt;</span> <span class="no">Event</span>
</span><span class='line'><span class="k">end</span>
</span></code></pre></td></tr></table></div></figure>




<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'>  <span class="o">&gt;</span> <span class="n">c</span> <span class="o">=</span> <span class="no">Conference</span><span class="o">.</span><span class="n">new</span>
</span><span class='line'>  <span class="o">&gt;</span> <span class="n">c</span><span class="o">.</span><span class="n">class</span>
</span><span class='line'>  <span class="o">=&gt;</span> <span class="no">Conference</span>
</span><span class='line'>  <span class="o">&gt;</span> <span class="n">c</span><span class="o">.</span><span class="n">class</span><span class="o">.</span><span class="n">superclass</span>
</span></code></pre></td></tr></table></div></figure>


<p>Event? Let&rsquo;s see.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="k">class</span> <span class="nc">Conference</span> <span class="o">&lt;</span> <span class="no">Event</span>
</span><span class='line'><span class="k">end</span>
</span></code></pre></td></tr></table></div></figure>




<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'>  <span class="o">&gt;</span> <span class="n">c</span> <span class="o">=</span> <span class="no">Conference</span><span class="o">.</span><span class="n">new</span>
</span><span class='line'>  <span class="o">&gt;</span> <span class="n">c</span><span class="o">.</span><span class="n">class</span>
</span><span class='line'>  <span class="o">=&gt;</span> <span class="no">Conference</span>
</span><span class='line'>  <span class="o">&gt;</span> <span class="n">c</span><span class="o">.</span><span class="n">class</span><span class="o">.</span><span class="n">superclass</span>
</span><span class='line'>  <span class="o">=&gt;</span> <span class="no">ChunkyBacon</span>
</span></code></pre></td></tr></table></div></figure>


<p>Bet you didn&rsquo;t see that coming. Ok, I&rsquo;ll admit, I hid something out of the original listing.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="k">class</span> <span class="nc">ChunkyBacon</span>
</span><span class='line'><span class="k">end</span>
</span><span class='line'>
</span><span class='line'><span class="no">Event</span> <span class="o">=</span> <span class="no">ChunkyBacon</span>
</span><span class='line'>
</span><span class='line'><span class="k">class</span> <span class="nc">Conference</span> <span class="o">&lt;</span> <span class="no">Event</span>
</span><span class='line'><span class="k">end</span>
</span></code></pre></td></tr></table></div></figure>




<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'>  <span class="o">&gt;</span> <span class="n">c</span> <span class="o">=</span> <span class="no">Conference</span><span class="o">.</span><span class="n">new</span>
</span><span class='line'>  <span class="o">&gt;</span> <span class="n">c</span><span class="o">.</span><span class="n">class</span>
</span><span class='line'>  <span class="o">=&gt;</span> <span class="no">Conference</span>
</span><span class='line'>  <span class="o">&gt;</span> <span class="n">c</span><span class="o">.</span><span class="n">class</span><span class="o">.</span><span class="n">superclass</span>
</span><span class='line'>  <span class="o">=&gt;</span> <span class="no">ChunkyBacon</span>
</span></code></pre></td></tr></table></div></figure>


<p>Remember, everything is executable. Thus, this would be just as valid:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="k">def</span> <span class="nf">gimme_a_class</span>
</span><span class='line'>  <span class="o">[</span><span class="nb">Array</span><span class="p">,</span> <span class="no">Hash</span><span class="p">,</span> <span class="nb">String</span><span class="o">][</span><span class="nb">rand</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span><span class="o">]</span>
</span><span class='line'><span class="k">end</span>
</span><span class='line'>
</span><span class='line'><span class="k">class</span> <span class="nc">Conference</span> <span class="o">&lt;</span> <span class="n">gimme_a_class</span><span class="p">()</span>
</span><span class='line'><span class="k">end</span>
</span></code></pre></td></tr></table></div></figure>


<p>Stupid? Yes, but valid nonetheless.</p>

<h3>Open classes</h3>

<p>In Ruby, you can open any class, even the built-in classes, to modify it. This is something that is called monkey-patching, or <a href="http://www.paulirish.com/2010/duck-punching-with-jquery/">duck punching</a> for extra giggles.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="k">class</span> <span class="nc">String</span>
</span><span class='line'>  <span class="n">alias_method</span> <span class="ss">:old_reverse</span><span class="p">,</span> <span class="ss">:reverse</span>
</span><span class='line'>  <span class="k">def</span> <span class="nf">reverse</span>
</span><span class='line'>    <span class="n">old_reverse</span><span class="o">.</span><span class="n">upcase</span>
</span><span class='line'>  <span class="k">end</span>
</span><span class='line'><span class="k">end</span>
</span><span class='line'>
</span><span class='line'><span class="o">&gt;</span> <span class="n">speaker</span> <span class="o">=</span> <span class="s2">&quot;Gary&quot;</span>
</span><span class='line'><span class="o">&gt;</span> <span class="n">speaker</span><span class="o">.</span><span class="n">reverse</span>
</span><span class='line'><span class="o">=&gt;</span> <span class="s2">&quot;YRAG&quot;</span>
</span></code></pre></td></tr></table></div></figure>


<h3>Everything is an Object</h3>

<p>Even methods. Thus you can even do functional style programming with Ruby. Think about it, you can use your favorite language to cook a delicious meal of callback spaghetti. Believe me, I&rsquo;ve tried.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="o">&gt;</span> <span class="nb">String</span><span class="o">.</span><span class="n">instance_method</span><span class="p">(</span><span class="ss">:reverse</span><span class="p">)</span>
</span><span class='line'><span class="o">=&gt;</span> <span class="c1">#&lt;UnboundMethod: String#reverse&gt;</span>
</span></code></pre></td></tr></table></div></figure>


<h3>Classes are Objects, too</h3>

<p><em>Wait, what?</em></p>

<p>But, classes are different, I hear you say. They have class methods, and stuff.</p>

<p>I&rsquo;ll let you into a secret. In Ruby, class methods are just like Ukraine in docent Bäckman&rsquo;s rethoric: they don&rsquo;t really exist. Wanna proof?</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="k">class</span> <span class="nc">Conference</span>
</span><span class='line'>  <span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">in_finland</span>
</span><span class='line'>    <span class="c1"># return conferences in Finland</span>
</span><span class='line'>  <span class="k">end</span>
</span><span class='line'><span class="k">end</span>
</span></code></pre></td></tr></table></div></figure>


<p>Here&rsquo;s an example of a class method in Ruby. Self is the current object, which in the case of a class definition is the class itself. Does this look familiar?</p>

<p>It should.</p>

<h3>Singleton Methods</h3>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'>  <span class="k">def</span> <span class="nc">speaker</span><span class="o">.</span><span class="nf">talk_length</span>
</span><span class='line'>  <span class="vi">@talk_length</span> <span class="o">||=</span> <span class="mi">30</span>
</span><span class='line'>  <span class="k">end</span>
</span></code></pre></td></tr></table></div></figure>


<p>Singleton methods are methods that are defined for a single object, not for the whole object class.</p>

<h3>Ruby method lookup</h3>

<p><img src="http://bearmetal.eu/images/c3ba7a9122156b8a63f17a9ea1744800.png" alt="" /></p>

<p>Above is a simple (and pretty, huh?) diagram of Ruby method lookup. Methods reside in the object&rsquo;s class, right of the object in the image. But where do singleton methods live? They can&rsquo;t sit in the class, since then they&rsquo;d be shared by all the objects of the same class. Neither can they be in the Object class, for the same reason.</p>

<p>Turns out they live in something called a singleton class.</p>

<h3>Singleton class</h3>

<p><img src="http://bearmetal.eu/images/1900b9d0d3a8368f8e56c39d8fc8ebc1.png" alt="Singleton classes in Ruby" /></p>

<p>Singleton class, a.k.a ghost class, metaclass, or eigenclass, is a special case of a class. It&rsquo;s a regular class except for a couple of details:</p>

<ul>
<li>It&rsquo;s hidden from the generic class hierarchy. Thus e.g. the <code>#ancestors</code> method for a class never lists singleton classes.</li>
<li>It cannot be directly inherited.</li>
<li>It only ever has a single instance.</li>
</ul>


<p>So, what are class methods? They&rsquo;re simply singleton methods for the class object itself. And like all singleton methods, they live in the singleton class of the object in question – in this case, the class object. <em>Because classes are just objects themselves</em>.</p>

<p><img src="http://bearmetal.eu/images/05cdd010de1fdf3005147e905f3c9ed7.png" alt="" /></p>

<p>This has an interesting corollary. Singleton classes are classes, and classes are objects, so…</p>

<p>…wait for it…</p>

<p>…a singleton class must have its own singleton class as well.</p>

<p>That&rsquo;s right, it&rsquo;s turtles…errr…singleton classes all the way down. Is it starting to feel like metaprogramming already? We have barely started.</p>

<h3>Generating Code Dynamically in Ruby</h3>

<p>We&rsquo;re going to have a look at four different ways to generate code dynamically in Ruby:</p>

<ul>
<li><code>eval</code></li>
<li><code>instance_eval</code> &amp; <code>class_eval</code></li>
<li><code>define_method</code></li>
<li><code>method_missing</code></li>
</ul>


<h3><code>eval</code></h3>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'>  <span class="n">meth</span> <span class="o">=</span> <span class="s2">&quot;my_method&quot;</span>
</span><span class='line'>  <span class="nb">eval</span> <span class="o">&lt;&lt;-</span><span class="no">END</span>
</span><span class='line'><span class="sh">    def #{meth}</span>
</span><span class='line'><span class="sh">      &quot;foo&quot;</span>
</span><span class='line'><span class="sh">    end</span>
</span><span class='line'><span class="no">  END</span>
</span></code></pre></td></tr></table></div></figure>


<p>Eval is the simplest and barest way to dynamically execute code in Ruby. It takes a string of code and then executes it in the current scope. You can also give eval an explicit scope using a <a href="http://www.ruby-doc.org/core-2.1.3/Binding.html">binding object</a> as the second argument.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="k">def</span> <span class="nf">get_binding</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
</span><span class='line'>  <span class="nb">binding</span>
</span><span class='line'><span class="k">end</span>
</span><span class='line'><span class="nb">eval</span><span class="p">(</span><span class="s1">&#39;a+1&#39;</span><span class="p">,</span> <span class="n">get_binding</span><span class="p">(</span><span class="mi">3</span><span class="p">))</span> <span class="c1"># =&gt; 4, because &#39;a&#39; in the context of get_binding is 3</span>
</span></code></pre></td></tr></table></div></figure>


<p>Eval is super powerful, but has a few huge drawbacks:</p>

<ul>
<li>It messes up syntax highlighting and autocompletion since the code is just a string as far as the editor goes.</li>
<li>It is a giant attack vector for code injection, unless you carefully make sure that no user-submitted data is passed to eval.</li>
</ul>


<p>For these reasons eval has slowly fallen out of favor, but there are still some cases where you have to drop down to bear metal (excuse the pun) means. As a rule of thumb however, you should as a first option resort to one of the following constructs.</p>

<h3><code>instance_eval</code></h3>

<p>Put simply, <code>instance_eval</code> takes a block of code and executes it in the context of the receiving object. It can – just like <code>eval</code> – take a string, but also a real code block:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="n">obj</span><span class="o">.</span><span class="n">instance_eval</span> <span class="k">do</span>
</span><span class='line'>  <span class="nb">self</span> <span class="c1"># =&gt; obj</span>
</span><span class='line'>  <span class="vi">@v</span> <span class="c1"># =&gt; obj&#39;s instance var</span>
</span><span class='line'><span class="k">end</span>
</span></code></pre></td></tr></table></div></figure>


<p>For the reasons above, you should probably use a code block with <code>instance_eval</code> instead of a string of code, unless you know what you&rsquo;re doing and have a good reason for your choice.</p>

<p>A very common usecase for <code>instance_eval</code> is to build domain-specific languages.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<span class='line-number'>12</span>
<span class='line-number'>13</span>
<span class='line-number'>14</span>
<span class='line-number'>15</span>
<span class='line-number'>16</span>
<span class='line-number'>17</span>
<span class='line-number'>18</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="k">class</span> <span class="nc">Turtle</span>
</span><span class='line'>  <span class="k">def</span> <span class="nf">move</span><span class="p">(</span><span class="o">&amp;</span><span class="n">block</span><span class="p">)</span>
</span><span class='line'>    <span class="nb">instance_eval</span><span class="p">(</span><span class="o">&amp;</span><span class="n">block</span><span class="p">)</span>
</span><span class='line'>  <span class="k">end</span>
</span><span class='line'>
</span><span class='line'>  <span class="k">def</span> <span class="nf">right</span><span class="p">(</span><span class="n">n</span><span class="p">);</span> <span class="k">end</span><span class="p">;</span>
</span><span class='line'>  <span class="k">def</span> <span class="nf">left</span><span class="p">(</span><span class="n">n</span><span class="p">);</span> <span class="k">end</span><span class="p">;</span>
</span><span class='line'>  <span class="k">def</span> <span class="nf">up</span><span class="p">(</span><span class="n">n</span><span class="p">);</span> <span class="k">end</span><span class="p">;</span>
</span><span class='line'>  <span class="k">def</span> <span class="nf">down</span><span class="p">(</span><span class="n">n</span><span class="p">);</span> <span class="k">end</span><span class="p">;</span>
</span><span class='line'><span class="k">end</span>
</span><span class='line'>
</span><span class='line'><span class="n">t</span> <span class="o">=</span> <span class="no">Turtle</span><span class="o">.</span><span class="n">new</span>
</span><span class='line'><span class="n">t</span><span class="o">.</span><span class="n">move</span> <span class="k">do</span>
</span><span class='line'>  <span class="n">right</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span>
</span><span class='line'>  <span class="n">up</span>
</span><span class='line'>  <span class="n">left</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
</span><span class='line'>  <span class="n">down</span>
</span><span class='line'><span class="k">end</span>
</span></code></pre></td></tr></table></div></figure>


<h3><code>class_eval</code></h3>

<p><code>class_eval</code> is the sister method for <code>instance_eval</code>. It changes the scope to inside the class definition of the used class. Thus, unlike <code>instance_eval</code>, it can only be called for classes and modules.</p>

<p>Because of this, a bit counterintuitively methods defined inside <code>class_eval</code> will become instance methods for that class&rsquo;s objects, while methods defined inside <code>ClassName.instance_eval</code> will become its class methods.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="o">[</span><span class="nb">String</span><span class="p">,</span> <span class="nb">Array</span><span class="p">,</span> <span class="no">Hash</span><span class="o">].</span><span class="n">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">cls</span><span class="o">|</span>
</span><span class='line'>  <span class="n">cls</span><span class="o">.</span><span class="n">class_eval</span> <span class="p">{</span> <span class="kp">include</span> <span class="no">HelloWorld</span> <span class="p">}</span>
</span><span class='line'><span class="k">end</span>
</span></code></pre></td></tr></table></div></figure>


<h3><code>define_method</code></h3>

<p><code>define_method</code> is the most straightforward and highest-level way to dynamically create new methods. It is just the same as using the normal def syntax, except:</p>

<ul>
<li>With <code>define_method</code> you can set the method name dynamically.</li>
<li>You pass a block to <code>define_method</code> as the method body.</li>
</ul>


<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="k">class</span> <span class="nc">Cat</span> <span class="o">&lt;</span> <span class="no">Animal</span>
</span><span class='line'>  <span class="o">[</span><span class="ss">:leg</span><span class="p">,</span> <span class="ss">:head</span><span class="p">,</span> <span class="ss">:tail</span><span class="o">].</span><span class="n">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">part</span><span class="o">|</span>
</span><span class='line'>    <span class="n">define_method</span><span class="p">(</span><span class="n">part</span><span class="p">)</span> <span class="k">do</span>
</span><span class='line'>      <span class="c1"># …</span>
</span><span class='line'>    <span class="k">end</span>
</span><span class='line'>  <span class="k">end</span>
</span><span class='line'><span class="k">end</span>
</span></code></pre></td></tr></table></div></figure>


<p>It is worth noting that you often use both <code>*_eval</code> and <code>define_method</code> together, e.g. when defining class methods.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="k">class</span> <span class="nc">Cat</span> <span class="o">&lt;</span> <span class="no">Animal</span>
</span><span class='line'>  <span class="nb">instance_eval</span> <span class="k">do</span>
</span><span class='line'>    <span class="o">[</span><span class="ss">:total_number</span><span class="p">,</span> <span class="ss">:sum_of_legs</span><span class="o">].</span><span class="n">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">calc</span><span class="o">|</span>
</span><span class='line'>      <span class="n">define_method</span><span class="p">(</span><span class="n">calc</span><span class="p">)</span> <span class="k">do</span>
</span><span class='line'>        <span class="c1"># creates a class method, such as Cat.total_number</span>
</span><span class='line'>      <span class="k">end</span>
</span><span class='line'>    <span class="k">end</span>
</span><span class='line'>  <span class="k">end</span>
</span><span class='line'><span class="k">end</span>
</span></code></pre></td></tr></table></div></figure>


<h3><code>method_missing</code></h3>

<p><code>method_missing</code> is a special case of dynamic code in Ruby in that it doesn&rsquo;t just by itself generate any dynamic code. However, you can use it to catch method calls that otherwise would go unanswered.</p>

<p><code>method_missing</code> is called for an object when the called method is not found in either the object&rsquo;s class or any of its ancestors. By default <code>method_missing</code> raises a <code>NoMethodError</code>, but you can redefine it for any class to work as you need it to.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<span class='line-number'>12</span>
<span class='line-number'>13</span>
<span class='line-number'>14</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="k">class</span> <span class="nc">Speaker</span>
</span><span class='line'>  <span class="k">def</span> <span class="nf">method_missing</span><span class="p">(</span><span class="n">met</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">)</span>
</span><span class='line'>    <span class="k">if</span> <span class="n">met</span><span class="o">.</span><span class="n">to_s</span> <span class="o">==</span> <span class="s2">&quot;speak&quot;</span>
</span><span class='line'>      <span class="s2">&quot;I might as well say something: </span><span class="si">#{</span><span class="n">args</span><span class="o">[</span><span class="mi">0</span><span class="o">]</span><span class="si">}</span><span class="s2">&quot;</span>
</span><span class='line'>    <span class="k">else</span>
</span><span class='line'>      <span class="k">super</span>
</span><span class='line'>    <span class="k">end</span>
</span><span class='line'>  <span class="k">end</span>
</span><span class='line'><span class="k">end</span>
</span><span class='line'>
</span><span class='line'><span class="n">gary</span> <span class="o">=</span> <span class="no">Speaker</span><span class="o">.</span><span class="n">new</span>
</span><span class='line'><span class="n">gary</span><span class="o">.</span><span class="n">talk</span><span class="p">(</span><span class="s2">&quot;Destroy it&quot;</span><span class="p">)</span> <span class="c1"># =&gt; NoMethodError</span>
</span><span class='line'><span class="n">gary</span><span class="o">.</span><span class="n">speak</span><span class="p">(</span><span class="s2">&quot;Just destroy it!&quot;</span><span class="p">)</span>
</span><span class='line'>  <span class="c1"># =&gt; &quot;I might as well say something: Just destroy it!&quot;</span>
</span></code></pre></td></tr></table></div></figure>


<p><code>method_missing</code> is an example of a hook method in Ruby. Hook methods are similar to event handlers in Javascript in that they are called whenever a certain event (such as an unanswered method call above) happens during runtime. There are a bunch of hook methods in Ruby, but we don&rsquo;t have time to dive deeper into them during this talk.</p>

<p><code>method_missing</code> differs from the previous concepts in this talk in that it doesn&rsquo;t by itself generate new methods. This has two implications:</p>

<ul>
<li>You don&rsquo;t need to know the name of potentially called methods in advance. This can be very powerful in e.g. libraries that talk to external APIs.</li>
<li>You can&rsquo;t introspect the methods caught by <code>method_missing</code>. This means that e.g. <code>#instance_methods</code> won&rsquo;t return the “ghost methods” that only <code>method_missing</code> catches. Likewise, <code>#respond_to?</code> will return false regardless of whether <code>method_missing</code> would have caught the call or not, unless you also overwrite the <a href="http://ruby-doc.org/core-2.1.3/Object.html#method-i-respond_to_missing-3F"><code>respond_to_missing?</code></a> method to be aware of the ghost method.</li>
</ul>


<h3>Example: <code>attr_accessor</code> Rewritten in Ruby</h3>

<p>To top off this talk, we&rsquo;re going to combine the topics we have learned so far to do a simple exercise. Namely, we&rsquo;re going to rewrite a simple Ruby language construct ourself, in pure Ruby.</p>

<p>Ruby  has a simple construct called <code>attr_accessor</code> that creates getter and setter methods for named instance variables of the class&rsquo;s object.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="k">class</span> <span class="nc">Animal</span>
</span><span class='line'>  <span class="kp">attr_accessor</span> <span class="ss">:legs</span><span class="p">,</span> <span class="ss">:diet</span>
</span><span class='line'><span class="k">end</span>
</span><span class='line'>
</span><span class='line'><span class="n">cat</span> <span class="o">=</span> <span class="no">Animal</span><span class="o">.</span><span class="n">new</span>
</span><span class='line'><span class="n">cat</span><span class="o">.</span><span class="n">legs</span> <span class="o">=</span> <span class="mi">4</span>
</span><span class='line'><span class="n">cat</span><span class="o">.</span><span class="n">legs</span> <span class="c1"># =&gt; 4</span>
</span><span class='line'><span class="n">cat</span><span class="o">.</span><span class="n">diet</span> <span class="o">=</span> <span class="s2">&quot;Small birds&quot;</span>
</span><span class='line'><span class="n">cat</span><span class="o">.</span><span class="n">diet</span> <span class="c1"># =&gt; &quot;Small birds&quot;</span>
</span></code></pre></td></tr></table></div></figure>


<p>While <code>attr_accessor</code> above looks like some kind of keyword, it is actually just a call to a class method<sup id="fnref:1"><a href="#fn:1" rel="footnote">1</a></sup>. Remember, the whole class definition is executable code and <code>self</code> inside the class definition is set to the class itself. Thus, the line is the same as:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="no">Animal</span><span class="o">.</span><span class="n">attr_accessor</span> <span class="ss">:legs</span><span class="p">,</span> <span class="ss">:diet</span>
</span></code></pre></td></tr></table></div></figure>


<p>So, how to add the method to the class?</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<span class='line-number'>12</span>
<span class='line-number'>13</span>
<span class='line-number'>14</span>
<span class='line-number'>15</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'>  <span class="k">class</span> <span class="nc">Animal</span>
</span><span class='line'>    <span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">nattr_accessor</span><span class="p">(</span><span class="o">*</span><span class="n">meths</span><span class="p">)</span>
</span><span class='line'>      <span class="n">meths</span><span class="o">.</span><span class="n">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">meth</span><span class="o">|</span>
</span><span class='line'>        <span class="c1"># getter</span>
</span><span class='line'>        <span class="n">define_method</span><span class="p">(</span><span class="n">meth</span><span class="p">)</span> <span class="k">do</span>
</span><span class='line'>          <span class="nb">instance_variable_get</span><span class="p">(</span><span class="s2">&quot;@</span><span class="si">#{</span><span class="n">meth</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">)</span>
</span><span class='line'>        <span class="k">end</span>
</span><span class='line'>
</span><span class='line'>        <span class="c1"># setter</span>
</span><span class='line'>        <span class="n">define_method</span><span class="p">(</span><span class="s2">&quot;</span><span class="si">#{</span><span class="n">meth</span><span class="si">}</span><span class="s2">=&quot;</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">wut</span><span class="o">|</span>
</span><span class='line'>          <span class="nb">instance_variable_set</span><span class="p">(</span><span class="s2">&quot;@</span><span class="si">#{</span><span class="n">meth</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">wut</span><span class="p">)</span>
</span><span class='line'>        <span class="k">end</span>
</span><span class='line'>      <span class="k">end</span>
</span><span class='line'>    <span class="k">end</span>
</span><span class='line'>  <span class="k">end</span>
</span></code></pre></td></tr></table></div></figure>


<p>  In the code above we define a new class method, <code>nattr_accessor</code><sup id="fnref:2"><a href="#fn:2" rel="footnote">2</a></sup>. Then we iterate over all the method names the method is called with<sup id="fnref:3"><a href="#fn:3" rel="footnote">3</a></sup>. For each method, we use <code>define_method</code> twice, to generate both the getter and setter methods. Inside them, we use the <code>instance_variable_get</code> and <code>instance_variable_get</code> methods to dynamically get and set the variable value. Using these methods we can again avoid having to evaluate a string of code, the way as with using <code>define_method</code>.</p>

<p> Let&rsquo;s now take a look whether our code works:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'>
</span><span class='line'>  <span class="k">class</span> <span class="nc">Cat</span> <span class="o">&lt;</span> <span class="no">Animal</span>
</span><span class='line'>    <span class="n">nattr_accessor</span> <span class="ss">:legs</span><span class="p">,</span> <span class="ss">:diet</span>
</span><span class='line'>  <span class="k">end</span>
</span><span class='line'>
</span><span class='line'>  <span class="n">c</span> <span class="o">=</span> <span class="no">Cat</span><span class="o">.</span><span class="n">new</span>
</span><span class='line'>  <span class="n">c</span><span class="o">.</span><span class="n">legs</span> <span class="o">=</span> <span class="mi">4</span>
</span><span class='line'>  <span class="n">c</span><span class="o">.</span><span class="n">diet</span> <span class="o">=</span> <span class="s2">&quot;Small birds&quot;</span>
</span><span class='line'>  <span class="n">c</span><span class="o">.</span><span class="n">legs</span> <span class="c1"># =&gt; 4</span>
</span><span class='line'>  <span class="n">c</span><span class="o">.</span><span class="n">diet</span> <span class="c1"># =&gt; &quot;Small birds&quot;</span>
</span></code></pre></td></tr></table></div></figure>


<p>But what if we want to make the method more reusable? Where should it go then?</p>

<p>We could obviously put it into the <code>Object</code> class:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<span class='line-number'>12</span>
<span class='line-number'>13</span>
<span class='line-number'>14</span>
<span class='line-number'>15</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'>  <span class="k">class</span> <span class="nc">Object</span>
</span><span class='line'>    <span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">nattr_accessor</span><span class="p">(</span><span class="o">*</span><span class="n">meths</span><span class="p">)</span>
</span><span class='line'>      <span class="n">meths</span><span class="o">.</span><span class="n">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">meth</span><span class="o">|</span>
</span><span class='line'>        <span class="c1"># getter</span>
</span><span class='line'>        <span class="n">define_method</span><span class="p">(</span><span class="n">meth</span><span class="p">)</span> <span class="k">do</span>
</span><span class='line'>          <span class="nb">instance_variable_get</span><span class="p">(</span><span class="s2">&quot;@</span><span class="si">#{</span><span class="n">meth</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">)</span>
</span><span class='line'>        <span class="k">end</span>
</span><span class='line'>
</span><span class='line'>        <span class="c1"># setter</span>
</span><span class='line'>        <span class="n">define_method</span><span class="p">(</span><span class="s2">&quot;</span><span class="si">#{</span><span class="n">meth</span><span class="si">}</span><span class="s2">=&quot;</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">wut</span><span class="o">|</span>
</span><span class='line'>          <span class="nb">instance_variable_set</span><span class="p">(</span><span class="s2">&quot;@</span><span class="si">#{</span><span class="n">meth</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">wut</span><span class="p">)</span>
</span><span class='line'>        <span class="k">end</span>
</span><span class='line'>      <span class="k">end</span>
</span><span class='line'>    <span class="k">end</span>
</span><span class='line'>  <span class="k">end</span>
</span></code></pre></td></tr></table></div></figure>


<p>But what if we don&rsquo;t want it everywhere, cluttering the inheritance chain? Let&rsquo;s put it in a module and reuse it where needed.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<span class='line-number'>12</span>
<span class='line-number'>13</span>
<span class='line-number'>14</span>
<span class='line-number'>15</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'>  <span class="k">module</span> <span class="nn">Nattr</span>
</span><span class='line'>    <span class="k">def</span> <span class="nf">nattr_accessor</span><span class="p">(</span><span class="o">*</span><span class="n">meths</span><span class="p">)</span>
</span><span class='line'>      <span class="n">meths</span><span class="o">.</span><span class="n">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">meth</span><span class="o">|</span>
</span><span class='line'>        <span class="c1"># getter</span>
</span><span class='line'>        <span class="n">define_method</span><span class="p">(</span><span class="n">meth</span><span class="p">)</span> <span class="k">do</span>
</span><span class='line'>          <span class="nb">instance_variable_get</span><span class="p">(</span><span class="s2">&quot;@</span><span class="si">#{</span><span class="n">meth</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">)</span>
</span><span class='line'>        <span class="k">end</span>
</span><span class='line'>
</span><span class='line'>        <span class="c1"># setter</span>
</span><span class='line'>        <span class="n">define_method</span><span class="p">(</span><span class="s2">&quot;</span><span class="si">#{</span><span class="n">meth</span><span class="si">}</span><span class="s2">=&quot;</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">wut</span><span class="o">|</span>
</span><span class='line'>          <span class="nb">instance_variable_set</span><span class="p">(</span><span class="s2">&quot;@</span><span class="si">#{</span><span class="n">meth</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">wut</span><span class="p">)</span>
</span><span class='line'>        <span class="k">end</span>
</span><span class='line'>      <span class="k">end</span>
</span><span class='line'>    <span class="k">end</span>
</span><span class='line'>  <span class="k">end</span>
</span></code></pre></td></tr></table></div></figure>


<p>Now we can use it our class:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="k">class</span> <span class="nc">Animal</span>
</span><span class='line'>  <span class="kp">include</span> <span class="no">Nattr</span>
</span><span class='line'>  <span class="n">nattr_accessor</span> <span class="ss">:legs</span><span class="p">,</span> <span class="ss">:heads</span>
</span><span class='line'><span class="k">end</span>
</span><span class='line'>
</span><span class='line'><span class="c1"># =&gt; NoMethodError: undefined method `nattr_accessor&#39; for Animal:Class</span>
</span><span class='line'><span class="n">from</span> <span class="p">(</span><span class="n">pry</span><span class="p">):</span><span class="mi">63</span><span class="ss">:in</span> <span class="sb">`&lt;class:Animal&gt;&#39;</span>
</span></code></pre></td></tr></table></div></figure>


<p>Oops. What happened?</p>

<p>We used include to get the Nattr module into Animal. However, include will take the methods in the module and make them <em>instance methods</em> of the including class. However, we need the method as a class method. What to do?</p>

<p>Fortunately, Ruby has a similar method called <code>extend</code>. It works the same way as include, except that it makes the methods from the module class methods<sup id="fnref:4"><a href="#fn:4" rel="footnote">4</a></sup> of our Animal class.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<span class='line-number'>12</span>
<span class='line-number'>13</span>
</pre></td><td class='code'><pre><code class='ruby'><span class='line'><span class="k">class</span> <span class="nc">Animal</span>
</span><span class='line'>  <span class="kp">extend</span> <span class="no">Nattr</span>
</span><span class='line'><span class="k">end</span>
</span><span class='line'>
</span><span class='line'><span class="k">class</span> <span class="nc">Cat</span> <span class="o">&lt;</span> <span class="no">Animal</span>
</span><span class='line'>  <span class="n">nattr_accessor</span> <span class="ss">:legs</span><span class="p">,</span> <span class="ss">:diet</span>
</span><span class='line'><span class="k">end</span>
</span><span class='line'>
</span><span class='line'><span class="n">c</span> <span class="o">=</span> <span class="no">Cat</span><span class="o">.</span><span class="n">new</span>
</span><span class='line'><span class="n">c</span><span class="o">.</span><span class="n">legs</span> <span class="o">=</span> <span class="mi">4</span>
</span><span class='line'><span class="n">c</span><span class="o">.</span><span class="n">diet</span> <span class="o">=</span> <span class="s2">&quot;Mice&quot;</span>
</span><span class='line'><span class="n">c</span><span class="o">.</span><span class="n">legs</span>
</span><span class='line'><span class="n">c</span><span class="o">.</span><span class="n">diet</span>
</span></code></pre></td></tr></table></div></figure>


<p><em>Now we&rsquo;re talking.</em></p>

<h3>Problems with metaprogramming</h3>

<p>Lemme tell you a story. About a dozen or so years ago I was living in Zürich, as an exchange student. I hadn&rsquo;t yet found a permanent apartment so I was living at some friends&#8217; place while they were abroad. A permanent internet connection wasn&rsquo;t an ubiquitous thing back then, and the Swiss aren&rsquo;t big into tv&rsquo;s, so I had to figure out things to do at nights. I was living alone, and as a somewhat geeky guy I wasn&rsquo;t that much into social life. Thus, I mostly read at nights. I had just found this <a href="http://www.joelonsoftware.com">Joel guy and a shitload of his writings</a>, so I used the printers at the university to print on the thin brownish paper (hey, it was free!) his somewhat ranting articles and then spent nights reading about <a href="http://www.joelonsoftware.com/articles/CamelsandRubberDuckies.html">camels and rubber duckies</a>, the <a href="http://www.joelonsoftware.com/articles/fog0000000043.html">Joel test</a>, – and <a href="http://www.joelonsoftware.com/articles/LeakyAbstractions.html">leaky abstractions</a>. And that is what metaprogramming in many cases is: an abstraction.</p>

<p>Now, there is nothing inherently wrong with abstractions – otherwise we&rsquo;d all be programming in Assembler – but we&rsquo;ll have to keep in mind that they always come at a cost. So keep in mind that metaprogramming is a super powerful tool to reduce duplication and to add power to your code, but you do have to pay a price for it.</p>

<p>Using too much metaprogramming, your code can become harder to:</p>

<ul>
<li>read,</li>
<li>debug, and</li>
<li>search for.</li>
</ul>


<p>So use it as any powerful but potentially dangerous tool: start simply but when the complexity gets out of hand, sprinkle some metaprogramming magic dust to get back on the driver&rsquo;s seat. Never use metaprogramming just for the sake of metaprogramming.</p>

<p>As Dave Thomas once said:</p>

<blockquote><p>“The only thing worth worrying about when looking at code is &lsquo;is it easy to change?&#8217;”</p></blockquote>

<p>Keep this in mind. Will metaprogramming make your code easier to change in this particular case? If yes, go for it. If not, don&rsquo;t bother.</p>

<h3>Where now?</h3>

<p>We&rsquo;ve only had time to scratch the surface of Ruby object model and metaprogramming. It&rsquo;s a fractal of sometimes mind-boggling stuff, which also makes it so interesting. If you want to take the next steps in you advanced Ruby object model and metaprogramming knowledge, I&rsquo;d recommend checking out the following:</p>

<ul>
<li><a href="https://pragprog.com/screencasts/v-dtrubyom/the-ruby-object-model-and-metaprogramming">Dave Thomas&rsquo;s screencasts at Prag Prog</a>. They&rsquo;re a bit dated as in they cover Ruby 1.8. However, <em>not that much</em> has changed since then. Watching them also makes you feel good because you can see the great Prag Dave use Textmate, make mistakes, and delete characters in the code one by one.</li>
<li>Paolo Perrotta&rsquo;s <a href="https://pragprog.com/book/ppmetr2/metaprogramming-ruby-2">Metaprogramming Ruby</a> was just updated to cover the latest Ruby and Rails versions. It&rsquo;s a very down-to-earth and easy read of a sometimes intimidating subject.</li>
<li>If you already think you know everything about the subject, I&rsquo;d recommend checking out Pat Shaughnessy&rsquo;s <a href="http://patshaughnessy.net/ruby-under-a-microscope">Ruby Under a Microscope</a>. It goes down to the level of how the Ruby object model is implemented in C (yeah, really), while still being an entertaining read.</li>
<li>Last but not least, read the source, Luke. Any non-trivial Ruby application is bound to have more than its share of metaprogramming sprinkled into it. Because, <strong>in Ruby, metaprogramming is just programming.</strong></li>
</ul>

<div class="footnotes">
<hr/>
<ol>
<li id="fn:1">
<p>Yeah, I know, singleton method of the class itself.<a href="#fnref:1" rev="footnote">&#8617;</a></p></li>
<li id="fn:2">
<p>Let&rsquo;s name it something other than the built-in method just to avoid name collisions and nasty surprises.<a href="#fnref:2" rev="footnote">&#8617;</a></p></li>
<li id="fn:3">
<p>The asterisk before the parameter name means that we can have a number of arguments, each of which will be passed to the method in an array called <code>meths</code>.<a href="#fnref:3" rev="footnote">&#8617;</a></p></li>
<li id="fn:4">
<p>Technically, it opens up the singleton class of the Animal class and throws the methods in there. Thus they&rsquo;ll become singleton methods for the Animal class, just like we want them to.<a href="#fnref:4" rev="footnote">&#8617;</a></p></li>
</ol>
</div>

]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Do you know the biggest reason for why enterprise software sucks?]]></title>
    <link href="http://bearmetal.eu//theden/do-you-know-the-biggest-reason-why-enterprise-software-sucks/"/>
    <updated>2014-01-15T11:43:00+02:00</updated>
    <id>http://bearmetal.eu//theden/do-you-know-the-biggest-reason-why-enterprise-software-sucks</id>
    <content type="html"><![CDATA[<p><em>We all know the story. Your company was going to get this big new shiny ERP software. It was going to replace a third of the workforce in the company, cut the costs in half and make everyone happy. In reality the project went two years over schedule, cost three times as much as envisioned, and the end result was a steaming pile of shit.</em></p>

<p><a href="http://www.flickr.com/photos/53326337@N00/8043877054/"><img src="https://farm9.staticflickr.com/8453/8043877054_883963cf80_c.jpg" alt="" /></a></p>

<p><small>Photo by <a href="http://www.flickr.com/photos/53326337@N00/8043877054/">Quinn Dombrowski</a>, used under the Creative Commons license.</small></p>

<p>At this point started the blame-throwing. The provider duped the client with waterfall and exorbitant change fees. The buyer didn&rsquo;t know how to act as a client in an information system project. The specs weren&rsquo;t good/detailed/strict/loose enough. The consultants just weren&rsquo;t that good in the first place. On and on and on.</p>

<p>While one or more of the above invariably are true in failed software projects, there&rsquo;s one issue that almost each and every failed enterprise software project has in common: <em>the buyers were not (going to be) the users of the software</em>.</p>

<p>This simple fact has huge implications. Ever heard that “the client didn&rsquo;t really know what they wanted”? Well, that&rsquo;s because they didn&rsquo;t. Thus, most such software projects are built with something completely different than the end user in mind. Be it the ego of the CTO, his debt to his mason brothers who happen to be in the software business<sup id="fnref:1"><a href="#fn:1" rel="footnote">1</a></sup>, or just the cheapest initial bid<sup id="fnref:2"><a href="#fn:2" rel="footnote">2</a></sup>. In any case, it&rsquo;s in the software provider&rsquo;s best interest to appeal to the decisionmaker, not the people actually using the system.</p>

<p>Of course, not every software buyer is as bad as described above. Many truly care about the success of the system and even its users. If for no other reason, at least because it has a direct effect on the company&rsquo;s bottom line. But even then, they just don&rsquo;t have the first-hand experience of working in the daily churn. They simply can&rsquo;t know what&rsquo;s best for the users. Of course, this gets even worse in the design-by-committee, big-spec-upfront projects.</p>

<p>Since it&rsquo;s not very likely that we could change the process of making large software project purchases any time soon, what can we as software vendors do? One word: <em>empathy</em>. If you just take a spec and implement it with no questions asked, shame on you. You deserve all the blame. Your job is not to implement what the spec says. Heck, your job isn&rsquo;t even to create what the client wants. Your job is to build what the client – no, the end users – need. For this – no matter how blasphemous it might sound to an engineer – you have to actually <em>talk</em> to the people that will be using your software.</p>

<p>This is why it&rsquo;s so important to put the software developers to actually do what the end-users would. <strong>If you&rsquo;re building call-center software, make the developers work in the call center a day or a week. If you&rsquo;re building web apps, make the developers and designers work the support queue, don&rsquo;t just outsource it to India.</strong></p>

<p>There is no better way to understand the needs for software you&rsquo;re building than to talk directly to its users or use it yourself for real, in a real-life situation. While there aren&rsquo;t that many opportunities to dog-fooding when building (perhaps internal) enterprise software for a client, there&rsquo;s nothing preventing you from sending your people to the actual cost center. Nothing will give as much insight to the needs and pains of the actual users. No spec will ever give you as broad a picture. No technical brilliance will ever make up for lacking domain knowledge. And no client will ever love you as much as the one in the project where you threw yourself (even without being asked) on the line of fire. That&rsquo;s what we here at Bear Metal insist on doing at the start of every project. I think you should, too.</p>

<hr />

<p><em>We at Bear Metal have some availability open for short and mid-term projects. If you&rsquo;re looking for help building, running, scaling or marketing your web app,  <a href="mailto:info@bearmetal.eu">get in touch</a>.</em></p>
<div class="footnotes">
<hr/>
<ol>
<li id="fn:1">
<p>It&rsquo;s surprising how often the same people actually represent both the buyer and the seller. This happens all the time e.g. in the patient care systems projects.<a href="#fnref:1" rev="footnote">&#8617;</a></p></li>
<li id="fn:2">
<p>Nevermind that the cheapest initial bid almost always balloons to something completely different in the end.<a href="#fnref:2" rev="footnote">&#8617;</a></p></li>
</ol>
</div>

]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Let Your Data Tell a Story]]></title>
    <link href="http://bearmetal.eu//theden/let-your-data-tell-a-story/"/>
    <updated>2013-09-24T16:57:00+03:00</updated>
    <id>http://bearmetal.eu//theden/let-your-data-tell-a-story</id>
    <content type="html"><![CDATA[<script async class="speakerdeck-embed" data-id="39c500c005dc01314f705605cd4f70bc" data-ratio="1.37081659973226" src="//speakerdeck.com/assets/embed.js"></script>


<p><em>This is a talk I gave at Monitorama.eu in Berlin, September 19, 2013.</em></p>

<p>Did you know that bear is Bär in German? Which, on the other hand, is berry in Swedish, and bears obviously eat berries as breakfast. Meanwhile, a berry is Beer in German, which does sound very German when you think about it. But I&rsquo;m already digressing.</p>

<p>Germans, and the Berliner especially, are of course very fond of bears, which is the only explanation I could come up with for why I was chosen<sup id="fnref:1"><a href="#fn:1" rel="footnote">1</a></sup> to give this talk here. In particular, they like polar bears here – Eisbären in the local lingo. But it wasn&rsquo;t always like that.</p>

<p>In 1930 in Stuttgart, an innkeeper threw a large feast serving smoked polar bear ham. The result: 100 falling ill and 13 dead because of trichinosis caused by <a href="https://en.wikipedia.org/wiki/Trichinella_spiralis">Trichinella spiralis</a>, a little fella that looks like this:</p>

<p><img src="https://upload.wikimedia.org/wikipedia/commons/e/e8/Trichinella_larv1_DPDx.JPG" alt="" /></p>

<p>The moral of the story: always cook your bear meat well done. And now, after hearing this tale, I&rsquo;ll guarantee you, you will remember it every time you&rsquo;re cooking polar bear meat. And <em>that</em> is the power of a story.</p>

<p>We&rsquo;ll get back to the topic of storytelling in a little bit, but let&rsquo;s first have a quick look at what we know about the human brain and mind.</p>

<h2>Duality of the mind – a farce in 2 acts</h2>

<h3>Act 1: Brain pixels and the human RAM</h3>

<p>In 1998, psychologists Daniel Simons and Daniel Levin carried out <a href="https://www.youtube.com/watch?v=FWSxSQsspiQ">an experiment</a>. They hired a professional actor to approach people walking on the street and ask them to give them route instructions on a map. While the targets were looking at the map intensely, something weird happened. Two workmen carrying a door walked between the helper and the actor. The door, of course, was smoke and mirrors. Behind it, the person who had asked for help, was swapped to another person. Most of the targets did not notice. The actor was swapped to another with different hair color, then different clothes, and finally from a man to a woman. And yet, more than half of the subjects failed to notice that they were talking to a completely different person.</p>

<p>What this tells us is that our attention is very, very limited. This comes mostly down to two things.</p>

<h4>Our focus of vision is tiny</h4>

<p>The human vision is a bit like a digital camera. Light is directed through a lens to a “sensor”, the retina. However, this human CMOS is nothing like the one made of silicon. While a digital camera sensor has an even grid of pixels, the brain pixels are anything but. In the center of our vision, called as fovea, we can resolve as much as 100 pixels in the area of a needle pin, at arm&rsquo;s length. This is more or less where the so-called retina screen resolution comes from.</p>

<p>However, the fovea, at that same length, is only about the size of a thumbnail. Outside that, the “pixel density” goes down really fast. In the periphery of our vision, we can&rsquo;t really detect any details at all.</p>

<p><a href="http://www.flickr.com/photos/jarkko/9911880963/"><img src="https://farm6.staticflickr.com/5502/9911880963_2790398f39_h.jpg" alt="" /></a></p>

<p>The obvious question here is, how then can we process a more or less sharp image of our surroundings? The answer is: we don&rsquo;t. But we cheat. We move our eyes rapidly to scan our vision, which creates an illusion of a sharper image than it really is.</p>

<p>But this isn&rsquo;t such an issue, is it? I mean, we can just memorize what we just saw to create a more comprehensive picture of what we just saw. Right? Well, yes and no.</p>

<h4>Our working memory is very small</h4>

<p>We can, indeed, store items in what is called short-term or working memory. To stay in computer metaphors<sup id="fnref:2"><a href="#fn:2" rel="footnote">2</a></sup>, it is a bit like RAM. It is fast, but limited, and when something new goes into it and it gets full, something else must be thrown out. However, unlike its tech counterpart, working memory in us humans has not grown during the last years or even centuries. It is still ridiculously small: somewhere around 3-4. No, I don&rsquo;t mean 3-4 gigs, or even megs. Hell, not even kilobytes or bytes. 3-4, period.</p>

<p>Let&rsquo;s look at a short demo video of this. Please don&rsquo;t continue reading this article further before you have watched it. It takes less than two minutes.</p>

<iframe width="560" height="315" src="//www.youtube.com/embed/IGQmdoK_ZfY" frameborder="0" allowfullscreen></iframe>


<p>Did you notice the gorilla (or one of the other changes if you had seen the original gorilla video beforehand)? About 50% of people don&rsquo;t, even though they are looking several times (this was proven with eye tracking equipment) right at the beast, which is quite an amazing demonstration of the limits of our attention.</p>

<p>So what does this lack of attention mean to us as graphic and visualization designers? To put it short, it means the world. As an example, you can&rsquo;t put two things the viewer should be comparing against each other very far from each other, because the viewer just can&rsquo;t keep the other one in her memory long enough to make the comparison. Thus the first rule of thumb is: <em>make comparisons viewable with as few eye fixations as possible, preferably one</em>.</p>

<p>The second rule is: <em>maximize the data-ink ratio</em>. The ratio, coined by the visualization guru Edward Tufte, means the amount of data conveyed by the visualization divided by the amount of “ink” used. To put it in another way, the less stuff you have that is only there for visual looks and doesn&rsquo;t have any particular meaning, the better. Good examples of this are needless and redundant color coding, the infamous PowerPoint pattern backgrounds<sup id="fnref:3"><a href="#fn:3" rel="footnote">3</a></sup>, and 3D effects now running amok in the Keynote world. Each of these makes the cognitive load of the viewer higher by fighting for her attention, which then leaves fewer resources in her brain left to actually make sense of the <em>real</em> information in the graph.</p>

<p>The whole field of human attention and cognitive science is huge both in general and applied to visuals in particular. We don&rsquo;t have the opportunity to delve into it deeper here, but here are some pointers for you to learn more:</p>

<ul>
<li><a href="http://www.interaction-design.org/encyclopedia/data_visualization_for_human_perception.html">Data Visualization for Human Perception</a> by Stephen Few</li>
<li><a href="http://www.amazon.com/Visual-Thinking-Kaufmann-Interactive-Technologies/dp/0123708966">Visual Thinking: For Design</a> by Colin Ware</li>
<li><a href="http://www.amazon.com/Invisible-Gorilla-Intuition-Deceives-ebook/dp/B003P9XE56/">The Invisible Gorilla</a> by Christopher Chabris and Daniel Simons</li>
</ul>


<h3>Act 2: Thinking, fast and slow</h3>

<p>In The Magazine, one the several things Marco Arment has sold during the past year, pediatrician Saul Hymes recently wrote an article called <a href="http://the-magazine.org/15/give-it-your-best-shot">Give It Your Best Shot</a>. In the article, Hymes writes about one of his patients, a three-week-old girl who went dead because of bacterial meningitis, an illness passed to him by her unvaccinated older brother.</p>

<p>It was all of course preventable. There has been a vaccine against the bacteria in question, Haemophilus influenzae type b since 1984. So afterwards Hymes asked the mother of the two whether she&rsquo;d now “give her children the benefit of modern medicine’s vaccinations.”. The answer was no.</p>

<p>What&rsquo;s going on here?</p>

<p>In his best-selling book, <a href="http://www.amazon.com/Thinking-Fast-and-Slow-ebook/dp/B00555X8OA/">Thinking, fast and slow</a>, the Nobel laureate psychologist Daniel Kahneman lays out his theory of human thinking, splitting it into two systems, which he calls quite unimagitatively systems 1 and 2. System 1 is fast, intuitive, automatic and direct. System 2 is slow, analytical, and not activated in many day-to-day tasks at all. It is also lazy, trusting the intuition of system 1 much more than it should. It wouldn&rsquo;t be such a problem if system 1 wasn&rsquo;t as prone to many errors and biases as it is. It draws conclusions long before the conscious mind does. What makes matters worse, we almost always think we made these intuitive, erroneous decisions knowingly.</p>

<p>And this, in many ways, is what is going on in the heads of the people in the anti-vaccination community. Let&rsquo;s look at some of the biases potentially at play here.</p>

<h4>Availability Bias</h4>

<blockquote><p><em>We prefer wrong information to no information.</em></p>

<p>– Rolf Dobelli in <a href="http://www.amazon.com/The-Thinking-Clearly-Rolf-Dobelli/dp/0062219685">The Art of Thinking Clearly</a></p></blockquote>

<p>Because of information readily available to us, we often make totally erroneous assumptions of how common or proven it actually is. If our grandfather smoked a lot but still lived to 100-years-old, we easily think that smoking can&rsquo;t be that bad for you. Or if a celebrity in the TV claims that her son got autism from vaccinations, hey, why not? We use statements like these to prove something, but they don&rsquo;t of course prove anything. <strong>The plural form of ‘anecdote’ is not ‘data’.</strong></p>

<p>Because of availability bias, we systematically overestimate the risk of catastrophes we see often in the media, such as terrorist attacks or natural disasters, and underestimate the boring, but much more likely causes of death, such as diabetes and cancers. We attach much more likelihood to spectacular outcomes. And what could be more spectacular than a centerfold model and her son with an illness obviously caused by greedy pharma companies and their conspiracies with public health organizations?</p>

<h4>Conjunction Fallacy</h4>

<p>Conjunction fallacy means that the more vividly something is presented, the more likely it is for us to believe it is the truth. At intuitive level, we have a soft spot for plausible stories.</p>

<p>So when Jenny McCarthy goes to Oprah and tells about her son that “My science is Evan, and he&rsquo;s at home. That&rsquo;s my science”, no matter that…</p>

<ul>
<li>the single study combining vaccines to autism has long since been disproven, its author has lost his doctor&rsquo;s license for fraud, and Lancet has finally published a retraction, and that…</li>
<li>based on some evidence, her son&rsquo;s (who supposedly was cured from autism in 2010 through diet and other means) symptoms point to Landau-Kleffner Sydrome or, in layman&rsquo;s terms, delayed development, not autism,…</li>
</ul>


<p>…people still cry and clap their hands. As Hymes writes,</p>

<blockquote><p>“To paraphrase George Lucas: So this is how science dies — to thunderous applause? In the court of public opinion, data, and statements, and science are no match for an emotional parent and her child.”</p></blockquote>

<h4>Story Fallacy</h4>

<p>We want our lives to follow a tight-nit story that is easy to follow. We talk about understanding surprising events, but that&rsquo;s not really true. We simply build the meaning into them afterwards.</p>

<p>Media is a champion at this. Just think about the rampant “Apple is doomed, just like with PCs in the 1980&#8217;s” narrative. No matter what the facts say, the tech journalists who subscribe to the above notion will distort and retrofit them to their preferred narrative. Hollywood is of course another master at it and this obviously gives an edge to McCarthy over her opponents, the science community who try to convince the public with hard data and statistics.</p>

<p>Unfortunately in this case, stories attract us (and you&rsquo;ll soon learn why) while the abstract makes us bored out of our minds. Thus, entertaining but irrelevant issues are often prioritized over relevant facts.</p>

<h4>Confirmation Bias</h4>

<p>Confirmation bias means that we systematically ignore and dismerit facts and opinions that disagree with our own beliefs and worldviews. If we really like BMW&rsquo;s, we very easily just disregard test articles that give them bad grades and eagerly read through every word in pieces that adore them. The more strongly held a belief is, the stronger the bias is as well.</p>

<p>When we combine these four biases, it&rsquo;s not so hard to understand why the science community has a hard time convincing the McCarthys of the world. As a result, there have recently been several outbreaks of measles in the US, something that already was completely eliminated from the country. The cases have almost without exception happened – like <a href="http://www.npr.org/2013/09/01/217746942/texas-megachurch-at-center-of-measles-outbreak">recently in North Texas</a> – in vaccine-skeptical communities.</p>

<p>The anti-vaccination community is an extreme example, of course. I mean, we&rsquo;re mostly talking about religious whackos, right? We, who are pro-science, would never succumb to such fallacies, right? Let me tell you about another cognitive bias.</p>

<h4>The Overconfidence Effect</h4>

<p>As proven over and over again, we systematically overestimate our knowledge, talent and our ability to predict. And not just by a little bit but on a giant scale. The effect doesn&rsquo;t deal with whether we&rsquo;re correct or wrong in single estimates. Rather, it measures the difference between what we know and what we think we know. The most surprising thing about the effect is that experts are no less susceptible to it than normal people – on the contrary. As Dobelli writes:</p>

<blockquote><p>If asked to forecast oil prices in five years time, an economics professor will be as wide of the mark as a zookeeper will. However, the professor will offer his forecast with certitude.</p></blockquote>

<h3>The positive side of stories</h3>

<p>But let&rsquo;s not be negative here. The flipside of all this is that stories are a very powerful way to get your point across and people to remember what you&rsquo;re trying to teach them. Why is this?</p>

<p>Quite simply, our brains are <a href="http://lifehacker.com/5965703/the-science-of-storytelling-why-telling-a-story-is-the-most-powerful-way-to-activate-our-brains">evolutionarily wired to respond strongly to stories</a>. When we listen to a presentation with mostly boring bullet points, it hits the language processing areas of the brain, where we simply decode words into meaning. And then what? Nothing.</p>

<p>On the other hand, when we&rsquo;re told stories, the aforementioned parts are not the only ones that fire. Any other areas in our brain that we&rsquo;d use when experiencing the events of the story are as well. So if we hear a story about a delicious dish, our sensory cortex gets fired up. If the story is about action sports, our motor cortex is activated. Thus, a good story can put our whole brains to work.</p>

<p>Because of this, in a way we&rsquo;re synchronizing our brains with our listeners. As Uri Hasson from Princeton says:</p>

<blockquote><p>&ldquo;When the woman spoke English, the volunteers understood her story, and their brains synchronized. When she had activity in her insula, an emotional brain region, the listeners did too. When her frontal cortex lit up, so did theirs. By simply telling a story, the woman could plant ideas, thoughts and emotions into the listeners&#8217; brains.&rdquo;</p></blockquote>

<p>So what do you need for a good story. Copyblogger <a href="http://www.copyblogger.com/marketing-stories/">lists the following five things</a>.</p>

<ol>
<li>A hero</li>
<li>A goal</li>
<li>An obstacle</li>
<li>A mentor</li>
<li>A moral</li>
</ol>


<p>Granted, telling stories visually is much harder than verbally. It should not be treated as impossible, though. After all, movies and cartoons are to a large degree visual. So while the above five points are mostly meant for verbal storytelling, keeping them in mind even when weaving narrative with visualization can be of huge help.</p>

<p>It is important to build continuum, a narrative to your visualizations. The information presented needs to be integrated, rather than a bunch of unrelated pieces. You also want to create relevant emotions and affect to your presentation, and here it helps to link it to the viewers existing knowledge. However you do it, try to make your message more memorable and thus likely to impact behavior.</p>

<p>And whatever you do, keep in mind both a story and a visualization has to make sense.</p>

<p><a href="http://wtfviz.net/post/61791256708/inverse-relationship"><img src="https://25.media.tumblr.com/38972b1522b97f4aec2de97161eac2d9/tumblr_msz2xaFSmK1sgh0voo1_1280.png" alt="" /></a></p>

<p>The above graph courtesy of <a href="http://wtfviz.net/">WTF Visualizations</a>.</p>

<p>And if you&rsquo;re still convinced you can&rsquo;t tell stories with visualizations, watch <a href="http://www.ted.com/talks/hans_rosling_shows_the_best_stats_you_ve_ever_seen.html">the TED talks by Hans Rosling</a>.</p>

<p>So, how did Saul Hymes solve the problem of fighting a convincing, storytelling opponent? By telling stories himself. So while he still quoted the relevant stats and facts about the risks of taking vs not taking vaccines, he also started telling vibrant, vivid stories of individual kids dying or going deaf in his hands. After all, he didn&rsquo;t have to convince people that taking vaccines is not dangerous. He had to convince them that <em>not</em> taking them <strong>is</strong>. And that is, of course, easy with a meaty story.</p>

<h2>In closing</h2>

<p>I want you to remember two things from this article.</p>

<ol>
<li>Our vision and short-term memory – and thus our attention – capacity are very limited. To present successful visualizations, we have to keep this in mind, plan for it and help the cognition of the viewers with cues.</li>
<li>Storytelling is not important (just) because it is entertaining. It is important because it works – it makes people understand and remember our lessons better. This power might be based on fallacies but it is still very much a real effect that you can and should use to do good.</li>
</ol>


<p>And wait, there&rsquo;s more. I&rsquo;ll just leave this thought here for you to ponder:</p>

<p><strong>If you&rsquo;re into data visualization, you&rsquo;re not in the data business – you&rsquo;re in the human communications business.</strong></p>

<p>Visualization is just a tool to attain goals. Keep that in mind.</p>
<div class="footnotes">
<hr/>
<ol>
<li id="fn:1">
<p>As a Bear Metal cubby.<a href="#fnref:1" rev="footnote">&#8617;</a></p></li>
<li id="fn:2">
<p>Isn&rsquo;t it awesome to describe real, natural things with metaphors from the tech world that no one would have understood just a few decades ago?<a href="#fnref:2" rev="footnote">&#8617;</a></p></li>
<li id="fn:3">
<p>It is fair to ask why they are provided in the first place. They certainly don&rsquo;t make the graphs look any better either, quite the contrary.<a href="#fnref:3" rev="footnote">&#8617;</a></p></li>
</ol>
</div>

]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Are you flying blind – How to Regain Control of Production Systems with the Help of Situation Awareness?]]></title>
    <link href="http://bearmetal.eu//theden/situation-awareness/"/>
    <updated>2013-09-03T16:57:00+03:00</updated>
    <id>http://bearmetal.eu//theden/situation-awareness</id>
    <content type="html"><![CDATA[<figure markdown="1">
  <a href="http://www.flickr.com/photos/robbn1/3391187126/">
    <img src="https://farm4.staticflickr.com/3454/3391187126_4e62f6a374_b.jpg">
  </a>

  <figcaption>
    <p>
      Photo by <a href="http://www.flickr.com/photos/robbn1/3391187126/">Robb North</a>
    </p>
  </figcaption>
</figure>


<p>A few months of work during a sabbatical yielded a product that nailed a problem in the preventative healthcare space. After a freemium window, the product gained good market traction and you spawn a new company with three coworkers. Customers are raving, sales trends are on the up, the engineering team is growing and there are conceptual products in the pipeline.</p>

<p>Three months down the line, there are 4 production applications, hordes of paying customers, a few big contracts with strict SLAs (service-level agreements) and enough resources to spin off a presence in Europe. A new feature that combines these products into a suite is slated for release. Engineering hauled ass for 2 months and sales is super stoked to be able to pitch it to customers.</p>

<h2>Shipping</h2>

<p>A few days before the feature release a set of new servers is provisioned to buffer against the upcoming marketing push. Due diligence on various fronts was completed, mostly through static analysis of the current production stack by various individuals. Saturday morning at 1am PST they deploy during a window with a historically low transaction volume. Representatives of a few departments sign off on the release, although admittedly there are still dark corners and the OK is mostly based off a few QA passes. Champagne pops, drinks are being had and everyone calls it a day. But then&hellip;</p>

<h2>When things go south</h2>

<p>At 9am PST various alerts flood the European operations team - only 25% of the platform&rsquo;s available, support is overwhelmed and stress levels go up across the board. Some public facing pages load intermittently, MySQL read load is sky high and application log streams are blank. This deployment, as with most naive releases, was flying blind. A snapshot of a working system prior to release isn&rsquo;t of much value if it can&rsquo;t be easily reproduced after rollout for comparison.</p>

<p>Based on assumptions about time, space and other variables there was a total lack of <strong>situation awareness</strong> and thus no visibility into expected impact of these changes. Running software that pays the bills is today more important than a flashy new feature. However, one must move forward and there are processes and tools available for mitigating risk.</p>

<h2>What is situation awareness?</h2>

<p>Situation awareness can be defined as an engineering team&rsquo;s knowledge of both the internal and external states of their production systems, as well as the environment in which it is operating. Internal states refer to health checks, statistics and other monitoring info. The external environment refers to things we generally can&rsquo;t directly control: Humans and their reactions; hosting providers and their networks; acts of god and other environmental issues.</p>

<p>It&rsquo;s thus <em>a snapshot in time of system status that provides the primary basis for decision making and operation of complex systems</em>. Experience with a given system gives team members the ability to remain aware of everything that is happening concurrently and to integrate that sense of awareness into what they&rsquo;re doing at any moment.</p>

<h2>How situation awareness could have helped?</h2>

<p>The new feature created a dependency tree between 4 existing applications, a lightweight data synchronization service (Redis) and the new nodes that were spun up. Initial investigation and root cause analysis revealed that the following went wrong:</p>

<ul>
<li>The Redis server was configured for only 1024 connections and it tanked over when backends warmed up as the client connection was lazily initialized.</li>
<li>Initial data synchronization (cache warmup) put excessive load on MySQL and other data stores also used for customer facing reporting.</li>
<li>The data payloads used for synchronization were often very large for outlier customers, effectively blocking the Redis server&rsquo;s event loop, also causing memory pressure.</li>
<li>The new nodes were spun up with a wrong Ruby major version and also missed critical packages required for normal operations.</li>
<li>A new feature that rolls the &ldquo;logger&rdquo; utility into some core init scripts piggybacked on this release. A syntax error fubar&#8217;ed output redirection and thus there weren&rsquo;t any log streams.</li>
</ul>


<p>Without much runtime introspection in place, it was very difficult to predict what the release impact would be. Although not everything could be covered ahead of time for this release, even with basic runtime analysis, monitoring and good logging it would have been possible to spot trends and avoid issues bubbling up systematically many hours later.</p>

<p>Another core issue here is the &ldquo;low traffic&rdquo; release window. It&rsquo;s often considered good practice to release during such times to minimize fallout for the worst case, however it&rsquo;s sort of akin to commercial Boeing pilots only training on Cessnas. Any residual and overlooked issues tend to also only surface hours later when traffic ramps up again. This divide between cause and effect complicates root cause analysis immensely. You&rsquo;d want to be able to infer errors from the system state, worst case QA or an employee and most definitely not customers interacting with your product at 9am.</p>

<p>One also cannot overlook the fact that suddenly each team now had a direct link with at least 3 other applications, new (misconfigured) backends and Redis at this point in time. Each team however only still mostly had a mental model of a single isolated application.</p>

<h2>Why situation awareness is so important?</h2>

<p>We at Bear Metal have been through a few technology stacks in thriving businesses and noticed a recurring theme and problem. Three boxes become fifty, ad-hoc nodes are spun up for testing, special slaves are provisioned for data analysis, applications are careless with resources and a new service quickly becomes a platform-wide single point of failure. Moving parts increase exponentially and so do potential points of failure.</p>

<p>Engineering, operations and support teams often have no clue what runs where, or what the dependencies are between them. This is especially true for fast growing businesses that reach a  critical mass - teams tend to become more specialized, information silos are common and thus total system visibility is also quite narrow. Having good knowledge of your runtime (or even just a perception) is instrumental in making informed decisions for releases, maintenance, capacity planning and discovering potential problems ahead of time. Prediction only makes sense once there&rsquo;s a good perception of &ldquo;current state&rdquo; in place to minimize the rendering of fail whales.</p>

<h2>Web operations and awareness</h2>

<p>Operations isn&rsquo;t about individuals, but teams. The goal is to have information exchange between team members and other teams being as passive as possible. Monitoring, alerting and other push based systems help a lot with passive learning about deployments. It&rsquo;s mostly effortless and easy for individuals to build up knowledge and trends over time.</p>

<p>However, when we actively need to search for information, we can only search for what we already know exists. It&rsquo;s impossible to find anything we&rsquo;re not aware of. Given the primary goal of an operations team is platform stability in the face of changes, time to resolution (TTR) is always critical and actively seeking out information when under pressure is a luxury.</p>

<p>Historically a systemwide view has always been the territory of the CTO, operations team and perhaps a handful of platform or integration engineers. Inline with devops culture, we need to acknowledge this disconnect and explore solutions for raising situation awareness of critical systems for all concerned.</p>

<h2>And now</h2>

<p>Take a minute and ponder the following :</p>

<ul>
<li>How well do you think you know your systems?</li>
<li>Are developers able to infer potential release risks themselves?</li>
<li>When things go south, how well informed is your support team and what information can they give customers?</li>
<li>Are you comfortable releasing at any time?</li>
</ul>


<p>In our next post, we&rsquo;ll explore some common components, variables and events required for being &ldquo;on top&rdquo; of your stack. In the meantime, what causes you the most pain when trying to keep up with your production systems? What would you write a blank cheque for? :-)</p>

<p><a href="https://news.ycombinator.com/item?id=6332734">Discuss on Hacker News</a>.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[It’s not about us, it’s about you (and not really about you, either)]]></title>
    <link href="http://bearmetal.eu//theden/its-not-about-us-its-about-you/"/>
    <updated>2013-08-14T15:51:00+03:00</updated>
    <id>http://bearmetal.eu//theden/its-not-about-us-its-about-you</id>
    <content type="html"><![CDATA[<figure markdown="1">
  <a href="http://www.flickr.com/photos/22711505@N05/8987537991/in/photolist-eGctVz-Rw9Az-RVB46-d1oXub-3od3r2-5maP4G-d1pEjS-2z4E45-7zg9zD-dk9TPx-dk9ViL-62gdUK-cbhisA-dk9VoE-6sjCDX-6sjCQ4-8EcAxc-d1pPAm-aSuUEX-7WR5Va-9SMEDS-mzFjV-9SrEU9-5bvnMg-bbvqpn-6S8uV4-6ScRvQ-8ypVzY-bbvqoz-7YzNHn-eecQ8W-d1pDv9-6QYGRg-62QwZR-fbZJU2-bXZ3iw-hZ9cW-hZ9cX-hZa5p-hZa5q-5iHQyg-4SnuGn-g5GYQ-9dbkFa-b5vRQc-d1pF7C-5oG5br-bcHysn-c3QYZW-c3QYiN-749XSZ/"><img src="https://farm3.staticflickr.com/2859/8987537991_795f7568ca_h.jpg" alt="" /></a>

  <figcaption markdown="1">
    <p>Photo by <a href="http://www.flickr.com/photos/22711505@N05/8987537991/in/photolist-eGctVz-Rw9Az-RVB46-d1oXub-3od3r2-5maP4G-d1pEjS-2z4E45-7zg9zD-dk9TPx-dk9ViL-62gdUK-cbhisA-dk9VoE-6sjCDX-6sjCQ4-8EcAxc-d1pPAm-aSuUEX-7WR5Va-9SMEDS-mzFjV-9SrEU9-5bvnMg-bbvqpn-6S8uV4-6ScRvQ-8ypVzY-bbvqoz-7YzNHn-eecQ8W-d1pDv9-6QYGRg-62QwZR-fbZJU2-bXZ3iw-hZ9cW-hZ9cX-hZa5p-hZa5q-5iHQyg-4SnuGn-g5GYQ-9dbkFa-b5vRQc-d1pF7C-5oG5br-bcHysn-c3QYZW-c3QYiN-749XSZ/">Ron Cogswell</a></p>
  </figcaption>
</figure>


<p>One of the saddest things to happen online was in 2007 when my all-time favorite author and presenter, Kathy Sierra, received death threaths and thus retreated from the public web. It also meant that she stopped writing her <a href="http://headrush.typepad.com">Creating Passionate Users</a> weblog, which had been a great inspiration for me for quite some time. Thank god she didn&rsquo;t <a href="http://ejohn.org/blog/eulogy-to-_why/">pull a _why</a> on it.</p>

<p>While it’s more than six years since Kathy’s last blog post (is it really that long?), there is no reason we shouldn’t apply her lessons even in today&rsquo;s online world.</p>

<p>Maybe the most famous mantra of Sierra was that in order to create passionate users you should make <em>them</em> kick ass. Sure, it’s nice if your UI boasts übercool 3D CSS transformations but if it doesn’t help your users shine, no one (well, except for some web geeks) will give a flying fuck.</p>

<p>She demonstrated this with the fact that very often companies spend a huge amount of effort and money to hone the living daylights off their marketing materials but don&rsquo;t really put that much time into what actually helps their users: tutorials and user manuals. Of course this had helped her immensively by creating a market for the visual <a href="http://www.headfirstlabs.com">Head First</a> book series on O&#8217;Reilly that she curated.</p>

<p>Apple has for a long time been a good example of helping its users kick ass. The user manual of the old Final Cut Pro 7 was also a great introduction to the art of video editing<sup id="fnref:1"><a href="#fn:1" rel="footnote">1</a></sup>. Likewise, most of Apple ads show things you can do and <em>create</em> with their products, not just random people dancing around the pool.</p>

<p>People care about how they can kick ass <em>themselves</em> and they need to be able to learn it to capitalize on it. Nowadays it seems that companies are much more interested in giving people free apps and then using psychological tricks to milk money out of them than helping them shine. Which, coincidentally, brings us back to Kathy Sierra.</p>

<p>To my pleasant surprise, I last week learned that Kathy is back with the pseudonym <a href="https://twitter.com/seriouspony">Serious Pony</a>, and <a href="http://seriouspony.com/blog/">a new blog</a> of the same name. The first article, <a href="http://seriouspony.com/blog/2013/7/24/your-app-makes-me-fat">Your app makes me fat</a>, is of the same awesome quality as her old pieces. In it, she tackles head-on the aforementioned gamification trend and the <a href="http://youarenotsosmart.com/2012/04/17/ego-depletion/">ego depletion</a> tax it puts on us as app users.</p>

<p>To honor Kathy, we wanted to start this blog off by not talking about us ourselves, because <a href="http://youarenotsosmart.com/2012/04/17/ego-depletion/">Bear Metal</a> isn&rsquo;t really about us, but you. And – assuming you are a developer, entrepreneur or content provider – not really about you either. It&rsquo;s about who we (you and us) serve. Because without them there is no market, no audience, no need, no problems to solve, no pains to relieve. Your customers should be the ones that matter to you. And they don&rsquo;t care about you or us. They care about whether your product can make them shine.</p>

<p>Can your product help <em>them kick ass</em>? <em>Does it</em>? Are you <em>communicating that effectively</em> to your current and potential customers? That is all that should matter.</p>
<div class="footnotes">
<hr/>
<ol>
<li id="fn:1">
<p>Unfortunately this can&rsquo;t be said about the manual of the new version, Final Cut Pro X.<a href="#fnref:1" rev="footnote">&#8617;</a></p></li>
</ol>
</div>

]]></content>
  </entry>
  
</feed>