Commits on Dec 27, 2008
  1. Upgrade to interim release of Webrat from github

    This should let us experiment with Webrat::Selenium support.
    emk committed Dec 27, 2008
  2. Add integration test for "reset password"

    Note that we actually extract the activation URL from the e-mail and
    pass it directly to 'visit'.
    emk committed Dec 27, 2008
  3. Write login integration tests using Webrat

    Why do we need integration tests? We've been suffering a lot of
    regressions in the Mephisto UI, because our functional tests don't reach
    high enough up towards the browser, and whole classes of bugs manage to
    slip through.
    What is Webrat? Webrat is a "browser simulator" written in Ruby.  It
    generates a DOM and allows us to fill in forms as though an actual user
    were interacting with the site.
    Why Webrat, and not Selenium, Watir, etc?  Webrat is recommended by the
    Cucumber project as the default way to write user stories; it's very
    fast; and it has a reasonable API.  Plus, Webrat actively maintained,
    and very recent versions of Webrat can be used as a front-end to
    Why Rails integration tests, and not Cucumber stories?  Since the people
    contributing to Mephisto will largely be programmers, I decided to write
    integration tests using a Ruby-based DSL.  Cucumber stories look really
    interesting, but with no actual clients in the loop, the text-based format
    is slightly less useful and has a steeper learning curve for programmers.
    Since we're switching to a new integration testing tool, I moved a bunch of
    code out of test_helper.rb and put it into our only existing integration
    test, caching_test.rb.  I also switched blueprints.rb to set up user
    passwords using 'password' and 'password_confirm' (instead of crypted and
    salted values) to make it easier for tests to override
    emk committed Dec 27, 2008
Commits on Dec 26, 2008
  1. Remove RSpec StoryRunner files

    RSpec's StoryRunner has apparently been deprecated by the RSpec project
    in favor of Cucumber.  And we're not using it anyway.
    emk committed Dec 26, 2008
  2. JavaScript: Fix authenticity_token problems

    One of these should be a GET; the other needed a token.
    emk committed Dec 26, 2008
  3. JavaScript: Fix asset search

    We desperately need some kind of unit testing framework for our
    JavaScript code, preferably with full support for integration testing
    against our application.
    I removed a 'page' parameter here that didn't seem to be doing anything.
    emk committed Dec 26, 2008
Commits on Dec 25, 2008
  1. Begin updating to latest Prototype

    Our version of Prototype is really old, and no longer compatible with
    current versions of RJS.  This breaks quite a bit of our admin interace
    in subtle ways.
    The patch upgrades our bundled version of Prototype to the version
    included with Rails 2.2.2.  We also remove some obsolete bits of
    application.js and make a few minor fixes so that it loads without
    I've only tested the JavaScript support on the articles page and the
    assets page.  Quite a bit of formerly broken stuff now seems to work,
    including updates to the list of attached files and updates to the
    contents of the bucket.  But there's still more to test and fix.
    emk committed Dec 25, 2008
Commits on Dec 24, 2008
  1. Added some notes about fixing JavaScript

    Our copy of Prototype is so old that it won't work correctly with the
    code generated by *.rjs.  But some of the stuff in application.js is
    incompatible with newer versions of Prototype.
    Fixing this is going to require testing all the JavaScript-enabled
    controllers by hand.
    emk committed Dec 24, 2008
  2. Modernize rjs: admin/articles

    I'm going to try to rename all the *.rjs files to *.js.rjs.  This is
    trickier than the other renamings, because our unit test coverage isn't
    perfect, and I'm trying to test everything by hand when possible.  So
    I'm going to do this one directory at a time.
    Other changes:
      - The live_preview and _preview stuff wasn't being used.
      - We didn't have a test case for the 'label' action.
    emk committed Dec 24, 2008
  3. Rename *.rxml files to *.xml.builder

    This also requires adding respond_to blocks to some of our actions.
    emk committed Dec 24, 2008
  4. Rename *.rhtml files to *.html.erb

    Let's go for the new-school approach here.
    emk committed Dec 24, 2008
Commits on Dec 23, 2008
  1. Change default user article filter to Textile

    A poll on #mephisto got several votes for Textile as the default, and no
    votes for Markdown.  Done.
    emk committed Dec 23, 2008
  2. Change default comment filter to Textile

    "Plain HTML" is not a very reasonable comment format for end-users, who
    probably want simple paragraph breaks to do the right thing.  So I'm
    defaulting this to Textile for new sites.
    I also considered defaulting comments to Markdown, but I didn't like the
    way that Markdown handled underscores_in_identifiers or * versus _.  For
    very simple stuff, Textile seems very slightly more intuitive than
    Markdown, though neither is perfect.
    emk committed Dec 23, 2008
Commits on Dec 22, 2008
  1. Fix theme controller bugs

    J.C. Zhu reported an InvalidAuthenticityToken error when trying to
    change themes.  This was one of several bugs in the theme switching
    code left over from the Rails 2.2 porting process and the security
    We needed to convince Ajax.Request to use a GET request when displaying
    the theme tools, and we needed to properly escape some text in the
    _tools template.
    I've also added a test case to make sure we actually render the _tools
    view successfully.
    emk committed Dec 22, 2008
  2. Unbundle tzinfo gem

    Sadly, unpacking tzinfo into vendor gems breaks Rails' time zone
    support and gives the following error when trying to access
      uninitialized constant TZInfo::Timezone::TimezoneProxy
    So we're going to instruct the user to install this gem using 'sudo
    rake gems:install' for now.
    emk committed Dec 22, 2008
Commits on Dec 20, 2008
  1. Update to emk-safe_erb 0.1.2

    This fixes script/generate and the standard Rails error page.
    emk committed Dec 20, 2008
  2. Test for safe_erb in ActionView::Template, not in ERB

    We've been installing safe_erb in all ERB templates, which breaks
    script/generate and lots of other important stuff.  But before we can
    fix this bug in our custom-hacked safe_erb, we need to narrow our
    Mephisto unit tests down so that they only test ActionView::Template.
    A safe_erb update will be along shortly.
    emk committed Dec 20, 2008
  3. add machinist to the tests

    technoweenie committed Dec 20, 2008
  4. add Event blueprint

    technoweenie committed Dec 20, 2008
  5. Security: Finish first stage of audit

    What we've done: We've tried to protect against attacks by the "public".
    Most of our attention has been directed towards XSS, CSRF and other
    attacks by users who aren't logged in.
    Our security audit was based on the following principles:
     1) Users with access to /admin are (unfortunately) fully trusted.
        There are simply too many ways for them to escalate their privileges
        right now, if they're willing to use XSS and other attacks.
     2) Things which look "suspicious" were simply fixed, without any
        attempt to determine whether they could be exploited in the wild.
     3) Whenever possible, we instituted broad, automatic protections
        against entire classes of attacks.  These include SafeERB and
        read-only GET requests.  This means that we don't need to audit
        every single view, controller and plugin for subtle errors.
    What still needs work: My hacked version of SafeERB is currently
    breaking script/generate.
    emk committed Dec 20, 2008
  6. Don't log "remember me" token

    This token can be trivially recovered from the database, so excluding it
    from the logs doesn't actually accomplish anything.  But there's no
    reason to include it in the logs, either.
    emk committed Dec 20, 2008
  7. Security: Attempt to block auth of nil tokens, etc.

    Some Rails authentication systems have suffered from a vulnerability
    involving nil or blank login tokens:
    This patch includes a bunch of test cases testing for possible attacks
    along these lines, and some sanity-checking code in our authentication
    Note that the tests and the code don't really "line up" here--most of
    the test methods passed already, and most of the sanity-checking code
    is probably unnecessary.  But again, better safe than sorry.
    emk committed Dec 20, 2008
  8. Security: Force all GET requests to be read-only

    The W3C makes a clear distinction between GET and POST requests.  GET
    requests should only cause "safe" actions, and the user should never be
    held accountable for making GET requests.  See the following for an
    The Rails 'protect_against_forgery' function (and possibly some web
    browsers) rely on the distinction between GET and POST to provide
    protection against CSRF attacks.  See:
    Unfortunately, enforcing these rules in rather difficult, especially in
    a large application with lots of controllers and plugins.  So this patch
    applies a rather heavy-handed fix: We globally block database writes
    during GET requests, and specifically override that policy in one or two
    All of our current overrides invoke User#reset_token!.  I haven't
    performed a full security analysis of allowing User#reset_token! (or
    updates to session[:user] based on our "remember me" token) in a GET
    request.  For now, I'm going to go ahead and allow this activity--if we
    actually have some sort of vulnerability here, it affects a wide range
    of web applications.
    Note that this patch may break some part of the /admin interface.  I've
    tried posting articles and other basic stuff, but I haven't used the
    lesser-known corners of /admin since making these changes.  Please
    report any problems.
    emk committed Dec 20, 2008
  9. Add unit test for img filtering

    Test case for a644733.
    emk committed Dec 20, 2008
  10. Merge branch 'experimental'

    emk committed Dec 20, 2008
  11. Security: Block <img ... /> tags when sanitizing

    A whole class of CSRF attacks uses the img tag:
      <img src="/admin/account/action_that_allows_get" />
    This will invoke action_that_allows_get using a GET request and first-
    party cookies.  There are some examples on Wikipedia:
    Note that really solid enforcement of the "use GET only for queries"
    rule will also prevent this kind of attack.  Also note that if you
    allow third-party cookies, this patch doesn't help you at all--any
    other site on the Internet could trigger this attack.
    emk committed Dec 20, 2008
  12. Security: Fix many broken filter regexps

    In Ruby, "foo\nbar" =~ /^bar/ will result in a match, because ^ matches
    at the start of any line, not at the start of the string.  In general,
    we want to use \A and \z in place of ^ and $, respectively.
    We rely heavily on regular expressions to filter untrusted data.  And
    many of these regular expressions can be fooled easily because they rely
    on ^ and $ when they shouldn't.  See comment_drop_test for a
    user-exploitable example.
    This patch does a bulk search-and-replace of the offending patterns.  It
    may easily have missed something somewhere, but it's a good start.
    emk committed Dec 20, 2008
Commits on Dec 19, 2008
  1. Attempt to escape strings in FlickrMacro

    This plugin had absolutely no HTML escaping.  I've attempted to fix
    this, but since the API key has expired, this code has not been tested.
    emk committed Dec 19, 2008
  2. Only accept known comment fields

    The comment form is most-exposed attack point in all of Mephisto,
    because it doesn't require an authenticity_token and can be used by
    anybody on the internet.
    In the interests of paranoia, this patch removes bulk assignment from
    the comment-posting code.  I don't see any way to exploit the previous
    code (several attr_accessible fields looked vulnerable, but don't
    actually exist any more).  But better safe than sorry.
    emk committed Dec 19, 2008
  3. Security: Fix XSS attack against new comment form

    WARNING: If you're one of the first people testing this commit, please
    use a backup database.
    How to reproduce: Create a new comment, and set all fields to
    <script>alert("Pwned")</script>.  Submit it.  You will see a JavaScript
    alert dialog, which is bad.
    What's happening: Untrusted fields in Comment objects are sanitized
    immediately before they're written to the database for the first time.
    But if validation fails, it leaves the application with an unsanitized
    comment object.  When the "can't submit comment" error is displayed,
    this unsanitized comment object can be passed straight throught to
    Liquid, which assumes that all HTML tags have been escaped.
    (This may look like "self XSS" attack only, but hostile pages can
    trigger it by tricking you into submitting a comment form back to your
    own site, preloaded with malicious data.)
    How we fix it: We make HTML escaping the responsibility of CommentDrop,
    not the Comment model.  This means that we need to unescape several
    existing fields in the database.
    Possible issues: This means that we're storing dangerous, untrusted data
    in our database, and that we need to rely on the proper use of 'h' and
    'CGI.escapeHTML'.  In the case of 'h', we're already using SafeERB, so
    insecure admin templates will be caught automatically, and dangerous
    data should never be sent to the user.  In the case of Liquid, we need
    to carefully examine our CommentDrop class to make sure that we're not
    passing any unescaped data through to the Liquid templates.  But this is
    a pretty manageable "proof obligation"--and remember that the old
    "sanitize on create" code actually suffered from XSS attacks, because it
    was too easy to do the sanitization in the wrong place.
    emk committed Dec 19, 2008
Commits on Dec 18, 2008
  1. Ignore .DS_Store files

    Everybody keeps adding this ignore rule to their github branches, so
    let's just go ahead and add it to master .gitignore file.  The .DS_Store
    file is created by MacOS X.
    emk committed Dec 18, 2008