Skip to content


Subversion checkout URL

You can clone with
Download ZIP
Branch: master
Commits on Feb 23, 2011
  1. @ketralnis

    February 2011 Merge

    ketralnis authored
    * Upgrade discount to 1.6.8
    * Start reading Votes, Saves, and Hides from Cassandra (while still dual-writing all three)
    * Some XSS fixes
    * Significant Gold buying and gifting improvements
      - Move /api/ipn to /ipn
    * Allow non-US countries to buy sponsored links
    * Increase scope
    * support
    * Allow postgres port number to be specified in ini file (this changes the format of the .ini file)
    * Upgrade Cassandra to 0.7
      - Change g.urlcache to LinksByURL
      - Translate storage-conf.xml to cassandra.yaml
      - TTL support (and enable on Hides)
      - Move permacache keyspace to inside reddit keyspace
    * The stalecache: a local memcached that contains slightly old information to speed up some lookups
    * Switch to patched Paste that is hopefully download.gz-proof
    * Don't store votes on things > 30 days old
    * Many many bugfixes/small features
Commits on Oct 20, 2010
  1. @Gelob @ketralnis

    Add gold to the footer (thanks gelob)

    Gelob authored ketralnis committed
Commits on Oct 19, 2010
  1. @quijote @ketralnis

    Generate comments trees in populate_db

    quijote authored ketralnis committed
Commits on Oct 18, 2010
  1. @ketralnis
  2. @ketralnis
  3. @ketralnis
  4. @ketralnis
  5. @ketralnis

    * Combine cassandra clusters into a single one

    ketralnis authored
    * Add a new Cassandra CF for comment-sort caching
    * Add JSONP support
    * Support non-auto-renewing PayPal IPNs
    * Gold accounting
    * Gold features: comments tracking, larger comment limit
    * Autorenewing gold
    * Google checkout support
    * Profile-page sorting for all
    * Title-text on the logo
    * Hardcache sharding and profiling
    * Self serve enhancements
    * Add /r/foo/faq
    * Make self-centred reddits allow for more verbose selftext
    * Much better down-handling of databases
    * Add the ability to take a thread-dump from a running process
    * Remove the comscore tracker
    * Add comments to modqueue (does not back-populate)
Commits on Jul 22, 2010
  1. @ketralnis

    21 Jul 2010 merge

    ketralnis authored
    * Use Flaptor's Indextank product for search, including santip's patch
    * for their new API
    * Add Cassandra ConsistencyLevels to the ini file, and storage-conf.xml
    * to the public repo
    * Patch contributed by umbrae in ticket #929: Add jumpToContent support
    * for Keyboard Accessibility
    * reddit gold
        - paypal/postcard support
        - friends with benefits
        - profile-page sorting for gold members
        - move domain listings into the permacache
Commits on Jun 26, 2010
  1. @KeyserSosa


    KeyserSosa authored
         * no-repeat on some background images in the sprite (#797)
         * fix deleted comments on the mobile site -- preserve author anonymity (#624
         * faulty permalinks on pages with all unicode title (#776)
         * no more spreadshirt
         * reorganize comment_tree and _builder.pyx to clean up cache handling
         * fix styling for Bug #78
         * Improve handling of /r/all+all (bug #699), etc (not always a 400 now)
         * Fix the time listings: sometimes we get passed unicode for self.time
         * Don't allow private reddits to show in /r/random
         * Try to allow remote sites to show their own favicons in the reddit
      Speed improvements:
      * CommentBuilder refactor rount 1:
         * create new sort and parent permacache entries so that we don't need to loa
         * update sorts and parents when a new comment comes in
         * update non-date sorts when a new comment vote comes in
         * add more trace info to timeouts in CommentBuilder
      * Some misc. performance hacks (incl. adding _utils.pyx)
      * Increase SR description box limit from 1k to 5k
      * Fix a bug where we weren't properly allowing hidden items to be hidden
        on time-filtered listings
      * Make Subreddit._by_name take a list of names like byID
      * Upload thumbs to S3 with reduced_redundancy==True
      * make rss feeds without a 'feed' get parameter act as if the user is not logged in
      * Require a boto that knows about reduced_redundancy
      * remove fast_queries from Message.add_props and put the onus on fetching the unread messages from the permacache
      * Store the list of popular subreddits in the permacache
        * make SubredditTopBar cacheable per-user.
      * add (as safe as we can make it) annotation for sql selects to track down requests hitting the db when they shouldn't be.
     User submitted features:
      * Merge and clean up Phire's selfs-only/no-selfs patch.
      * Don't show expired items on time-filtered controversy listings
      * Also add the 'hide' button to the toolbar just for breakneckridge.
        Next time someone calls me prickly, breakneckridge had better step
        up to the plate for me or I'm rolling it back.
Commits on Jun 17, 2010
  1. @KeyserSosa

    Upgrade Instructions

    KeyserSosa authored
       * Uninstall the python Cassandra package that we previously depended on (it has a namespace conflict with the new package we depend on).  To find it:
         $ python -c "import cassandra; print cassandra.__file__"
        and rm -r the .egg directory after "site-packages/"
       * This version relies on cython, so if "make" fails, you may have to install cython via your distro's package manager.
         $ cd reddit/r2
         $ python develop # possibly with "sudo" depending on your install
         $ make
       * Cassandra is now required for the caching layer.  An example storage-conf.xml can be found in reddit/srv/cassandra.  Make sure that the additional <Keyspace> items are included in your conf file.
       * remove the query_queue_reader services if they are running.  add new instead.  Suggested cron:
        0    */2 *   *   *   $SCRIPTS/ year '("month","year")'
        */5  *   *   *   *   $SCRIPTS/ week '("day","week")'
        *    *   *   *   *   $SCRIPTS/ hour '("hour",)'
       where $SCRIPTS is the location of this script directory
    Features and Bugfixes
     * Mobile reddit:
       * templates are in r2/templates of the form *.compact
       * css is r2/public/static/css/compact.css
         * beginning of a sass-based ( compact.scss
       * reachable via .compact extension or from the "i" subdomain.
     * Cassandra is now *required*, and votes are currently written out to both cassandra and postgres (part of an eventual migration).
     * attempt to make the db connection code a little smarter.
       * A dropped DB connection will mark the connection dead and randomly attempt to reconnect.
       * A dropped db connection on start will permanently mark the connection as dead.
     * Calculate the time-filtered top/controversy listings using mapreduce instead of prec_links (new cron job in reddit/scripts)
     * allow default user/pass for database to be specified with '*' to fallback on db_user and db_pass in the INI file
     * Search feedback buttons
     * make deleted comments not show up in your inbox.
     * move last_visited into cassandra
     * Swallow rare, race-conditiony POST_save/hide/subscribe problems
     * Apparently we haven't been breaking properly for the past few weeks.
Commits on May 28, 2010
  1. @KeyserSosa

    * Comply with the spec on 304 errors so Chrome won't barf download.g…

    KeyserSosa authored
    …z all over the place
     * The error pages have been broken for the past week.
     * Fix /r/all+all
     * Provides about.json to the api for subreddits (it already works for users)
     * Make sure Accounts have data before checking their name and msgtime
     * new bigger arrows for mobile site and a the alien is fingerless now
     * Try to update the _fast_query_timestamps less often to alleviate some of the load it causes on the Accounts DB
     * Swallow rare, race-conditiony POST_save/hide/subscribe problems
Commits on May 21, 2010
  1. @KeyserSosa

    * transparency updates to some of the pngs thanks to ytknows

    KeyserSosa authored
     * bugfixes in unicode handling
     * Nag nonparticpating jurors 20% of the time
     * fix thumbnails for sponsored links
       * non admins can't change the thumb of a live link
       * the uploader updates the status properly to 'saved'
Commits on May 20, 2010
  1. @KeyserSosa

    * Improvements to the email verification system

    KeyserSosa authored
    * addition of the 'powered by reddit' footer
Commits on May 18, 2010
  1. @jedberg @KeyserSosa

    * Add services and scripts to public repos for ease of reddit instal…

    jedberg authored KeyserSosa committed
     * bring discount up to date
     * fix consistency errors in the self-serve traffic page
Commits on May 17, 2010
  1. @jedberg @KeyserSosa

    New Features

    jedberg authored KeyserSosa committed
        * Two new 404 aliens
        * enable a comment page cache if the user isn't author of any of the comments and the page is a non-permalink.  votes and friends are updated by javascript
        * descriptive example.ini
        * create "trusted_sponsor" who get auto-accepted
        * split off url cache from the permacache, with major clean-up to app_globals
        * Move is_banned_domain() call into POST_submit() so it can take the sanitized url
        * Redirect /r/asdfasdfasdf to the search page, not the create page.
        * updates to
        * orangered envelope goes to unread, not all.
        * Remove "enable whitelisting" checkbox, and switch to new failiens
        * Put verification link on email prefs page
        * split off a minimal api controller for onload calls (which don't care about or c.user, making them cheaper)
        * Assume admins want to keep their cookies forever just like everyone else
        * /randomrising bugfix
        * JSON error page was rendering in html.  it now returns "{ error: ${code} }"
  2. @KeyserSosa

    New Features:

    KeyserSosa authored
        * Make the new targeted self-serve promotion the default
          * add TheOatmeal's comic to our source.  (
          * add targetting
          * add rerunning or suplementing existing campaigns
          * move all sponsored link listings to be precomputed
        * deputy moderation
          * /about/trials and /about/modqueue
          * Have trials train the spam filter
          * Allow admins to see all trials, site-wide, at /r/all/about/trials
          * Allow trials to be ended while votes are still coming in when the jury opinion is near-unanimous
        * offload scraping to
        * Use boto for s3cp instead of forking curl
        * migrated all graphs to use Flot
        * Implement per-user suspiciousness quota and print over-quota trace
          * Keep track of how many questionable links a user has outstanding at any given time
        * add links to the FAQ and walkthru
        * add a differnt thumbnail for self posts (thanks to licensplate)
        * Add some visual tracking for comments trees
        * Relabel:  ban / unban => remove / approve
        * Use big remove / approve buttons when the listing relates to moderation,
          or when the item has reports or is on trial
        * Merge unban, ignore, and a new approve this non-banned link into one
        * Be a little smarter about the way we allow caching of subreddit stylesheets
        * make cassandra's cache chain cache negative results (and yes that commit message is bigger than the patch)
        * add All, friends, random, and mod to topbar
        * add a local render precache (optional)
        * better noimage.png from paradox460
        * Add a 'legacy' mode to CMemcache that will allow us to use it as a drop-in replacement for testing purposes. Modifies python-memcached to use the same 'crc' hashing algorithm that pylibmc does, and removes a feature in python-memcached that we aren't using. Contributed by Joe Terranova <>
        * Everyone sees cake
        * Added g.system_user
        * improve validation of media embed, and remove a todo from by putting all of the child div stuff in pages
        * Feed/JSON updates
          * Add selftext to RSS feeds
          * Add subreddit names to Comments in JSON
          * add is_self param to indicate self posts
        * Don't shade the background of child comments of spam
        * shorted the cache lifetime for AllSR to 1 min
        * Tell everyone that we actually own the trademarks we claim to own (footer update)
        * You no longer get an orange alien when one of your reddits sends an automated message to a user
  3. @ketralnis @KeyserSosa


    ketralnis authored KeyserSosa committed
        * Cassandra
          * Add new cassandra libraries that we'll need to
          * Select Cassandra seeds at random rather than in order
          * Bugfix in CassandraCache.delete and a faster permacache migration function
          * Like other caches, CassandraCaches need to be able to take (and ignore) a 'time' parameter
          * add Cassandra to the permacache chain
        * beginning of jury duty (later called deputy moderation)
        * Make /r/friends much cheaper at the expense of sorting
        * Add Jury.delete_old(), which removes Account-Trial relations > 3 days old
        * Make the pretty_button() template function's callback optional, so that
          actionless pretty-buttons can be used on the admin details page
        * make .embed listings work for permalink pages (think of this as a first pass to getting blog comments working).  Adds 'limit' and 'depth' parameter to permalink pages
        * Added final redditheader.html pretty-button class
        * new iframe ads; also make button.js static
        * Usage sampling
        * Stop adding batched time query recalculations to the queue at all except through the catch_up_batch_queries function
        * Superflous comma might be causing IE7 to barf
        * Change the byurl keys again, to fit in memcaches 251 character limit
        * Indentation error causing non-sponsors to be able to get to the advert listing
        * Move to a custom build of pylibmc that doesn't hold the GIL during blocking operations
        * Convert some cache.gets to cache.get_multis, and implement our own thread-safety around pylibmc's client
        * Make search caching a little smarter for time searches
        * Make the ads not be cached for 30 seconds each, ie. more random
        * fix deleted things on profile pages
  4. @KeyserSosa

    New features:

    Mike authored KeyserSosa committed
        * Discount 1.6.1
          * Lines beginning with spaces are considered code. I don't know why didn't trigger this.
          * tables in mark down: why not?
          * validation of resulting HTML vial libxml to prevent hax.
        * private RSS and JSON feeds
        * optional whitelists for subreddits
        * Moderator messaging
        * destination sanitization to cut down on XSRF
        * cosmetic fix to spam and reported listing
        * make the rss feeds on messages useful
        * /admin/errors
        * Of the types of listings hitting the precomputers (top/controversy by hour/day/week/month/year), the ones over long periods of time don't change often. So we can try to run them at most once per day, and then merge in the day's listings.
        * google analytics
        * logging queue
        * Created empty commentspanel.xml, errorpage.xml, login.xml
        * add subreddit rules/info box to submit page
        * add 'via' link on messages in moderator inbox
        * add a show=all get parameter to link listings to optionally ignore hiding preferences.
        * Raise edited timelimit to three mins
        * Remove UI that makes it look like you can edit deleted selftexts
        * Make it clearer to admins when a link is deleted
        * Fix [S] leak on deleted comments
        * Fix /user/[deleted] misrendering
        * New house ads system
        * updated so that minimalcontrollers actually can use the page cache.
        * Added /admin/usage
        * Reduce the number of results that we request from Solr and simplify that caching a bit
        * Require a secret key to shut down app-servers
        * Make get_title a little more resilient to malformed documents and slow remote servers
        * Cause the SearchBuilder to only byID the results that it's going to render instead of all 1000
        * Remove ability for an author to XSS himself
        * fix spam listings and an xsrf
        * More verbose VDestination
        * Fixing the famous ?limit=0.1 error, and one last password-validation one
        * distinguish deleted comments' and deleted links' error messages
        * Don't allow ridiculously long log lines to widen the page
        * Bug with HardCache.add() when existing key is expired
        * Add adminbox next to domain
  5. @ketralnis

    New features:

    ketralnis authored
        * Activate negative-result caching for HardCache chain
        * begin migration to pylibmc:
           * Add pylibmc to the list of required packages in preparation for the replacement of the memcached library
           * Start using pylibmc for the rendercaches
        * Tweak the computation of the normalized hot page to be a bit faster when the precomputer is available, by relying on the precomputer's internal permacached structure.
        * Default to a SelfEmptyingCache for scripts run from `paster run'.  Note that processes that run forever are still responsible for resetting their local-caches, but this can now be done with g.reset_caches()
        * threaded messaging patch part 1: backend changes only.  This will allow to be run to move new onto inbox and will start tracking message trees for users.
        * Specify some queries to run at most once per day
        * Refactored safemarkdown() and added soup testing
        * Added _byID_rel()
        * Made error messages more verbose for:
           1. byID lookups of too-big thing_ids
           2. memcache failures
           3. Solr Nones
        * Award._all_awards() now sorts by date
        * Trophy.by_{account,award}() now cache properly
        * new feedback page with helpful links
        * Try to reduce the length of the query-queue by not adding known-long queries at all, rather than adding them and skipping them
        * whitespace clean up
        * simplify the 'why did my CC get denied' email checking.
        * added missing translation strings and users now get PMs when they are added as translators
        * Fix a bug in unsaving
        * BeautifulSoup stopped hosting 3.0.7a, but 3.0.8 still uses the good parser
        * Better search error handling
        * Properly reset the cache-chains (incl. the hardcache; d'oh!) per-request
        * Fix an attribute error on listings where some items have author_ids and some don't
        * Bug when forcing recalculation of memoized functions
        * the subreddit creation and edit form aren't dealing with errors properly
        * buttons fix
  6. @KeyserSosa

    New features:

    KeyserSosa authored
        * self service sponsored links (initial version with no targeting)
           * credit card processing with (and corresponding interaction code)
        * hardcache -- for persistent cache keys with expiration
        * Awards code
        * messaging overhaul round 1.  Added mark as unread as well as message sorting
        * add TedScraper and improve youtube scraper (ala tritelife); move traffic link on sponsored links and make them still visible when the promotion is over
        * compressify all of our pngs, and set it up so thumbs, sr images, and the sprite get compressed every time they are generated
        * rate limit logins (naive way)
        * add a limit to the length of the moderator sidebox.
        * add over18 thumbnail handling and NSFW label on nsfw content
        * make NSFW label optional, and make the operation of the pref checkboxes sensible
        * add the option (for moderators) to remove a subreddit from the default set
        * Combine the queue handling code to be easily plumbed, and use it to precompute /comments
        * trap duplicate vote error
        * Fixed new mail mouseover on toolbar
        * spammers can't send empty bodied emails any more (they will get rejected as they should).
Commits on Dec 1, 2009
  1. @KeyserSosa


    KeyserSosa authored
     - add confidence sorting to comments
       * common values are precomputed for speedier response
       * best is made the default sort on comment pages
     - messages will now be delivered once one is moderator/contributor/banned
     - UI updates to messaging page, including added show parent functionality to messages
     - Remove the rate-limit on comments on your own self-posts
     - Give users some leeway in editing their comments: don't show an edit star if the edit is within the first few minutes of a comment's lifetime
     - Office Assistant will help users when they write to admins
     - Replace the postgres-based query_queue with an AMQP based one
       * Set up amqp queues for async tasks such as search updates and the scrapers
       * service monitor updates, adding queue-tracking support
     - Allow find_recent_broken_things to specify both from_time and to_time
     - add a ini file parameter to disallow db writes (to create read-only reddit instances for crawlers)
    New features
     - self-serve advertisement:
       * complete overhaul of sponsored link code
       * functions for talking with
       * added pay domain and https support
       * added ability to share traffic from sponsored links
       * auto-reject promotions that are too old and unpaid for
     - awards
     - allow widget to have its links to have a target (in case it is iframed)
     - automatic_reddits:
       * Don't show automatic_reddits in the horizontal topbar
     - Listing numbers are always in order with no gaps
     - add support for sprites for common (r2.lib.contrib.nymph)
     - added a takedown page for dealing with DMCA requests properly
       * status code 404 on takedown pages
       * JSON returns same string as in the explanation text
       * nofollow on markdown in explanation
       * title and image optional
     - Added /c/(comment_id) for admins
     - updates to JS to rate-limit voting, commenting, and anything else that could be just as easily done by a script-kiddie to cheat.
     - make ad frame dynamic and add tracking pixel
     - add the ability to add a sponsored banner to the rightbox of a reddit
     - add the ability to show custom css on cnamed and/or non-cnamed versions of a reddit
     - allow us to ignore reports from report-spammers.
     - Fix sorting of duplicate links (patch by Chromakode)
     - fix traffic bug on main traffic page when it is the first of the month.
     - toolbar redirects to comments page on self posts rather than generating the frame
     - half-assed unicode handling in menus giving us bugs again.  Switched to the whole-ass approach
     - added Thing._byID36
     - Support /help/foo/bar
Commits on Sep 28, 2009
  1. remove markdown onmouseover exploit

    Christopher Slowe authored
Commits on Aug 12, 2009
  1. @KeyserSosa

    API updates:

    KeyserSosa authored
     * make created_utc reference GMT
     * add subreddit_id to comments
     * generate useful media embed info
     * fix to url validator
  2. @KeyserSosa
Commits on Aug 11, 2009
  1. @ketralnis
  2. @ketralnis

    Significantly simplify spam and reporting, and move all of that code …

    ketralnis authored
    …into the public repository
Commits on Aug 10, 2009
  1. @ketralnis
Commits on Aug 6, 2009
  1. @jedberg
Commits on Jul 31, 2009
  1. @ketralnis
  2. @spez

    fix a bug where adding a comment to a comment tree that isn't cached …

    spez authored
    …creates a
    broken comment page.
  3. @spez

    removed some extraneous menus. fixed the 'Send message' link on the m…

    spez authored
    …oderators page. removed the fancy styling in the subreddit info boxes.
  4. @spez
Commits on Jul 30, 2009
  1. @spez

    lock around thing._incr(). remove a lot of cruft that was required wh…

    spez authored
    …en _incr wasn't locked.
  2. @spez
Something went wrong with that request. Please try again.