Permalink
Switch branches/tags
Nothing to show
Commits on Nov 14, 2016
  1. Remove live config "precomputed_comment_sort_read_chance"

    This was used to gradually ramp up reads of the precomputed comment
    orders. We've been running for a while with this set to always read,
    so the setting can be removed.
    bsimpson63 committed Aug 8, 2016
  2. _get_qa_comment_scores: Take cid_tree as input

    Previously it wasn't clear that the CommentTree would be looked up and
    that there might be some consistency concerns.
    bsimpson63 committed Jun 28, 2016
  3. write_comment_orders: Add counters to track writes vs. noops

    This will let us get a sense of how much work is actually done. I'm looking
    at splitting the CommentOrderer update out into a separate queue and need
    to understand how many writes actually happen.
    bsimpson63 committed Nov 9, 2016
  4. add_comments: Remove warning about writing scores first

    This warning was no longer true--any missing scores are automatically
    calculated and updated.
    
    We actually have the opposite issue--the CommentTree must be updated
    before writing scores because the QA score reads it.
    bsimpson63 committed Jun 28, 2016
  5. voting: Don't update comment scores on automatic initial vote

    Score updates are processed through commentstree_q. When a new comment
    is created an automatic initial vote (by the comment's author) is created.
    This results in two messages in commentstree_q: one from the vote and one
    from queries.new_comments. Don't create the message from the vote because it
    is redundant. This will let us reduce the volume of messages in commentstree_q
    which is currently very high.
    bsimpson63 committed Nov 10, 2016
  6. feature: Fix race condition when live_config is updated

    Instead of checking _featurestate_cache for a key's existence and then
    retrieving it, just get it and then check for a miss. Doing the two step
    process can result in a KeyError if _featurestate_cache is cleared between
    the existence check and the retrieval.
    bsimpson63 committed Nov 8, 2016
  7. db_manager: Don't setup a connection for some dbs on startup

    "email", "authorize", "hc", and "traffic" databases aren't used that
    often and maybe we can reduce the number of connections to pg-05 by
    waiting to establish a connection until it's actually needed.
    bsimpson63 committed Nov 8, 2016
Commits on Nov 3, 2016
  1. voting: Don't use fastlane vote processor

    The fastlane processor was meant to handle votes on both Links and Comments,
    but it can't do that easily anymore now that the vote processing has been
    split. It's not a big deal now because Link vote processing is much faster
    now that the query updating has been separated and sharded. The Comment vote
    consumer/queue was getting some benefit from the fastlane and it can be
    resurrected if we run into problems.
    bsimpson63 committed Nov 3, 2016
  2. Don't error on unicode in the votestream.

    Change the error from a 500 to a line in the error log with more info about what/why it failed.
    This seems to be the result of someone's crappy votebot.
    KeyserSosa committed with bsimpson63 May 25, 2016
Commits on Nov 1, 2016
  1. Add ability for all oauth tokens to be invalidated on password change

    Add 'revoke_sessions' method to 'POST_update_password'
    KTAtkinson committed Nov 1, 2016
Commits on Oct 26, 2016
  1. link votes: Use a queue to update links by domain queries

    The queue can be sharded by domain to minimize lock contention
    and the consumer will batch updates to the same links (e.g. several
    votes for the same link) and to the same domain (e.g. votes for different
    links to a single domain).
    bsimpson63 committed Oct 17, 2016
  2. link votes: Use a queue to update links by subreddit queries

    The queue can be sharded by subreddit id to minimize lock contention
    and the consumer will batch updates to the same links (e.g. several
    votes for the same link) and to the same subreddit (e.g. votes for different
    links submitted to a single subreddit).
    bsimpson63 committed Oct 17, 2016
  3. link votes: Use a queue to update links by author queries

    The queue can be sharded by author id to minimize lock contention
    and the consumer will batch updates to the same links (e.g. several
    votes for the same link) and to the same author (e.g. votes for different
    links submitted by a single author).
    bsimpson63 committed Oct 17, 2016
  4. Unshard vote_link_q

    bsimpson63 committed Oct 17, 2016
  5. voting: Use separate methods for updating cached queries

    This makes the individual tasks of the consumer more readable.
    bsimpson63 committed Oct 17, 2016
  6. Inline vote processor work

    bsimpson63 committed Oct 12, 2016
  7. Add timings in Link vote processor

    This will help us understand which parts are most time consuming
    and whether we can benefit by breaking up the updates into multiple
    queues.
    bsimpson63 committed Oct 11, 2016
  8. link.py: Move cast_vote import to within method

    This gets around some circular import issues and is simpler than changing
    the imports in voting.py.
    bsimpson63 committed Oct 18, 2016
Commits on Oct 25, 2016
  1. LocalCache.simple_get_multi: Use exception handling to get keys

    Doing the 2 step process of:
    1. check for existence of key
    2. get key
    
    introduces a race condition if the key is deleted between the two steps.
    bsimpson63 committed Oct 25, 2016
Commits on Oct 19, 2016
  1. raven: Specify all application code path prefixes

    The raven client inspects the traceback and attempts to figure out which
    parts belong to the app and which belong to external libraries. It uses
    a whitelist of paths to identify application code. Previously we had been
    using the list of repository names, but that was incorrect because the
    plugin "liveupdate" is actually called "reddit_liveupdate" in the traceback.
    The whitelist was also incomplete because it didn't account for scripts run with
    paster run which can have a path like /opt/something/script.py.
    bsimpson63 committed Oct 19, 2016
  2. RavenErrorReporter: Don't install_sys_hook

    This creates a sys.excepthook handler that reports any exceptions
    to Sentry. This results in double reporting errors when in script
    mode because the exception is re-raised and caught by that handler.
    bsimpson63 committed Oct 6, 2016
  3. Add page-based experiment support

    We can conduct experiments that impact how pages are rendered across
    users, bucketing pages according to the fullname, so that search engines
    will crawl and index the same experimental content that users see. We
    support subreddit listings pages, comments pages, and comment permalink
    pages. We use the link fullname for both comments pages and comment
    permalink pages, so that they are bucketed together.
    prashtx committed Oct 3, 2016
  4. Tell crawlers not to index comment permalink pages

    We don't want to spend crawl budget or rank on what are essentially
    duplicate pages. In case we have inbound links to these pages, we don't
    want the robots.txt to prevent crawlers from accessing them.
    prashtx committed Jul 15, 2016
Commits on Oct 18, 2016
  1. Add automatic search engine integration.

    This adds the required metadata and files to support automatic search engine
    additions to Chrome / Chromium.
    wting committed Oct 18, 2016
Commits on Oct 12, 2016