Permalink
Commits on Jan 26, 2012
Commits on Jan 25, 2012
  1. [bug 721184, 716999] Fixes wiki indexing

    Actively remove from the index and prevent from being indexed wiki documents
    that are redirects or have no revisions.
    willkg committed Jan 25, 2012
  2. Fix ES indexing timeout

    I missed a spot earlier when I changed indexing to use an ES with
    timeout=settings.ES_INDEXING_TIMEOUT. This only affects incremental
    indexing where it gets its own ES.
    
    In the process of fixing that one line, I renamed es_utils.get_es
    to es_utils.get_indexing_es so it's clearer what it's for.
    willkg committed Jan 25, 2012
  3. [bug 720865] Remove race conditions that were causing mapping conflic…

    …ts in production.
    
    * Remove support for putting different doctypes in different indices.
    * Remove the ability to limit reindexing to given doctypes.
    * Make get_mapping() return less boilerplate. Factor that up.
    erikrose committed with willkg Jan 25, 2012
Commits on Jan 18, 2012
  1. [bug 718826, 715932] Make ES indexing less sucky

    * the ES connection already has code for forcing bulk, so we don't need to
      repeat that. this changes the code to push the setting to ES.
    * this also tweaks the estimation code so that it shows minutes and seconds and
      shows the total delta later. Now I can stop running
      "time ./manage.py esreindex".
    * fix esreindex so that you can specify doctypes. This will appropriately
      create/delete indexes so that what you don't want to delete won't get
      deleted.
    * adds basic handling for bad data.
    
      This does a log.exception, but we really should log more than that and/or
      make it more obvious to developers that there's bad data out there.
    
      In the meantime, this allows us to continue indexing.
    * reduced memory usage of indexing by iterating over ids---now it runs on
      my laptop.
    * ghanges _get_index() to get_es_index(). We use it so often it might as
      well be part of the "public API".
    * fixed create/delete indexes so that to switch doctypes to their own index
      is now just a change in settings---no code changes needed.
    * fix DEBUG = True case by reseting queries
    * this also adds a bunch of helpful comments, moves reindex_model to
      SearchMixin.index_all, and has some other cosmetic code cleanup.
    
    End result of this is that indexing doesn't die if it hits bad data, indexing
    takes much less memory to run, you can specify specific doctypes to index at
    the command line, and the code is better.
    willkg committed Jan 13, 2012
Commits on Jan 17, 2012
  1. [bug 714917] Test zendesk calls.

    rlr committed Jan 13, 2012
  2. [bug 714798] Marketplace AAQ.

    * New form for marketplace question flow.
    * Added zendesk lib to vendor.
    * Integrated to zendesk.
    * Linked to from marketplace landing page.
    * Tests.
    rlr committed Jan 11, 2012
  3. fix merge

    readevalprint committed Jan 17, 2012
Commits on Jan 13, 2012
  1. [bug 716008] Fix textarea width.

    James Socol committed Jan 13, 2012
Commits on Jan 12, 2012
  1. [bug 717274] Make sure the threadlocal task set is always instantiate…

    …d, not just on the thread that imported the module.
    erikrose committed Jan 12, 2012
  2. Factor up the triplication from live-indexing signal registration and…

    … handling.
    
    This completely restores my motivation to work on search stuff. Ahhhhh.
    
    Make add_index_task and add_unindex_task into instance methods that do just a single document. We never used them for more than one, and it simplifies things.
    erikrose committed Jan 12, 2012
Commits on Jan 11, 2012
  1. sleep after executing curl in prime_app()

    jasonthomas committed with James Socol Jan 9, 2012
  2. [bug 709080] Refactor ES code into SearchMixin

    This reduces the triplification of everything in es_search modules.
    This reduces the triplification of everything in es_search modules.
    This reduces the triplification of everything in es_search modules.
    
    Also, I introduced ES_INDEXING_TIMEOUT, fixed documentation, and
    cleaned up some other things while I was poking around.
    
    Also also, I nixed all the ES mapping constants. Calling put_mapping
    with typos will kick up an error, so there's no need to additionally
    have constants around to prevent typos.
    willkg committed Jan 6, 2012
  3. [bug 712706] Reindex when tags change

    Reindex questions and documents when tags are added and removed.
    willkg committed Jan 6, 2012
Commits on Jan 10, 2012
  1. [bug 714795] Initial support for marketplace.

    * New landing page at /marketplace
    * Added "marketplace" product for KB articles
    * Fixed base template not linking to right landing page
    rlr committed Jan 4, 2012
Commits on Jan 6, 2012
  1. [bug 715536] Multi-index-task squashing

    Incremental updates (not delete!) tasks all get tossed in a thread-local
    queue. At request_finished, there's a handler that creates tasks from the
    queued tasks. Because the queue is a set, this removes task duplicates
    so we're only creating one index task per request per document.
    
    This also fixes refresh calls so they pass timesleep=0--we don't need to
    wait for a second to update the index.
    
    Note: I didn't change deletes because they only get triggered by one
    model per document (e.g. Question model for question documents in the
    index) and therefore aren't going to hit the same issue we have with
    inserts/updates where in a single http request, we could trigger > 1
    for a specific object.
    willkg committed Jan 5, 2012
Commits on Jan 5, 2012
  1. Always turn the elasticsearch waffle flag on during ES tests.

    It doesn't cost us significant performance, it'll go away eventually, and it saves repetition in test code.
    erikrose committed Jan 5, 2012
  2. Make the rest of our ES tests use modelmakers rather than direct (noi…

    …sier) calls to model constructors.
    erikrose committed Jan 5, 2012
  3. Refactor wiki tests to jibe with elastic testcase changes.

    test_added was redundant with test_deleted, so I removed it. What was the uuid stuff for?
    erikrose committed Jan 5, 2012