Permalink
Commits on Feb 18, 2013
  1. Some search fixes

    * Fixes excerpts to use the dw1delta index (which falls back to dw1).
    
    * Fixes the UI to properly save the 'include comments' flag.
    
    * Makes the scheduler faster (for re-copies).
    
    * Adds the 'touchtime' column which lets us properly index updated
    content. The old method relied on the id field incrementing which wasn't
    true if content got edited.
    zorkian committed Feb 18, 2013
  2. Merge pull request #267 from afuna/import-top-entry

    Skip both DOOO entries with a later date, and stickies
    afuna committed Feb 18, 2013
  3. Skip both DOOO entries with a later date, and stickies

    Sticky entries are not marked, so we have to make a best effort by
    comparing the jitemids of all non-DOOO entries that we got
    afuna committed Feb 18, 2013
Commits on Feb 15, 2013
  1. Optimize the sphinx copier

    The copier is now resilient and efficient against journals with huge
    numbers of comments. It's slightly less efficient with huge numbers of
    entries, but it still seems to work fine for all of DW's accounts.
    
    Thanks to <user name="cfud"> and <user name="sixwordstories"> for making
    this necessary. :-)
    zorkian committed Feb 15, 2013
  2. Add source data to copier jobs

    This allows us to track where a copy job was scheduled from.
    
    Patch by <user name="fu">.
    zorkian committed Feb 15, 2013
Commits on Feb 13, 2013
  1. Merge pull request #261 from afuna/fail-return-bug4855

    (Bug 4855) Do not proceed when we fail
    afuna committed Feb 13, 2013
Commits on Feb 12, 2013
  1. (Bug 4855) Do not proceed when we fail

    When we fail, cut it short. Don't continue.
    afuna committed Feb 12, 2013
Commits on Feb 9, 2013
  1. Merge pull request #251 from alierak/searchdb-bug4871

    (Bug 4871) rename search column journal_id to journalid
    alierak committed Feb 9, 2013
Commits on Feb 8, 2013
  1. Fix miscellaneous Sphinx DB usage

    These two codepaths use the Sphinx DB. The table changed.
    zorkian committed Feb 8, 2013
  2. Fix case where OpenIDs get underscores

    URLs should have dashes. This is one case where that happens.
    zorkian committed Feb 8, 2013
  3. Sphinx improvements, comment indexing

    Performance improvements:
    
    * Do not use AUTO_INCREMENT, use a better PRIMARY KEY that InnoDB that
    reflects the data access patterns. (All accesses are either single-items
    or groups-by-journal.)
    
    * Use COMPRESS to save disk space.
    
    * Don't copy entire journals every time something updates.
    
    * Only copy once when imported -- at the end.
    
    Functionality improvements:
    
    * Add comment indexing and searching.
    
    * Support delta indices for faster indexing cycles.
    
    IMPORTANT: These changes are backwards-incompatible with search
    installations. If you deploy this code, you will need to update
    your database schema and delete the table and reimport (to get the
    compression).
    zorkian committed Feb 1, 2013
Commits on Feb 6, 2013
  1. Merge pull request #249 from afuna/import-dooo-first-entry-bug4855

    (Bug 4855) Get the first non-DOOO entry when importing
    afuna committed Feb 6, 2013
  2. (Bug 4855) Get the first non-DOOO entry when importing

    Getting the latest entry via the protocol gives us the latest entry
    according to eventtime (user-provided time) rather than logtime.
    Sometimse this could be a DOOO entry, which then means that everything
    after that DOOO entry will not be imported. So let's try to find the
    first non-DOOO entry, as that will be the latest according to logtime.
    afuna committed Feb 1, 2013
  3. Merge pull request #248 from afuna/fix-logging

    Reversed the logic for logging this statement. Oops
    afuna committed Feb 6, 2013
Commits on Feb 1, 2013
Commits on Jan 25, 2013
  1. Merge pull request #237 from afuna/importer-claimed_by-bug4854

    (Bug 4865) Temporary fail the import if we weren't able to load a user
    zorkian committed Jan 25, 2013
  2. (Bug 4865) Temporary fail the import if we weren't able to load a user

    * fail the import with a list of users we couldn't load, if we weren't
      able to load users from the remote site
    * fix bug which prevented us from retrying users after they've timed out
      once
    
    Based on discussion over at dreamwidth/dw-free#198:
    
    The error is caused by the openid user ($ou) not actually existing. The
    most common reason is that we timed out while fetching data on the $ou.
    
    We can't just let the importer continue in this case, because that would
    make the comment be posted (wrongly) as by an anonymous user. So we've
    decided to make the import temp-fail with the list of users we failed
    on. We can pick it up again on a retry.
    
    Problem again: subtle bug meant that once we failed to fetch a
    particular user, we never again retried. If it timed out, we returned
    (and stored in the db) 0, instead of NULL. Later on, when retrieving the
    values, we check for defined-ness. Fix is to set an explicit return
    undef for the future. For existing cases, we check for truth value of
    $oid, instead of defined-ness. (This also means that we finally log data
    on this user if something goes wrong)
    
    Note that the temp-fail listed above means that a problem user which we
    somehow never fetch, could still cause all imports where they commented
    to fail entirely. But now we know who they are, and we have a working
    retry mechanism in place to guard against timeouts.
    
    *If* the retries don't work for some unforeseen reason, we have
    something to work with once the repeated failure to import has been
    reported.
    afuna committed Jan 25, 2013
Commits on Jan 15, 2013
  1. Merge pull request #226 from afuna/importer-log-bug4844

    (Bug 4844) Pass in a $log sub
    afuna committed Jan 15, 2013
  2. (Bug 4844) Pass in a $log sub

    So that context may determine where this warning should be printed.
    afuna committed Jan 15, 2013
Commits on Jan 14, 2013
  1. Merge pull request #223 from afuna/screened-read-bug4841

    (Bug 4841) Use the correct variable
    afuna committed Jan 14, 2013
  2. (Bug 4841) Use the correct variable

    Also add comments clarifying the difference between the three user
    variables
    afuna committed Jan 14, 2013
  3. Merge pull request #214 from ninetyd/release-1.13

    Bug 4834 - Summertime: resource credit missing from Tennis Court
    afuna committed Jan 14, 2013
Commits on Jan 12, 2013
Commits on Jan 11, 2013
  1. Merge pull request #210 from afuna/spellcheck-security-bug4826

    (Bug 4826) Remember the previous security level after spellcheck
    afuna committed Jan 11, 2013
  2. (Bug 4826) Remember the previous security level after spellcheck

    * for reasons related to changing minsecurity, we rebuild the security
      level dropdown with JS. We go back to the dropdown's displayed
      value only if it the user had already explicitly selected that level,
      rather than us having adjusted it because of minsecurity.
    
      See SHA: d1b7b5f for a detailed breakdown of this case.
    
      Because we wait until the user has explicitly chosen a security level
      from the dropdown, this means that the value of the dropdown on load
      is ignored. This is not an issue when making a new entry
      (minsecurity), and not an issue when editing (that code path is not
      triggered on edit), but is an issue when spellchecking (the initial
      value *has* been explicitly selected by the user).
    
      This is much longer than the actual code, but the solution then is to
      mark the form as having been spell-checked, and then trust that
      initial value as being one the user chose deliberately.
    
    * also removes a couple TODOs re: spellcheck which are no longer
      relevant since spellcheck is handled for those cases
    afuna committed Jan 11, 2013
  3. Merge pull request #208 from ninetyd/release-1.13

    Misc. Styles: minor fixes and improvements on Icons Page
    afuna committed Jan 11, 2013
  4. Merge pull request #209 from afuna/auth-digest-bug4829

    (Bug 4829) Fix authenticate() call and returned status
    afuna committed Jan 11, 2013
  5. (Bug 4829) Fix authenticate() call and returned status

    Two things:
    * Fixed the method call (oh perl, you let us shoot ourselves in the foot
      sometimes)
    * Explicitly set the returned status code instead of just the status
    * line (see Bug 4832 for more info)
    afuna committed Jan 11, 2013
  6. Adjust output of the pbadm script.

    zorkian committed with afuna Jan 11, 2013
Commits on Jan 9, 2013
  1. Bug 4823 - Summertime: minor fix on Icons page

    Bug 4824 - Abstractia: minor display issue on Icons page
    Bug 4825 - Database: add shadow around icons on icons page
    http://bugs.dwscoalition.org/show_bug.cgi?id=4823
    http://bugs.dwscoalition.org/show_bug.cgi?id=4824
    http://bugs.dwscoalition.org/show_bug.cgi?id=4825
    -- Consistent vertical alignment in Summertime
    -- Consistent display and vertical alignement in Abstractia
    -- Add shadow effect around icons on Icons page for a consistent look
    ninetyd committed Jan 9, 2013
  2. Merge pull request #204 from momijizukamori/release-1.13

    Fix for misaligned icon pages
    afuna committed Jan 9, 2013
  3. Merge pull request #203 from afuna/middle-click-bug4821

    (Bug 4821) Only handle left-click for tracking new entries
    afuna committed Jan 9, 2013