Commits on Feb 24, 2012
  1. @garlick

    prep for 1.0.1 tag

    garlick committed Feb 24, 2012
  2. @garlick

    Add 'sharefd' diod.conf export option.

    This replaces the global SHAREFD flag.
    garlick committed Feb 24, 2012
Commits on Feb 23, 2012
  1. @garlick

    Add limited sharing of file descriptors

    This is a candidate performance/scalability enhancement for the case
    where a parallel job tries to load many shared libraries, or otherwise
    read the same files simultaneously.  We were running out of file descriptors
    in the Pynamic test case, for example, when running with ~2048 tasks
    opening around 500 files each.
    
    First, upon walking a fid to a pathname, put the pathname in a hash
    and create references to it rather than recreating it in every fid.
    This should save a bit of memory.  It also provides a place where fids
    associated with the same file can coordinate sharing.
    
    Next, encapsulate the file descriptor and other "open file" state formerly
    stored in the fid in an IOCtx struct, linked to the fid.  In addition to
    linking to the fid, add it to a linked list on the Path struct.
    
    If a path hashes to an existing entry, and a candidate IOCtx exists
    on the path's list, evaluate the criteria for sharing the IOCtx.  The
    criteria are:
    - must have same path
    - must refer to a regular file (not directory, etc)
    - must be opened by same user
    - must be opened with identical open flags
    - must be opened O_RDONLY (no writing)
    
    The path hash can be viewd by monitoring ctl:files, which has the format:
    
     <refs> <tot open> <act open> <path>
    
    Advisory locking would have been complicated by this change were we
    not promoting record locking to full file locking.  Since only read-only
    file descriptors are shared, there should be no issue with promotion
    of read locks to write locks.
    
    There is a global feature flag to enable sharing that is turned on by
    default in this patch.  TODO: add config file support to enable on a
    per-file system basis, and make it off by default.
    garlick committed Feb 22, 2012
Commits on Feb 22, 2012
  1. @garlick
Commits on Feb 17, 2012
  1. @garlick

    prep for 1.0.0 release

    garlick committed Feb 17, 2012
  2. @garlick

    Turn off fidpool debugging

    garlick committed Feb 17, 2012
Commits on Feb 15, 2012
  1. @garlick
Commits on Feb 14, 2012
  1. @garlick

    Turn off flush debugging

    garlick committed Feb 14, 2012
Commits on Feb 13, 2012
  1. @garlick

    Put back request lock

    Reverting the part of the previous commit which removed the mutex
    and refcount from each request.  I'm not sure I understand why, but
    kern/t31 was failing with this change in place.
    garlick committed Feb 13, 2012
Commits on Feb 12, 2012
  1. @garlick

    Remove vestigual request locking and pool

    Mutex and refcount around Npreq are no longer needed.
    The request pool is probably best left to tcmalloc.
    garlick committed Feb 12, 2012
Commits on Feb 11, 2012
  1. @garlick

    prep for 1.0-pre64

    garlick committed Feb 11, 2012
  2. @garlick

    Fix a race when setting req->flushreq

    We set req->flushreq when walking the work queue holding srv->lock.
    The original reply is sent while req is still in the work queue,
    thus there is a race where flushreq could be set just after the
    postprocess function tests it and the reply discarded.  Defer the
    flushreq reply until after the req has been removed from the work queue.
    garlick committed Feb 11, 2012
  3. @garlick

    remove unused structure member

    garlick committed Feb 11, 2012
  4. @garlick
  5. @garlick
  6. @garlick
  7. @garlick

    Change default rwdepth from 32 to 1.

    Avoid tickling a bug in the kernel client until we fix it.
    garlick committed Feb 10, 2012
  8. @garlick

    Tack flushes on to the original request

    This avoids the case where an Rflush is returned for Treaddir (say)
    and then a Tclunk is sent for the Treaddir's fid causing readdir to
    core dump.  (the refcount on the fid doesn't protect the DIR * from
    being closed/freed underneath.
    garlick committed Feb 10, 2012
  9. @garlick

    Avoid dynamically allocating rlerror response

    This simplifies error handling a bit since in handling an out
    of memory error, we are less likely to get another one while sending
    the error response.
    garlick committed Feb 10, 2012
Commits on Feb 10, 2012
  1. @garlick
  2. @garlick

    Eliminiate extra server request queue

    This removes the 'D' state in dtop request view.
    Completed, unreplied requests will included in the 'R' (running) state.
    garlick committed Feb 10, 2012
Commits on Feb 9, 2012
  1. @garlick
  2. @garlick
  3. @garlick

    Note any EBADF errors from close when fid is freed

    Just chased down a double-close problem and this would have helped.
    Should never happen unless something is wrong with file descriptor
    accounting.
    garlick committed Feb 9, 2012
  4. @garlick

    prep for 1.0-pre63 tag

    garlick committed Feb 9, 2012
  5. @garlick
  6. @garlick

    Clean up directory code a bit

    garlick committed Feb 8, 2012
Commits on Feb 8, 2012
  1. @garlick
  2. @garlick

    Don't attempt to fdopendir() every Tlopened file

    This appears to have eliminated some spurious EBADF errors that were
    occuring when running a recursive find in a diod-backed file system.
    garlick committed Feb 8, 2012
  3. @garlick

    Fix fid accounting on flushed Tclunk/Tremove/Twalk

    These operations, if successfully flushed, need to be undone
    with regard to fid accounting at least.  See Tflush(9p)
    garlick committed Feb 8, 2012
  4. @garlick

    Add 'loosefid' flag to work around buggy client

    v9fs does not currently handle a flushed Tclunk/Tremove properly.
    It should behave as though these ops were never sent, meaning the fid
    remains valid and cannot be reused.  However it unconditionally frees
    them internally.  This must be fixed, however it is ueful to be able
    to interoperate with older clients, so we create this workaround and
    enable it by default.
    garlick committed Feb 8, 2012
  5. @garlick

    revert blocking np_fid_create

    garlick committed Feb 8, 2012
Commits on Feb 1, 2012
  1. @garlick
Commits on Jan 31, 2012
  1. @garlick
Commits on Jan 27, 2012
  1. @garlick

    Properly handle a flushed walk.

    Also, further improvements reducing the race between clunked
    walk/clunk/remove and walk newfid.  Introduce the a blocking version
    of np_fid_create that makes the server tolerant of either ordering
    of flush response and flushed request retirement, with fid accounting.
    garlick committed Jan 27, 2012