Commits on Feb 29, 2012
  1. prep for 1.0.5 release

    garlick committed Feb 29, 2012
  2. Mmap first 'maxmmap' of a shared file

    garlick committed Feb 29, 2012
    Instead of calling pread multiple times on a shared io context,
    mmap up to 'maxmmap' of a file and call memcpy for read requests contained
    in that area.  This is intended to help parallel executable load time.
    The default value of 'maxmmap' is 4MB.  It can be changed via the
    diod.conf maxmmap variable or the diod server -m,--maxmmap command
    line option.
Commits on Feb 28, 2012
  1. prep for 1.0.4 release

    garlick committed Feb 28, 2012
  2. Hold the path lock no longer than necessary

    garlick committed Feb 28, 2012
    This will have a very minor effect, if any.
Commits on Feb 24, 2012
  1. prep for 1.0.3 tag

    garlick committed Feb 24, 2012
  2. Prep for 1.0.2 tag

    garlick committed Feb 24, 2012
  3. Add 'exportopts' diod.conf config option

    garlick committed Feb 24, 2012
    This will enable exportopts="sharefd" to be enabled globally
    in conjunction with exportall=1, rather than requiring all file
    systems to be listed in the config file in order to activate that
  4. prep for 1.0.1 tag

    garlick committed Feb 24, 2012
  5. Add 'sharefd' diod.conf export option.

    garlick committed Feb 24, 2012
    This replaces the global SHAREFD flag.
Commits on Feb 23, 2012
  1. Add limited sharing of file descriptors

    garlick committed Feb 23, 2012
    This is a candidate performance/scalability enhancement for the case
    where a parallel job tries to load many shared libraries, or otherwise
    read the same files simultaneously.  We were running out of file descriptors
    in the Pynamic test case, for example, when running with ~2048 tasks
    opening around 500 files each.
    First, upon walking a fid to a pathname, put the pathname in a hash
    and create references to it rather than recreating it in every fid.
    This should save a bit of memory.  It also provides a place where fids
    associated with the same file can coordinate sharing.
    Next, encapsulate the file descriptor and other "open file" state formerly
    stored in the fid in an IOCtx struct, linked to the fid.  In addition to
    linking to the fid, add it to a linked list on the Path struct.
    If a path hashes to an existing entry, and a candidate IOCtx exists
    on the path's list, evaluate the criteria for sharing the IOCtx.  The
    criteria are:
    - must have same path
    - must refer to a regular file (not directory, etc)
    - must be opened by same user
    - must be opened with identical open flags
    - must be opened O_RDONLY (no writing)
    The path hash can be viewd by monitoring ctl:files, which has the format:
     <refs> <tot open> <act open> <path>
    Advisory locking would have been complicated by this change were we
    not promoting record locking to full file locking.  Since only read-only
    file descriptors are shared, there should be no issue with promotion
    of read locks to write locks.
    There is a global feature flag to enable sharing that is turned on by
    default in this patch.  TODO: add config file support to enable on a
    per-file system basis, and make it off by default.
Commits on Feb 22, 2012
Commits on Feb 17, 2012
  1. prep for 1.0.0 release

    garlick committed Feb 17, 2012
  2. Turn off fidpool debugging

    garlick committed Feb 17, 2012
Commits on Feb 15, 2012
Commits on Feb 14, 2012
  1. Turn off flush debugging

    garlick committed Feb 14, 2012
Commits on Feb 13, 2012
  1. Put back request lock

    garlick committed Feb 13, 2012
    Reverting the part of the previous commit which removed the mutex
    and refcount from each request.  I'm not sure I understand why, but
    kern/t31 was failing with this change in place.
Commits on Feb 12, 2012
  1. Remove vestigual request locking and pool

    garlick committed Feb 12, 2012
    Mutex and refcount around Npreq are no longer needed.
    The request pool is probably best left to tcmalloc.
Commits on Feb 11, 2012
  1. prep for 1.0-pre64

    garlick committed Feb 11, 2012
  2. Fix a race when setting req->flushreq

    garlick committed Feb 11, 2012
    We set req->flushreq when walking the work queue holding srv->lock.
    The original reply is sent while req is still in the work queue,
    thus there is a race where flushreq could be set just after the
    postprocess function tests it and the reply discarded.  Defer the
    flushreq reply until after the req has been removed from the work queue.
  3. Change default rwdepth from 32 to 1.

    garlick committed Feb 11, 2012
    Avoid tickling a bug in the kernel client until we fix it.
  4. Tack flushes on to the original request

    garlick committed Feb 11, 2012
    This avoids the case where an Rflush is returned for Treaddir (say)
    and then a Tclunk is sent for the Treaddir's fid causing readdir to
    core dump.  (the refcount on the fid doesn't protect the DIR * from
    being closed/freed underneath.
  5. Avoid dynamically allocating rlerror response

    garlick committed Feb 11, 2012
    This simplifies error handling a bit since in handling an out
    of memory error, we are less likely to get another one while sending
    the error response.
Commits on Feb 10, 2012
  1. Eliminiate extra server request queue

    garlick committed Feb 10, 2012
    This removes the 'D' state in dtop request view.
    Completed, unreplied requests will included in the 'R' (running) state.
Commits on Feb 9, 2012
  1. Note any EBADF errors from close when fid is freed

    garlick committed Feb 9, 2012
    Just chased down a double-close problem and this would have helped.
    Should never happen unless something is wrong with file descriptor
  2. prep for 1.0-pre63 tag

    garlick committed Feb 9, 2012