Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Commits on Mar 2, 2012
  1. @garlick

    prep for 1.0.6 tag

    garlick authored
  2. @garlick
  3. @garlick

    Change default maxmmap to 0

    garlick authored
    I measure any performance advantage for pynamic when this is
    enabled, and it does increase the virtual memory size of diod
    to obscene levels.  Leave the code here in case it becomes interesting
    to tune this later on.
  4. @garlick

    factor ioctx, path, ppool out to ioctx.c

    garlick authored
    Make IOCtx and Path abstract data types with accessors.
    Add system call wrappers for all the I/O system calls
    so that they work against IOCtx, not fd or dir.
Commits on Feb 29, 2012
  1. @garlick
  2. @garlick
  3. @garlick

    prep for 1.0.5 release

    garlick authored
  4. @garlick
  5. @garlick

    Mmap first 'maxmmap' of a shared file

    garlick authored
    Instead of calling pread multiple times on a shared io context,
    mmap up to 'maxmmap' of a file and call memcpy for read requests contained
    in that area.  This is intended to help parallel executable load time.
    
    The default value of 'maxmmap' is 4MB.  It can be changed via the
    diod.conf maxmmap variable or the diod server -m,--maxmmap command
    line option.
Commits on Feb 28, 2012
  1. @garlick

    prep for 1.0.4 release

    garlick authored
  2. @garlick

    Hold the path lock no longer than necessary

    garlick authored
    This will have a very minor effect, if any.
  3. @garlick
Commits on Feb 24, 2012
  1. @garlick

    prep for 1.0.3 tag

    garlick authored
  2. @garlick
  3. @garlick

    Prep for 1.0.2 tag

    garlick authored
  4. @garlick

    Add 'exportopts' diod.conf config option

    garlick authored
    This will enable exportopts="sharefd" to be enabled globally
    in conjunction with exportall=1, rather than requiring all file
    systems to be listed in the config file in order to activate that
    option.
  5. @garlick

    prep for 1.0.1 tag

    garlick authored
  6. @garlick

    Add 'sharefd' diod.conf export option.

    garlick authored
    This replaces the global SHAREFD flag.
Commits on Feb 23, 2012
  1. @garlick

    Add limited sharing of file descriptors

    garlick authored
    This is a candidate performance/scalability enhancement for the case
    where a parallel job tries to load many shared libraries, or otherwise
    read the same files simultaneously.  We were running out of file descriptors
    in the Pynamic test case, for example, when running with ~2048 tasks
    opening around 500 files each.
    
    First, upon walking a fid to a pathname, put the pathname in a hash
    and create references to it rather than recreating it in every fid.
    This should save a bit of memory.  It also provides a place where fids
    associated with the same file can coordinate sharing.
    
    Next, encapsulate the file descriptor and other "open file" state formerly
    stored in the fid in an IOCtx struct, linked to the fid.  In addition to
    linking to the fid, add it to a linked list on the Path struct.
    
    If a path hashes to an existing entry, and a candidate IOCtx exists
    on the path's list, evaluate the criteria for sharing the IOCtx.  The
    criteria are:
    - must have same path
    - must refer to a regular file (not directory, etc)
    - must be opened by same user
    - must be opened with identical open flags
    - must be opened O_RDONLY (no writing)
    
    The path hash can be viewd by monitoring ctl:files, which has the format:
    
     <refs> <tot open> <act open> <path>
    
    Advisory locking would have been complicated by this change were we
    not promoting record locking to full file locking.  Since only read-only
    file descriptors are shared, there should be no issue with promotion
    of read locks to write locks.
    
    There is a global feature flag to enable sharing that is turned on by
    default in this patch.  TODO: add config file support to enable on a
    per-file system basis, and make it off by default.
Commits on Feb 22, 2012
  1. @garlick
Commits on Feb 17, 2012
  1. @garlick

    prep for 1.0.0 release

    garlick authored
  2. @garlick

    Turn off fidpool debugging

    garlick authored
Commits on Feb 15, 2012
  1. @garlick
Commits on Feb 14, 2012
  1. @garlick

    Turn off flush debugging

    garlick authored
Commits on Feb 13, 2012
  1. @garlick

    Put back request lock

    garlick authored
    Reverting the part of the previous commit which removed the mutex
    and refcount from each request.  I'm not sure I understand why, but
    kern/t31 was failing with this change in place.
Commits on Feb 12, 2012
  1. @garlick

    Remove vestigual request locking and pool

    garlick authored
    Mutex and refcount around Npreq are no longer needed.
    The request pool is probably best left to tcmalloc.
Commits on Feb 11, 2012
  1. @garlick

    prep for 1.0-pre64

    garlick authored
  2. @garlick

    Fix a race when setting req->flushreq

    garlick authored
    We set req->flushreq when walking the work queue holding srv->lock.
    The original reply is sent while req is still in the work queue,
    thus there is a race where flushreq could be set just after the
    postprocess function tests it and the reply discarded.  Defer the
    flushreq reply until after the req has been removed from the work queue.
  3. @garlick

    remove unused structure member

    garlick authored
  4. @garlick
  5. @garlick
  6. @garlick
  7. @garlick

    Change default rwdepth from 32 to 1.

    garlick authored
    Avoid tickling a bug in the kernel client until we fix it.
  8. @garlick

    Tack flushes on to the original request

    garlick authored
    This avoids the case where an Rflush is returned for Treaddir (say)
    and then a Tclunk is sent for the Treaddir's fid causing readdir to
    core dump.  (the refcount on the fid doesn't protect the DIR * from
    being closed/freed underneath.
  9. @garlick

    Avoid dynamically allocating rlerror response

    garlick authored
    This simplifies error handling a bit since in handling an out
    of memory error, we are less likely to get another one while sending
    the error response.
Something went wrong with that request. Please try again.