Skip to content
This repository has been archived by the owner on Sep 12, 2018. It is now read-only.

StorageDriver interface changes and cleanup #820

Merged

Commits on Dec 3, 2014

  1. Use int64 for ReadStream and WriteStream offsets

    This change brings the storagedriver API in line with the Go standard library's
    use of int64 for offsets. The main benefit is simplicity in interfacing with
    the io library reducing the number of type conversions in simple code.
    stevvooe committed Dec 3, 2014
    Configuration menu
    Copy the full SHA
    66107df View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b047c92 View commit details
    Browse the repository at this point in the history
  3. Replace StorageLayer.CurrentSize interface call with Stat

    To support single-flight Size and ModTime queries against backend storage file,
    we are replacing the CurrentSize call with a Stat call. A FileInfo interface is
    provided for backends to provide a type, with a default implementation called
    FileInfoInternal, for use by driver implementations.
    
    More work needs to follow this change to update all the driver implementations.
    stevvooe committed Dec 3, 2014
    Configuration menu
    Copy the full SHA
    ac660e7 View commit details
    Browse the repository at this point in the history
  4. Remove size argument and using io.Reader for StorageDriver.WriteStream

    We are change the the rpc call for WriteStream to not require the size
    argument, opting to drive the process with io.Reader. The main issue was that
    io.Reader may return io.EOF before reaching size, making the error handling
    around this condition for callers more complex. To complement this, WriteStream
    now returns the number of successfully written bytes.
    
    The method no longer requires an io.ReadCloser, opting to require just an
    io.Reader. This keeps the reader under the control of the caller, which
    provides more flexibility.
    
    This also begins to address some of the problems described in docker-archive#791.
    stevvooe committed Dec 3, 2014
    Configuration menu
    Copy the full SHA
    2e3ecdc View commit details
    Browse the repository at this point in the history

Commits on Dec 4, 2014

  1. Update testsuite with storagedriver interface changes

    This change updates the testsuite to migrate to the new driver interface. This
    includes the new Stat call, changes to int64 over uint64 and the changes to the
    WriteStream signature. Several test cases have been added to vet
    implementations against various assumptions.
    stevvooe committed Dec 4, 2014
    Configuration menu
    Copy the full SHA
    2037b1d View commit details
    Browse the repository at this point in the history
  2. Migrate filesystem driver to new storagedriver calls

    The filesystem driver has been migrated to impleemnt the storagedriver
    interface changes. Most interetingly, this provides a filesystem-based
    implementation of the Stat driver call. With this comes some refactoring of
    Reads and Write to be much simpler and more robust.
    
    The IPC tests have been disabled to stability problems that we'll have to
    troubleshoot at a later date.
    stevvooe committed Dec 4, 2014
    Configuration menu
    Copy the full SHA
    ab9570f View commit details
    Browse the repository at this point in the history

Commits on Dec 5, 2014

  1. Refactor inmemory driver for Stat and WriteStream methods

    This change started out as simply updating the existing inmemory driver to
    implement the new Stat call. After struggling with the map based
    implementation, it has been refactored to be a tree-based implementation.
    
    This process has exposed a few missing error cases in the StorageDriver API
    that should be addressed in the coming weeks.
    stevvooe committed Dec 5, 2014
    Configuration menu
    Copy the full SHA
    2ebc373 View commit details
    Browse the repository at this point in the history
  2. Update storage package to use StorageDriver.Stat

    This change updates the backend storage package that consumes StorageDriver to
    use the new Stat call, over CurrentSize. It also makes minor updates for using
    WriteStream and ReadStream.
    stevvooe committed Dec 5, 2014
    Configuration menu
    Copy the full SHA
    70ab06b View commit details
    Browse the repository at this point in the history
  3. Add checks for ReadStream offset boundary conditions

    Several checks for ReadStream with offset around boundary conditions were
    missing. The new checks ensure negative offsets are detected and io.EOF is
    returned properly when trying to read past the end of a file. The filesystem
    and inmemory driver have been updated accordingly.
    
    An outline of missing checks for List are also part of this commit. Action will
    be taken here based on discussion in issue docker-archive#819.
    stevvooe committed Dec 5, 2014
    Configuration menu
    Copy the full SHA
    d703a86 View commit details
    Browse the repository at this point in the history
  4. Disable s3, azure and ipc packages and testing

    The packages causing build errors are being disabled for now to let us split up
    the work in the different driver implementations without blocking integration
    into the main branch. The s3 and azure implementations need some effort to add
    Stat support. The ipc package needs that work plus some care around hanging
    send calls.
    stevvooe committed Dec 5, 2014
    Configuration menu
    Copy the full SHA
    8cb0e33 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    2f78886 View commit details
    Browse the repository at this point in the history
  6. Address PathNotFoundError in (*manifestStore).Exists

    Exists was returning an error when encountering a PathNotFoundError when it
    should just return false without an error.
    stevvooe committed Dec 5, 2014
    Configuration menu
    Copy the full SHA
    1a75fcc View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2014

  1. Configuration menu
    Copy the full SHA
    a7017d6 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    14e7adb View commit details
    Browse the repository at this point in the history
  3. Disable race detector during tests due to memory usage

    The tests are using way too much memory with the race detector enabled causing
    the build machines to fall over. Cursory profiling shows no leaks but it may
    need a closer look. For now, it will be disabled but this cannot be permanent.
    stevvooe committed Dec 6, 2014
    Configuration menu
    Copy the full SHA
    e15e07c View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2014

  1. Configuration menu
    Copy the full SHA
    e364e71 View commit details
    Browse the repository at this point in the history