StorageDriver interface changes and cleanup #820
StorageDriver interface changes and cleanup #820
Commits on Dec 3, 2014
-
Use int64 for ReadStream and WriteStream offsets
This change brings the storagedriver API in line with the Go standard library's use of int64 for offsets. The main benefit is simplicity in interfacing with the io library reducing the number of type conversions in simple code.
Configuration menu - View commit details
-
Copy full SHA for 66107df - Browse repository at this point
Copy the full SHA 66107dfView commit details -
Configuration menu - View commit details
-
Copy full SHA for b047c92 - Browse repository at this point
Copy the full SHA b047c92View commit details -
Replace StorageLayer.CurrentSize interface call with Stat
To support single-flight Size and ModTime queries against backend storage file, we are replacing the CurrentSize call with a Stat call. A FileInfo interface is provided for backends to provide a type, with a default implementation called FileInfoInternal, for use by driver implementations. More work needs to follow this change to update all the driver implementations.
Configuration menu - View commit details
-
Copy full SHA for ac660e7 - Browse repository at this point
Copy the full SHA ac660e7View commit details -
Remove size argument and using io.Reader for StorageDriver.WriteStream
We are change the the rpc call for WriteStream to not require the size argument, opting to drive the process with io.Reader. The main issue was that io.Reader may return io.EOF before reaching size, making the error handling around this condition for callers more complex. To complement this, WriteStream now returns the number of successfully written bytes. The method no longer requires an io.ReadCloser, opting to require just an io.Reader. This keeps the reader under the control of the caller, which provides more flexibility. This also begins to address some of the problems described in docker-archive#791.
Configuration menu - View commit details
-
Copy full SHA for 2e3ecdc - Browse repository at this point
Copy the full SHA 2e3ecdcView commit details
Commits on Dec 4, 2014
-
Update testsuite with storagedriver interface changes
This change updates the testsuite to migrate to the new driver interface. This includes the new Stat call, changes to int64 over uint64 and the changes to the WriteStream signature. Several test cases have been added to vet implementations against various assumptions.
Configuration menu - View commit details
-
Copy full SHA for 2037b1d - Browse repository at this point
Copy the full SHA 2037b1dView commit details -
Migrate filesystem driver to new storagedriver calls
The filesystem driver has been migrated to impleemnt the storagedriver interface changes. Most interetingly, this provides a filesystem-based implementation of the Stat driver call. With this comes some refactoring of Reads and Write to be much simpler and more robust. The IPC tests have been disabled to stability problems that we'll have to troubleshoot at a later date.
Configuration menu - View commit details
-
Copy full SHA for ab9570f - Browse repository at this point
Copy the full SHA ab9570fView commit details
Commits on Dec 5, 2014
-
Refactor inmemory driver for Stat and WriteStream methods
This change started out as simply updating the existing inmemory driver to implement the new Stat call. After struggling with the map based implementation, it has been refactored to be a tree-based implementation. This process has exposed a few missing error cases in the StorageDriver API that should be addressed in the coming weeks.
Configuration menu - View commit details
-
Copy full SHA for 2ebc373 - Browse repository at this point
Copy the full SHA 2ebc373View commit details -
Update storage package to use StorageDriver.Stat
This change updates the backend storage package that consumes StorageDriver to use the new Stat call, over CurrentSize. It also makes minor updates for using WriteStream and ReadStream.
Configuration menu - View commit details
-
Copy full SHA for 70ab06b - Browse repository at this point
Copy the full SHA 70ab06bView commit details -
Add checks for ReadStream offset boundary conditions
Several checks for ReadStream with offset around boundary conditions were missing. The new checks ensure negative offsets are detected and io.EOF is returned properly when trying to read past the end of a file. The filesystem and inmemory driver have been updated accordingly. An outline of missing checks for List are also part of this commit. Action will be taken here based on discussion in issue docker-archive#819.
Configuration menu - View commit details
-
Copy full SHA for d703a86 - Browse repository at this point
Copy the full SHA d703a86View commit details -
Disable s3, azure and ipc packages and testing
The packages causing build errors are being disabled for now to let us split up the work in the different driver implementations without blocking integration into the main branch. The s3 and azure implementations need some effort to add Stat support. The ipc package needs that work plus some care around hanging send calls.
Configuration menu - View commit details
-
Copy full SHA for 8cb0e33 - Browse repository at this point
Copy the full SHA 8cb0e33View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2f78886 - Browse repository at this point
Copy the full SHA 2f78886View commit details -
Address PathNotFoundError in (*manifestStore).Exists
Exists was returning an error when encountering a PathNotFoundError when it should just return false without an error.
Configuration menu - View commit details
-
Copy full SHA for 1a75fcc - Browse repository at this point
Copy the full SHA 1a75fccView commit details
Commits on Dec 6, 2014
-
Configuration menu - View commit details
-
Copy full SHA for a7017d6 - Browse repository at this point
Copy the full SHA a7017d6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 14e7adb - Browse repository at this point
Copy the full SHA 14e7adbView commit details -
Disable race detector during tests due to memory usage
The tests are using way too much memory with the race detector enabled causing the build machines to fall over. Cursory profiling shows no leaks but it may need a closer look. For now, it will be disabled but this cannot be permanent.
Configuration menu - View commit details
-
Copy full SHA for e15e07c - Browse repository at this point
Copy the full SHA e15e07cView commit details
Commits on Dec 8, 2014
-
Configuration menu - View commit details
-
Copy full SHA for e364e71 - Browse repository at this point
Copy the full SHA e364e71View commit details