Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inplace Import via SymlinkFileTransfer (See #11573) #1989

Merged
merged 19 commits into from
Jan 23, 2014

Commits on Jan 17, 2014

  1. Fix header of ImportLibrary

    joshmoore committed Jan 17, 2014
    Configuration menu
    Copy the full SHA
    ce57b30 View commit details
    Browse the repository at this point in the history
  2. Refactor uploadFile to FileTransfer

    The uploadFile method has a number of responsibilities
    that were not easily modifiable. Rather than insert
    further `if (symlink) {} then {}` complications, its
    been extracted behind the new `FileTransfer` interface
    to permit others to inject their own logic.
    joshmoore committed Jan 17, 2014
    Configuration menu
    Copy the full SHA
    fa5c29c View commit details
    Browse the repository at this point in the history
  3. Add trivial log statement

    joshmoore committed Jan 17, 2014
    Configuration menu
    Copy the full SHA
    b2265d1 View commit details
    Browse the repository at this point in the history
  4. Refactor to use TransferState

    The number of parameters passed to the `transfer` method
    was daunting at best. Further, much of the uploadFile method
    was doing logging so that moving the execution into the state
    class, the FileTransfer implementors can be ignorant of most
    of the arguments.
    joshmoore committed Jan 17, 2014
    Configuration menu
    Copy the full SHA
    1ac90e1 View commit details
    Browse the repository at this point in the history
  5. Refactor out AbstractFileTransfer for SymlinkFileTransfer

    The rest of the reusable logic from uploadFile has now been
    moved to AbstractFileTransfar and re-used by the new
    SymlinkFileTransfer class.
    
    Several remote-access methods were needed in the ImportLibrary
    itself, though these could perhaps live in a more general
    gateway-style utility.
    joshmoore committed Jan 17, 2014
    Configuration menu
    Copy the full SHA
    01a3ad7 View commit details
    Browse the repository at this point in the history
  6. Implement command-line passing

    The following options are now all available
    for choosing which transfer method to use:
    
    ```
     bin/omero import -t upload
     bin/omero import --transfer upload
    
     bin/omero import -t symlink
     bin/omero import -- -t symlink
    
     bin/omero import -t your.class.name.Here
    ```
    joshmoore committed Jan 17, 2014
    Configuration menu
    Copy the full SHA
    569208d View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    618faea View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    52180d5 View commit details
    Browse the repository at this point in the history

Commits on Jan 21, 2014

  1. Configuration menu
    Copy the full SHA
    9725a55 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7c0463f View commit details
    Browse the repository at this point in the history

Commits on Jan 22, 2014

  1. Introduce afterSuccess in the FileTransfer interface

    A simple `mv` command is not possible since 1) the file
    must be present for subsequent readds and 2) it's not
    safe to remove the file until the import has completed
    successfully.
    
    Now, users of the `FileTransfer` API have the responsibility
    of calling `afterSuccess` once the `ImportLibrary` has done
    its work. An exception may be thrown if cleanup is not
    possible.
    joshmoore committed Jan 22, 2014
    Configuration menu
    Copy the full SHA
    4e2d88a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f3812a9 View commit details
    Browse the repository at this point in the history
  3. Update CLI help string

    joshmoore committed Jan 22, 2014
    Configuration menu
    Copy the full SHA
    eab929b View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    8fcee93 View commit details
    Browse the repository at this point in the history
  5. Hide Advanced arguments behind --transfer-help

    ```
    $ bin/omero import -- --transfer-help
    ...
    Advanced arguments:
      --transfer=ARG    File transfer method
          Examples:
           -t upload           # Default
           -t ln               # Use hard-link. Locally only!
           -t ln_rm            # Caution! Hard-link followed by rm. Locally only!
           -t ln_s             # Use symlnk. Locally only!
           -t some.class.Name  # Use a class on the CLASSPATH
    
    ex. importer-cli --tranasfer=ln_s foo.tiff
    
    Report bugs to <ome-users@lists.openmicroscopy.org.uk>
    
    ```
    joshmoore committed Jan 22, 2014
    Configuration menu
    Copy the full SHA
    4aa9f8a View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    257d38e View commit details
    Browse the repository at this point in the history
  7. Add Java7 TODOs

    joshmoore committed Jan 22, 2014
    Configuration menu
    Copy the full SHA
    2a8c58c View commit details
    Browse the repository at this point in the history

Commits on Jan 23, 2014

  1. Add CommentAnnotation(ns=.../fileTransfer) with class of FT

    With this change, an annotation is added to the `Fileset` object
    on import describing which `FileTransfer` implementation was used.
    
    Use of the default `UploadFileTransfer` is ignored; otherwise, the
    FQN is stored as the content of the annotation. If we decided to
    move to subclasses of `Fileset` these values should suffice.
    
    Representative output after an import includes:
    
    ```
    ome2=> \x
    ome2=> select discriminator, id, ns, textValue from annotation order by id desc limit 2;
    -[ RECORD 1 ]-+---------------------------------------------------
    discriminator | /type/OriginalFile/
    id            | 3102
    ns            | openmicroscopy.org/omero/import/logFile
    textvalue     |
    -[ RECORD 2 ]-+---------------------------------------------------
    discriminator | /basic/text/comment/
    id            | 3101
    ns            | openmicroscopy.org/omero/import/fileTransfer
    textvalue     | ome.formats.importer.transfers.SymlinkFileTransfer
    ```
    joshmoore committed Jan 23, 2014
    Configuration menu
    Copy the full SHA
    c1d285a View commit details
    Browse the repository at this point in the history
  2. Fix 'tranasfer' typo

    joshmoore committed Jan 23, 2014
    Configuration menu
    Copy the full SHA
    9f3706f View commit details
    Browse the repository at this point in the history