Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

12176 attach fsfiles #2503

Merged
merged 30 commits into from
May 19, 2014
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
c591292
Extend the importMIF test method
joshmoore Apr 25, 2014
0a76867
Initial test to create pre-FS data
joshmoore Apr 25, 2014
c07d6b0
Add uploadFileset method
joshmoore Apr 29, 2014
354f8e6
Move getManagedRepository to client.py
joshmoore May 4, 2014
0c41ce0
Better exception handling of versionInfo
joshmoore May 4, 2014
b905e2e
clients.py: provide write_stream helper
joshmoore May 4, 2014
d7cb285
Use importFileset for synthetic data
joshmoore May 1, 2014
eb512fd
Working re-attachment (Pixels remaining)
joshmoore May 5, 2014
3b42192
Rename BEC.groupPermissions for clarity
joshmoore May 14, 2014
6d40623
Remove new ManagedRepositoryI.uploadFileset method
joshmoore May 14, 2014
7cbec9c
Refactor post-process check to allowUpdateOrDelete
joshmoore May 14, 2014
2777445
ImageBinariesRequest command with deletePixels
joshmoore May 14, 2014
52cc2c4
Fix AbstractGraphTest compilation
joshmoore May 14, 2014
59cec3b
Implement pyramid delete
joshmoore May 14, 2014
1efdd84
Remove unneeded command options
joshmoore May 14, 2014
f73feec
Fix NullHandleException on slice2html
joshmoore May 15, 2014
e0ac7a6
Refactor ManageImageBinaries (was ImageBinariesRequest)
joshmoore May 15, 2014
f116e39
Test cleanup after refactoring
joshmoore May 15, 2014
200be7d
Add null check to requireFileset
joshmoore May 15, 2014
1bc962e
Add --extended option to 'fs archived'
joshmoore May 15, 2014
49a8d20
Renamed 'fs archived' to 'fs images'
joshmoore May 15, 2014
44eb427
Remove extra slash from repos
joshmoore May 15, 2014
dfee589
Print ERR if MIB is not available
joshmoore May 15, 2014
e18c31c
Strip uploadOnly from ManagedImportProcessI
joshmoore May 15, 2014
3f2bd44
Improve exception logging
joshmoore May 15, 2014
3673409
Minor wording improvement
joshmoore May 15, 2014
52e5d49
Fix 'compile' issue importMIF
joshmoore May 16, 2014
28680b4
Remove extra a
joshmoore May 16, 2014
6c44690
Also 'outer join' pixelsFileMaps.parent
joshmoore May 16, 2014
f59c44f
Handle null sizes for better sorting
joshmoore May 19, 2014
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -362,6 +362,7 @@

<bean class="omero.cmd.RequestObjectFactoryRegistry" lazy-init="false">
<constructor-arg ref="extendedMetadata"/>
<constructor-arg ref="aclVoter"/>
<constructor-arg ref="roles"/>
<constructor-arg ref="/OMERO/Pixels"/>
<property name="iceCommunicator" ref="Ice.Communicator"/>
Expand Down
13 changes: 8 additions & 5 deletions components/blitz/resources/omero/Repositories.ice
Original file line number Diff line number Diff line change
Expand Up @@ -432,11 +432,14 @@ module omero {
["ami"] interface ManagedRepository extends Repository {

/**
* Returns the directory which should be the import location for
* the set of paths passed in. Each set of paths consitutes a
* single import session. In order to prevent files from being
* overwritten or interfering with one another, a new directory
* may be created for the current session.
* Returns an [ImportProcess] which can be used to upload files.
* On [ImportProcess::verifyUpload], an [omero::cmd::Handle] will be
* returned which can be watched for knowing when the server-side import
* is complete.
*
* Once the upload is complete, the [ImportProcess] must be closed.
* Once [omero::cmd::Handle::getResponse] returns a non-null value, the
* handle instance can and must be closed.
**/
ImportProcess* importFileset(omero::model::Fileset fs, ImportSettings settings) throws ServerError;

Expand Down
45 changes: 45 additions & 0 deletions components/blitz/resources/omero/cmd/FS.ice
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,51 @@ module omero {
**/
omero::RTypeDict seriesMetadata;
};

/**
* Queries and modifies the various binary artifacts
* which may be linked to an [omero::model::Image].
*
* This can be useful, e.g., after converting pre-OMERO-5
* archived original files into [omero::model::Fileset].
*
* The command works in several stages:
*
* 1. loads an [omero::model::Image] by id, failing if none present.
* 2. renames Pixels file to '*_bak'
* 3. deletes existing Pyramidfiles if present;
*
* This command can be run multiple times with different settings
* to iteratively test if the migration is working.
**/
class ManageImageBinaries extends Request {

long imageId;
bool togglePixels;
bool deletePyramid;

};

/**
* [Response] from a [ManageImageBinaries] [Request].
* If no action is requested, then the fields of this
* instance can be examined to see what would be done
* if requested.
*/
class ManageImageBinariesResponse extends Response {

omero::RLong filesetId;
omero::api::LongList archivedFiles;
bool pixelsPresent;
bool pyramidPresent;
long archivedSize;
long pixelSize;
long pyramidSize;
long thumbnailSize;

};


};
};

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -136,12 +136,6 @@ void setOffset(long offset) {
/**
* Create and register a servant for servicing the import process
* within a managed repository.
*
* @param repo
* @param fs
* @param location
* @param settings
* @param __current
*/
public ManagedImportProcessI(ManagedRepositoryI repo, Fileset fs,
ImportLocation location, ImportSettings settings, Current __current)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -159,13 +159,33 @@ public Ice.Object tie() {
// INTERFACE METHODS
//

/**
* Return a template based directory path. The path will be created
* by calling {@link #makeDir(String, boolean, Ice.Current)}.
*/
public ImportProcessPrx uploadFileset(Fileset fs, ImportSettings settings,
Ice.Current __current) throws omero.ServerError {

ImportLocation location = internalImport(fs, settings, __current);
return createUploadProcess(fs, location, settings, __current, true);

}

/**
* Return a template based directory path. The path will be created
* by calling {@link #makeDir(String, boolean, Ice.Current)}.
*/
public ImportProcessPrx importFileset(Fileset fs, ImportSettings settings,
Ice.Current __current) throws omero.ServerError {

ImportLocation location = internalImport(fs, settings, __current);
return createImportProcess(fs, location, settings, __current);

}

private ImportLocation internalImport(Fileset fs, ImportSettings settings,
Ice.Current __current) throws omero.ServerError {

if (fs == null || fs.sizeOfUsedFiles() < 1) {
throw new omero.ApiUsageException(null, null, "No paths provided");
}
Expand Down Expand Up @@ -198,10 +218,9 @@ public ImportProcessPrx importFileset(Fileset fs, ImportSettings settings,

// If any two files clash in that chosen basePath directory, then
// we want to suggest a similar alternative.
ImportLocation location =
suggestImportPaths(relPath, basePath, paths, readerClass, settings.checksumAlgorithm, __current);
return suggestImportPaths(relPath, basePath, paths, readerClass,
settings.checksumAlgorithm, __current);

return createImportProcess(fs, location, settings, __current);
}

public ImportProcessPrx importPaths(List<String> paths,
Expand Down Expand Up @@ -331,6 +350,19 @@ protected ImportProcessPrx createImportProcess(Fileset fs,
job.linkOriginalFile((omero.model.OriginalFile) new IceMapper().map(of));
}

return createUploadProcess(fs, location, settings, __current, false);
}

/**
* Creating the process will register itself in an appropriate
* container (i.e. a SessionI or similar) for the current
* user and therefore this instance no longer needs to worry
* about the maintenance of the object.
*/
protected ImportProcessPrx createUploadProcess(Fileset fs,
ImportLocation location, ImportSettings settings,
Current __current, boolean uploadOnly) throws ServerError {

// Create CheckedPath objects for use by saveFileset
final int size = fs.sizeOfUsedFiles();
final List<CheckedPath> checked = new ArrayList<CheckedPath>();
Expand All @@ -343,8 +375,8 @@ protected ImportProcessPrx createImportProcess(Fileset fs,
// Since the fileset saved validly, we create a session for the user
// and return the process.

final ManagedImportProcessI proc = new ManagedImportProcessI(this, managedFs,
location, settings, __current);
final ManagedImportProcessI proc = new ManagedImportProcessI(this,
managedFs, location, settings, __current);
processes.addProcess(proc);
return proc.getProxy();
}
Expand All @@ -358,12 +390,16 @@ protected ImportProcessPrx createImportProcess(Fileset fs,
protected Class<? extends FormatReader> getReaderClass(Fileset fs, Current __current) {
for (final Job job : fs.linkedJobList()) {
if (job instanceof UploadJob) {
final FilesetVersionInfo versionInfo = ((UploadJob) job).getVersionInfo(__current);
final String readerName = versionInfo.getBioformatsReader(__current).getValue();
final Class<?> potentialReaderClass;
Class<?> potentialReaderClass;
try {
final FilesetVersionInfo versionInfo = ((UploadJob) job).getVersionInfo(__current);
final String readerName = versionInfo.getBioformatsReader(__current).getValue();
potentialReaderClass = Class.forName(readerName);
} catch (ClassNotFoundException e) {
} catch (NullPointerException npe) {
log.debug("No info provided for reader class");
continue;
} catch (Exception e) {
log.warn("Error getting reader class", e);
continue;
}
if (FormatReader.class.isAssignableFrom(potentialReaderClass)) {
Expand Down
16 changes: 15 additions & 1 deletion components/blitz/src/omero/cmd/RequestObjectFactoryRegistry.java
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@

import ome.io.nio.AbstractFileSystemService;
import ome.io.nio.PixelsService;
import ome.security.ACLVoter;
import ome.security.ChmodStrategy;
import ome.services.chgrp.ChgrpStepFactory;
import ome.services.chown.ChownStepFactory;
Expand All @@ -28,6 +29,7 @@
import omero.cmd.basic.DoAllI;
import omero.cmd.basic.ListRequestsI;
import omero.cmd.basic.TimingI;
import omero.cmd.fs.ManageImageBinariesI;
import omero.cmd.fs.OriginalMetadataRequestI;
import omero.cmd.graphs.ChgrpI;
import omero.cmd.graphs.ChmodI;
Expand All @@ -47,16 +49,21 @@ public class RequestObjectFactoryRegistry extends

private final ExtendedMetadata em;

private final ACLVoter voter;

private final Roles roles;

private final PixelsService pixelsService;

private/* final */OmeroContext ctx;

public RequestObjectFactoryRegistry(ExtendedMetadata em, Roles roles,
public RequestObjectFactoryRegistry(ExtendedMetadata em,
ACLVoter voter,
Roles roles,
PixelsService pixelsService) {

this.em = em;
this.voter = voter;
this.roles = roles;
this.pixelsService = pixelsService;

Expand Down Expand Up @@ -149,6 +156,13 @@ public Ice.Object create(String name) {
return new OriginalMetadataRequestI(pixelsService);
}
});
factories.put(ManageImageBinariesI.ice_staticId(),
new ObjectFactory(ManageImageBinariesI.ice_staticId()) {
@Override
public Ice.Object create(String name) {
return new ManageImageBinariesI(pixelsService, voter);
}
});
return factories;
}

Expand Down
Loading