Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fslite stage2 #102

Closed
wants to merge 52 commits into from
Closed

Fslite stage2 #102

wants to merge 52 commits into from

Conversation

ximenesuk
Copy link
Contributor

There's still plenty to do on this but it would be good to get it into some build and working with Insight.

A handful of file formats including SPW, big image and multi-image formats import okay in local testing of cli and dropbox import. Areas that definitely need attention (beyond integration with GUI importers):

  • thumbnails in multi-image formats (last in series is used by default?)
  • refactoring of repository services
  • complete overhaul of import tests
  • no performance testing has been done
  • ...and lots more!

Colin Blackburn and others added 30 commits February 22, 2012 11:17
@ximenesuk
Interface changes, implementation and tests. See ome#909
ec8d689
@ximenesuk
Added makeDir method to repository to enable testing of import strategy.
ca3dfd5
@ximenesuk
Adding fileById(id) to Repository interface to catch existing use on …
7973cd0 
…file(id) in RawFileStoreI
@ximenesuk
Fully implemented root() to return a loaded OriginalFile object
bbaa4aa
@chris-allan @ximenesuk
Archive using the legacy repository service.
2bc50ac
@chris-allan @ximenesuk
Handle endianness correctly in Bio-Formats pixel buffers. (See ome#6124)
e1fa84f
@chris-allan @ximenesuk
Switch server side infrastructure to use repository archived files.
3a038b1
@ximenesuk
Added a test for RepoRawFileStore close(). close() fails but it does …
c685a65 
…successfully close the file before failing.
@ximenesuk
More useful path for archive location, for now using AbstractFileSyst…
5a6a6b9 
…emService and hard-wired template.
@ximenesuk
Single id path for all files in a set, still a flat structure.
1992ba4
@ximenesuk
Initial crude attempt at nested structure.
5d19929
@ximenesuk
Removed erroneous stuff added for testing.
1c068a3
@chris-allan @ximenesuk
Modifying import to use target file and prefix.
0681ba0
@chris-allan @ximenesuk
Ensure that the proper paths are used.
feb6bb6
@chris-allan @ximenesuk
Ensure that targets of any MIME type are used.
1e1840a
@chris-allan @ximenesuk
Enabling FS lite across the board.
839ed79
@ximenesuk
Added import path template to properties, see ome#6223
96da236
@ximenesuk
Move template-based directory look-up to the repository.
860e3bf
@ximenesuk
Fixing injection of template to legacy repo only.
fbf92b5
@chris-allan @ximenesuk
Allow the pixels path to be retrieved.
c25aa06
@ximenesuk
Initial work on returning unique path name. See #7691
6022bf3
@ximenesuk
getCurrentRepoDir() now takes parent directory as string.
a359dd1
@ximenesuk
Initial changes to get minimal paths within the ManagedRepository.
d0803c3
@ximenesuk
Pass in z option through cli to try alternative workflow
3a0f5c3
@ximenesuk
Initial restructuring to give both import lines
ba064ae
@ximenesuk
Standard import vaguely working including archiving
7000ca2
@ximenesuk
Enable fslite importing
949b19d
@ximenesuk
Pass in y option through cli to try add dropbox to workflow
789d871
@ximenesuk
importMetadata added to repo.
165dfcb 
writeBlock also added to support testing though might be useful in later workflow.
@ximenesuk
Removing default import workflow including archiving
0d78fe4
@ximenesuk
Further work on removing default import workflow including archiving
2306a52
@ximenesuk
Rationalise boolean flags.
afa5239 
Remove fslite, dropbox and import options -y and -z used for testing.
Use isMetadataOnly and add import option -m for metadata-only (DropBox).
@ximenesuk
Fixing bug in checkBounds when dealing with muli-image format with di…
84e24cd 
…fferent ZCT sizes
@ximenesuk
Fix to allow tests to build.
af2e0d1
@joshmoore
Copy link
Member

@jburel , you ok to add this to all future merge-blues? Are you planning any for the moment, or with multi-group merged has that slowed down?

@jburel
Copy link
Member

jburel commented Feb 29, 2012

@joshmoore: I am planning to start working on that tomorrow.

@ximenesuk
Copy link
Contributor Author

The following tickets referring to specific file formats have been covered under the more general fs-lite work:
→ Task #7227: FS-lite: to support Zeiss LSM
→ Task #7232: FS-lite: to support PE
→ Task #7228: FS-lite: to support Olympus
→ Task #7229: FS-lite: to support Olympus
→ Task #7234: FS-lite: to support Gatan
→ Task #7230: FS-lite: to support AP
The formats have been tested.

jburel and others added 13 commits March 1, 2012 13:54
@jburel
Remove archived option from display (see #8179)
c7160a4
@jburel
Review import sequence and pass usedFiles (see #8181)
6008467
@jburel
Remove thumbnail from view (see #8186)
8bf7a35 
Do not load thumbnail and turn off the view option.
@ximenesuk
Handle null usedFiles. See #8180
ac560f8
@ximenesuk
File upload bracketted by import events. See #8184
3da64ca
@ximenesuk
Initial changes to allow for server-side import. See #8209
b7a84c8
@ximenesuk
importMetadata now takes something like an ImportContainer. See #8209…
ef4eeb3 
… and #8223
@ximenesuk
Removed unused method and altered logging. See #8209
7c4a313
@ximenesuk
RepositoryImportContainer almost fully populated. See #8209 and #8223
3884f68
@ximenesuk
FILE_UPLOAD events added. See #8184
908b1c9
@ximenesuk
Add some event output to command line immport
ec1a97b 
This could be expanded or reduced to debug-only as needed.
@ximenesuk
Refactor call to repo import and change cli logging.
6f349a3 
The changes to the cli logging revert many of the changes in the previous commit as I had completely overlooked the LoggingImportMonitor.
@ximenesuk
Simple file delete added with tests for that and other relevant repo …
19c936e 
…methods See #8241.
@ximenesuk
Copy link
Contributor Author

The delete functionality added in 19c936e is fairly simple and designed to delete a file set immediately following an upload before any changes have been made to the db. However, this still raises some questions:

  • should files actually be deleted or moved to some hidden trash area? (Maybe an import/upload can be resumed later?)
  • should deletes be honoured if there is some db reference to a file?

@ximenesuk
Copy link
Contributor Author

Closing this PR as the work really needs to be rebased on develop as due to the chgrp work there are too many conflicts with Insight code.

@ximenesuk ximenesuk closed this Mar 22, 2012
chris-allan added a commit to chris-allan/openmicroscopy that referenced this pull request Sep 9, 2015
@chris-allan
Merge pull request ome#102 from emilroz/matlab_multigroup
e6aef0e 
Matlab multigroup
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@ximenesuk @joshmoore @jburel @chris-allan