Add reader for PicoQuant .bin files. (rebased onto dev_5_0) #1287

sbesson · 2014-08-23T12:48:26Z

This is the same as gh-1245 but rebased onto dev_5_0.

This adds support for one of the formats covered by https://trac.openmicroscopy.org.uk/ome/ticket/8059#
NB I am led to believe that the other format is a FIFO rather than imaging format in which case it could prove 'challenging'.

…loaded to obviate future memory issues. NB loading is 4-5 times slower.

sbesson · 2014-08-23T12:48:27Z

--rebased-from #1245

bpindelski · 2014-08-25T13:04:27Z

Tested by importing to octopus. From the from_skyking/picoquant folder all .bin (2) and image files (.png, .bmp) are imported - the rest is rejected (.pdf, .log etc.). In Insight, the .bin files have a decent amount of time points, but are completely black - is that expected? If so, then this looks OK to merge.

imunro · 2014-08-25T18:38:52Z

The first time-points are typically dark (before the laser pulse that excites the fluorescence) & insight displays the fist time-point by default so it's not unexpected. If all the time-points are black however that would be a problem.

bpindelski · 2014-08-26T08:23:03Z

@imunro Encouraged by your comment, I wanted to automatically scan each time-point by using the "Play" button in the image viewer. Insight sadly gave me

java.lang.Exception: Abnormal termination due to an uncaught exception.
java.lang.IllegalArgumentException: (minimum <= value <= maximum) is false
    at javax.swing.SpinnerNumberModel.<init>(SpinnerNumberModel.java:108)
    at javax.swing.SpinnerNumberModel.<init>(SpinnerNumberModel.java:130)
    at org.openmicroscopy.shoola.agents.imviewer.util.player.MoviePlayerUI.initComponents(MoviePlayerUI.java:194)
    at org.openmicroscopy.shoola.agents.imviewer.util.player.MoviePlayerUI.<init>(MoviePlayerUI.java:432)
    at org.openmicroscopy.shoola.agents.imviewer.util.player.MoviePlayerDialog.<init>(MoviePlayerDialog.java:126)
    at org.openmicroscopy.shoola.agents.imviewer.view.ImViewerControl.getMoviePlayer(ImViewerControl.java:753)
    at org.openmicroscopy.shoola.agents.imviewer.view.ImViewerComponent.playMovie(ImViewerComponent.java:1915)
    at org.openmicroscopy.shoola.agents.imviewer.actions.PlayMovieAction.actionPerformed(PlayMovieAction.java:182)
    at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2028)
    at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2351)
    at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:387)
    at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:242)
    at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:236)
    at java.awt.AWTEventMulticaster.mouseReleased(AWTEventMulticaster.java:272)
    at java.awt.Component.processMouseEvent(Component.java:6414)
    at javax.swing.JComponent.processMouseEvent(JComponent.java:3275)
    at java.awt.Component.processEvent(Component.java:6179)
    at java.awt.Container.processEvent(Container.java:2084)
    at java.awt.Component.dispatchEventImpl(Component.java:4776)
    at java.awt.Container.dispatchEventImpl(Container.java:2142)
    at java.awt.Component.dispatchEvent(Component.java:4604)
    at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4618)
    at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4279)
    at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4209)
    at java.awt.Container.dispatchEventImpl(Container.java:2128)
    at java.awt.Window.dispatchEventImpl(Window.java:2492)
    at java.awt.Component.dispatchEvent(Component.java:4604)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:717)
    at java.awt.EventQueue.access$400(EventQueue.java:82)
    at java.awt.EventQueue$2.run(EventQueue.java:676)
    at java.awt.EventQueue$2.run(EventQueue.java:674)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:86)
    at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:690)
    at java.awt.EventQueue$3.run(EventQueue.java:688)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:86)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:687)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:296)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:211)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:201)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:196)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:188)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)
Abnormal termination due to an uncaught exception.
java.lang.IllegalArgumentException: (minimum <= value <= maximum) is false
    at javax.swing.SpinnerNumberModel.<init>(SpinnerNumberModel.java:108)
    at javax.swing.SpinnerNumberModel.<init>(SpinnerNumberModel.java:130)
    at org.openmicroscopy.shoola.agents.imviewer.util.player.MoviePlayerUI.initComponents(MoviePlayerUI.java:194)
    at org.openmicroscopy.shoola.agents.imviewer.util.player.MoviePlayerUI.<init>(MoviePlayerUI.java:432)
    at org.openmicroscopy.shoola.agents.imviewer.util.player.MoviePlayerDialog.<init>(MoviePlayerDialog.java:126)
    at org.openmicroscopy.shoola.agents.imviewer.view.ImViewerControl.getMoviePlayer(ImViewerControl.java:753)
    at org.openmicroscopy.shoola.agents.imviewer.view.ImViewerComponent.playMovie(ImViewerComponent.java:1915)
    at org.openmicroscopy.shoola.agents.imviewer.actions.PlayMovieAction.actionPerformed(PlayMovieAction.java:182)
    at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2028)
    at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2351)
    at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:387)
    at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:242)
    at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:236)
    at java.awt.AWTEventMulticaster.mouseReleased(AWTEventMulticaster.java:272)
    at java.awt.Component.processMouseEvent(Component.java:6414)
    at javax.swing.JComponent.processMouseEvent(JComponent.java:3275)
    at java.awt.Component.processEvent(Component.java:6179)
    at java.awt.Container.processEvent(Container.java:2084)
    at java.awt.Component.dispatchEventImpl(Component.java:4776)
    at java.awt.Container.dispatchEventImpl(Container.java:2142)
    at java.awt.Component.dispatchEvent(Component.java:4604)
    at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4618)
    at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4279)
    at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4209)
    at java.awt.Container.dispatchEventImpl(Container.java:2128)
    at java.awt.Window.dispatchEventImpl(Window.java:2492)
    at java.awt.Component.dispatchEvent(Component.java:4604)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:717)
    at java.awt.EventQueue.access$400(EventQueue.java:82)
    at java.awt.EventQueue$2.run(EventQueue.java:676)
    at java.awt.EventQueue$2.run(EventQueue.java:674)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:86)
    at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:690)
    at java.awt.EventQueue$3.run(EventQueue.java:688)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:86)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:687)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:296)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:211)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:201)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:196)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:188)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)
Exception in thread "AWT-EventQueue-0"

    at org.openmicroscopy.shoola.env.ui.UserNotifierImpl.showErrorDialog(UserNotifierImpl.java:191)
    at org.openmicroscopy.shoola.env.ui.UserNotifierImpl.notifyError(UserNotifierImpl.java:291)
    at org.openmicroscopy.shoola.env.AbnormalExitHandler.doTermination(AbnormalExitHandler.java:147)
    at org.openmicroscopy.shoola.env.AbnormalExitHandler.terminate(AbnormalExitHandler.java:85)
    at org.openmicroscopy.shoola.env.AWTExceptionHanlder.handle(AWTExceptionHanlder.java:99)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at java.awt.EventDispatchThread.handleException(EventDispatchThread.java:412)
    at java.awt.EventDispatchThread.processException(EventDispatchThread.java:318)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:312)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:211)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:201)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:196)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:188)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)

/cc @jburel

bpindelski · 2014-08-26T08:26:21Z

@imunro Manually scanning through the first 10% of time-points of both the sample images does indeed reveal their content, so the reader looks OK. The Insight fatal crash is a bit of a blocker though.

imunro · 2014-08-26T08:28:54Z

Have you seen that with any other file types? Can't immediately see how how a reader could cause that.
Only thing I can think is that my reader could potentially use a lot of memory could that affect insight ? @jburel

jburel · 2014-08-26T08:38:45Z

Which user did you log in as? so I can have a look

bpindelski · 2014-08-26T08:46:53Z

@jburel I've used user-1. There is a dataset called picoquant. Tried re-importing the images - still same error. Surprisingly, it doesn't happen with develop.

jburel · 2014-08-26T08:51:30Z

@bpindelski: I fixed an issue with movie player on develop so that will explain

imunro · 2014-08-26T08:53:14Z

Replicated the issue using DaisyPollen.bin from that dataset. Only happens on playing the data as a movie. Stepping through is fine.

bpindelski · 2014-08-26T08:59:18Z

@imunro, @jburel It seems that this crash is solely Insight-related (as J-M mentioned, it's fixed on develop, so that fix will need backporting). This PR is OK to merge then.

jburel · 2014-08-26T10:54:41Z

see ome/openmicroscopy#2908 for fixes in movie player

jburel · 2014-08-26T15:08:56Z

Testing using a local client build including fixes to movie-player (see ome/openmicroscopy#2908) against octopus. The problem does not occur.
It is a bit slow to play a movie across small t but it works.

Add reader for PicoQuant .bin files. (rebased onto dev_5_0)

sbesson · 2014-08-27T05:17:42Z

Looking at snoopycrimecop@80cbe7e. Should we have a set of proper documentation pages for the PicoQuant format (would require to edit components/autogen/src/format-pages.txt)

imunro · 2014-08-27T09:10:23Z

I'll try & have a look as soon as I can.

hflynn · 2014-08-27T15:01:36Z

Documentation would be handy @imunro - this won't show up on the supported formats table if it isn't documented.

imunro added 14 commits August 23, 2014 13:46

Add PQBinReader for PicoQuant .bin files

9c0d7a1

Modfiy readers.txt to include PQBinReader

c934e03

Correct HEADER_SIZE and add isThisType check.

49e0df1

Cosmetic changes as requested by rleigh.

ed9a21f

Correct Reader in comment.

e557df8

Remove unnecessary declaration of rowBuf.

80dd319

Remove extra setting of m.littleendian.

658f018

Remove extra call to in.order.

f5fb115

Store pixelResolution as PixelsPhysicalSize

42747bd

Untested changes to limit memory useage.

6ba1c1a

Add a size threshold to PQBinReader beyond which the data is not pre-…

aa8bd51

…loaded to obviate future memory issues. NB loading is 4-5 times slower.

Add a similar size limit to SDTreader to obviate memory issues.

b078430

Set preLoad on closing.

d7d7076

Lower threshold to test both code paths.

935e763

sbesson force-pushed the rebased/dev_5_0/develop-PQBin branch from 0000000 to 935e763 Compare August 23, 2014 12:48

sbesson mentioned this pull request Aug 23, 2014

Add reader for PicoQuant .bin files. #1245

Merged

sbesson added the dev_5_0 label Aug 25, 2014

melissalinkert added a commit that referenced this pull request Aug 26, 2014

Merge pull request #1287 from sbesson/rebased/dev_5_0/develop-PQBin

5782d8c

Add reader for PicoQuant .bin files. (rebased onto dev_5_0)

melissalinkert merged commit 5782d8c into ome:dev_5_0 Aug 26, 2014

sbesson deleted the rebased/dev_5_0/develop-PQBin branch August 27, 2014 05:13

sbesson mentioned this pull request Aug 27, 2014

Update release notes for 5.0.4 #1281

Merged

This was referenced Aug 28, 2014

Add documentation for PicoQuant .bin format #1294

Merged

Add documentation for PicoQuant .bin format (rebased onto develop) #1297

Merged

sbesson added this to the 5.0.4 milestone Sep 1, 2014

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add reader for PicoQuant .bin files. (rebased onto dev_5_0) #1287

Add reader for PicoQuant .bin files. (rebased onto dev_5_0) #1287

sbesson commented Aug 23, 2014

sbesson commented Aug 23, 2014

bpindelski commented Aug 25, 2014

imunro commented Aug 25, 2014

bpindelski commented Aug 26, 2014

bpindelski commented Aug 26, 2014

imunro commented Aug 26, 2014

jburel commented Aug 26, 2014

bpindelski commented Aug 26, 2014

jburel commented Aug 26, 2014

imunro commented Aug 26, 2014

bpindelski commented Aug 26, 2014

jburel commented Aug 26, 2014

jburel commented Aug 26, 2014

sbesson commented Aug 27, 2014

imunro commented Aug 27, 2014

hflynn commented Aug 27, 2014

Add reader for PicoQuant .bin files. (rebased onto dev_5_0) #1287

Add reader for PicoQuant .bin files. (rebased onto dev_5_0) #1287

Conversation

sbesson commented Aug 23, 2014

sbesson commented Aug 23, 2014

bpindelski commented Aug 25, 2014

imunro commented Aug 25, 2014

bpindelski commented Aug 26, 2014

bpindelski commented Aug 26, 2014

imunro commented Aug 26, 2014

jburel commented Aug 26, 2014

bpindelski commented Aug 26, 2014

jburel commented Aug 26, 2014

imunro commented Aug 26, 2014

bpindelski commented Aug 26, 2014

jburel commented Aug 26, 2014

jburel commented Aug 26, 2014

sbesson commented Aug 27, 2014

imunro commented Aug 27, 2014

hflynn commented Aug 27, 2014