Performance Regression over X11 #60

jimhester · 2013-07-18T17:53:29Z

There is a major performance regression in IGV that has existed since version 2.2 was released, which makes newer versions unbearably slow when using X11. I assume it is still the same issue as https://groups.google.com/forum/#!topic/igv-help/jR1Z8ITFcXM

I have tried to disable anti-aliasing to fix the problem as mentioned in that email chain, but the preference does not seem to be checked everywhere, in particular the read display themselves seem to still be antialiased. Performance is as you would expect very bad due to this problem.

2.1.24 performance is very snappy over X11, so I will continue to use that, but it would be nice to fix this performance regression so that those of us using IGV over the network can use the new features!

jrobinso · 2013-07-18T18:04:35Z

Is performance good when not viewing reads (other track types)? I have always found X11 performance unbearable with IGV, its good to know it worked for some people on some versions. I use and generally recommend a vnc server for running remotely.

I will track down anti-aliasing and be sure its turned off, and post a comment here when an update is ready to test.

jimhester · 2013-07-19T12:56:02Z

Performance is pretty bad regardless of what you are doing unfortunately. I profiled both versions with hprof, see results at https://gist.github.com/jimhester/6038886.

However found a user side fix at http://www.oracle.com/technetwork/java/javase/java2d-142140.html#gcgiv which does work, disabling the use of pixmaps by the X11 pipeline with -Dsun.java2d.pmoffscreen=false does fix the performance problems. So even if you don't try to fix the poor AA performance you might want to put a note about that option somewhere!

Thanks for developing IGV, it is a very useful tool for our lab!

yarden · 2013-07-30T12:12:40Z

I am having the same issue over X11. IGV works but within a minute or two of clicking around, it hangs over X11 and yields this error:

ERROR [2013-07-30 08:09:37,635]  [DefaultExceptionHandler.java:42] [AWT-EventQueue-2]  Unhandled exception
java.lang.NullPointerException
    at javax.swing.SwingUtilities.getWindowAncestor(SwingUtilities.java:137)
    at javax.swing.SwingUtilities.windowForComponent(SwingUtilities.java:455)
    at javax.swing.ToolTipManager.showTipWindow(ToolTipManager.java:309)
    at javax.swing.ToolTipManager$insideTimerAction.actionPerformed(ToolTipManager.java:629)
    at javax.swing.Timer.fireActionPerformed(Timer.java:313)
    at javax.swing.Timer$DoPostEvent.run(Timer.java:244)
    at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:226)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:673)
    at java.awt.EventQueue.access$300(EventQueue.java:96)
    at java.awt.EventQueue$2.run(EventQueue.java:634)
    at java.awt.EventQueue$2.run(EventQueue.java:632)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:108)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:643)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:275)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:200)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:190)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:185)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:177)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:138)

Is this an unrelated issue, or does it also have something to do with X11?

The fix suggested for performance of adding Dsun.java2d.pmoffscreen=false does make a huge difference, but for me it leads to other issues. For example, clicking on menus sometimes only opens them for a split second and then they close, so you can click on "File" and then by the time you move your cursor down to select something from the menu, the menu disappears...

jrobinso · 2013-07-30T15:10:49Z

I'll have to investigate. In the meantime, could you try turning
tooltips off (use the ballon icon in the toolbar) and see if that
resolves the issue? Let me know, that would be good information.

I am having the same issue over X11. IGV works but within a minute or
two of clicking around, it hangs over X11 and yields this error:

|ERROR [2013-07-30 08:09:37,635] [DefaultExceptionHandler.java:42] [AWT-EventQueue-2] Unhandled exception
java.lang.NullPointerException
at javax.swing.SwingUtilities.getWindowAncestor(SwingUtilities.java:137)
at javax.swing.SwingUtilities.windowForComponent(SwingUtilities.java:455)
at javax.swing.ToolTipManager.showTipWindow(ToolTipManager.java:309)
at javax.swing.ToolTipManager$insideTimerAction.actionPerformed(ToolTipManager.java:629)
at javax.swing.Timer.fireActionPerformed(Timer.java:313)
at javax.swing.Timer$DoPostEvent.run(Timer.java:244)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:226)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:673)
at java.awt.EventQueue.access$300(EventQueue.java:96)
at java.awt.EventQueue$2.run(EventQueue.java:634)
at java.awt.EventQueue$2.run(EventQueue.java:632)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:108)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:643)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:275)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:200)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:190)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:185)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:177)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:138)
|

Is this an unrelated issue, or does it also have something to do with
X11? Thanks.

—
Reply to this email directly or view it on GitHub
#60 (comment).

yarden · 2013-08-03T15:03:03Z

What is the ballon icon? Is it the yellow button? If so, should I choose Never show details from it?

I also tried upgrading to make sure I'm using latest version (or snapshot), and whenever I download zips from the IGV website and try to use them, I get errors like these:

$ unzip IGV_2.3.13.zip 
Archive:  IGV_2.3.13.zip
checkdir error:  IGV_2.3.13.zip exists but is not directory
                 unable to process IGV_2.3.13.zip/.
checkdir error:  IGV_2.3.13.zip exists but is not directory
                 unable to process IGV_2.3.13.zip/igv.bat.
checkdir error:  IGV_2.3.13.zip exists but is not directory
                 unable to process IGV_2.3.13.zip/igv.command.
checkdir error:  IGV_2.3.13.zip exists but is not directory
                 unable to process IGV_2.3.13.zip/igv.sh.
checkdir error:  IGV_2.3.13.zip exists but is not directory
                 unable to process IGV_2.3.13.zip/batik-codec.jar.
checkdir error:  IGV_2.3.13.zip exists but is not directory
                 unable to process IGV_2.3.13.zip/goby-io-igv.jar.
checkdir error:  IGV_2.3.13.zip exists but is not directory
                 unable to process IGV_2.3.13.zip/igv.jar.
checkdir error:  IGV_2.3.13.zip exists but is not directory
                 unable to process IGV_2.3.13.zip/readme.txt.

Is there something off with the zip files? That's normally how I unzip files on Linux/Mac OS X.

jrobinso · 2013-08-04T01:03:17Z

Yarden,

Apologies, I get this error too. Something gone wrong in the build
process. I'm investigating.

Jim

I'll try tooltips off but I first tried upgrading to make sure I'm
using latest version (or snapshot), and whenever I download zips from
the IGV website and try to use them, I get errors like these:

|$ unzip IGV_2.3.13.zip
Archive: IGV_2.3.13.zip
checkdir error: IGV_2.3.13.zip exists but is not directory
unable to process IGV_2.3.13.zip/.
checkdir error: IGV_2.3.13.zip exists but is not directory
unable to process IGV_2.3.13.zip/igv.bat.
checkdir error: IGV_2.3.13.zip exists but is not directory
unable to process IGV_2.3.13.zip/igv.command.
checkdir error: IGV_2.3.13.zip exists but is not directory
unable to process IGV_2.3.13.zip/igv.sh.
checkdir error: IGV_2.3.13.zip exists but is not directory
unable to process IGV_2.3.13.zip/batik-codec.jar.
checkdir error: IGV_2.3.13.zip exists but is not directory
unable to process IGV_2.3.13.zip/goby-io-igv.jar.
checkdir error: IGV_2.3.13.zip exists but is not directory
unable to process IGV_2.3.13.zip/igv.jar.
checkdir error: IGV_2.3.13.zip exists but is not directory
unable to process IGV_2.3.13.zip/readme.txt.
|

Is there something off with the zip files? That's normally how I unzip
files on Linux/Mac OS X.

—
Reply to this email directly or view it on GitHub
#60 (comment).

jrobinso · 2013-08-04T01:18:10Z

I fixed the problem with the zip file, you can download it again.
Apologies.

|checkdir error: IGV_2.3.13.zip exists but is not directory
unable to process IGV_2.3.13.zip/.
checkdir error: IGV_2.3.13.zip exists but is not directory
unable to process IGV_2.3.13.zip/igv.bat.
checkdir error: IGV_2.3.13.zip exists but is not directory|

yarden · 2013-08-04T19:33:36Z

Thanks for looking into it so quickly Jim! The IGV_snapshot.zip still gives me this:

$ unzip IGV_snapshot.zip 
Archive:  IGV_snapshot.zip
checkdir error:  IGV_snapshot.zip exists but is not directory
                 unable to process IGV_snapshot.zip/.
checkdir error:  IGV_snapshot.zip exists but is not directory
                 unable to process IGV_snapshot.zip/igv.bat.
checkdir error:  IGV_snapshot.zip exists but is not directory
                 unable to process IGV_snapshot.zip/igv.command.
checkdir error:  IGV_snapshot.zip exists but is not directory
                 unable to process IGV_snapshot.zip/igv.sh.
checkdir error:  IGV_snapshot.zip exists but is not directory
                 unable to process IGV_snapshot.zip/batik-codec.jar.
checkdir error:  IGV_snapshot.zip exists but is not directory
                 unable to process IGV_snapshot.zip/goby-io-igv.jar.
checkdir error:  IGV_snapshot.zip exists but is not directory
                 unable to process IGV_snapshot.zip/igv.jar.
checkdir error:  IGV_snapshot.zip exists but is not directory
                 unable to process IGV_snapshot.zip/readme.txt.

yarden · 2013-08-04T19:39:59Z

When I change igv.sh to this in IGV_2.3.13:

exec java -Xmx2000m \
        -Dproduction=true \
        -Dapple.laf.useScreenMenuBar=true \
        -Dsun.java2d.pmoffscreen=false -Xmx2g -DenableSashimi="true" \
        -Djava.net.preferIPv4Stack=true \
        -jar "$prefix"/igv.jar "$@" &

It's still quite slow on X11, probably a bit slower than the old release I was using. (btw is the -DenableSashimi="true" still needed or is it on by default?)

Thanks, --Yarden

jrobinso · 2013-08-04T21:42:01Z

Oh sorry, I fixed the release. Get that one, they are actually in sync
at the moment.

Thanks for looking into it so quickly Jim! The IGV_snapshot.zip still
gives me this:

|$ unzip IGV_snapshot.zip
Archive: IGV_snapshot.zip
checkdir error: IGV_snapshot.zip exists but is not directory
unable to process IGV_snapshot.zip/.
checkdir error: IGV_snapshot.zip exists but is not directory
unable to process IGV_snapshot.zip/igv.bat.
checkdir error: IGV_snapshot.zip exists but is not directory
unable to process IGV_snapshot.zip/igv.command.
checkdir error: IGV_snapshot.zip exists but is not directory
unable to process IGV_snapshot.zip/igv.sh.
checkdir error: IGV_snapshot.zip exists but is not directory
unable to process IGV_snapshot.zip/batik-codec.jar.
checkdir error: IGV_snapshot.zip exists but is not directory
unable to process IGV_snapshot.zip/goby-io-igv.jar.
checkdir error: IGV_snapshot.zip exists but is not directory
unable to process IGV_snapshot.zip/igv.jar.
checkdir error: IGV_snapshot.zip exists but is not directory
unable to process IGV_snapshot.zip/readme.txt.
|

—
Reply to this email directly or view it on GitHub
#60 (comment).

jrobinso · 2013-08-04T21:44:16Z

BTW, the workaround for this problem until it is fixed is to rename the
zip file, example to tmp.zip, then unzip it.

Thanks for looking into it so quickly Jim! The IGV_snapshot.zip still
gives me this:

|$ unzip IGV_snapshot.zip
Archive: IGV_snapshot.zip
checkdir error: IGV_snapshot.zip exists but is not directory
unable to process IGV_snapshot.zip/.
checkdir error: IGV_snapshot.zip exists but is not directory
unable to process IGV_snapshot.zip/igv.bat.
checkdir error: IGV_snapshot.zip exists but is not directory
unable to process IGV_snapshot.zip/igv.command.
checkdir error: IGV_snapshot.zip exists but is not directory
unable to process IGV_snapshot.zip/igv.sh.
checkdir error: IGV_snapshot.zip exists but is not directory
unable to process IGV_snapshot.zip/batik-codec.jar.
checkdir error: IGV_snapshot.zip exists but is not directory
unable to process IGV_snapshot.zip/goby-io-igv.jar.
checkdir error: IGV_snapshot.zip exists but is not directory
unable to process IGV_snapshot.zip/igv.jar.
checkdir error: IGV_snapshot.zip exists but is not directory
unable to process IGV_snapshot.zip/readme.txt.
|

—
Reply to this email directly or view it on GitHub
#60 (comment).

yarden · 2014-02-11T16:46:34Z

Hi all,

Is there a definitive fix to the slowness over X11? I'm trying to update the Whitehead version of IGV so that it runs faster. Thanks! Best, Yarden

jrobinso · 2014-02-11T16:51:25Z

Hi,

No definite fix, we haven't been able to reproduce or diagnose the
problem. IGV over remote X11 has never been recommended, is that how
it is run @ the whitehead?

Jim

Hi all,

Is there a definitive fix to the slowness over X11? I'm trying to
update the Whitehead version of IGV so that it runs faster. Thanks!
Best, Yarden

—
Reply to this email directly or view it on GitHub
#60 (comment).

yarden · 2014-02-11T16:57:35Z

Hi Jim,

That's definitely how most people run it, I believe, and at least me. I ssh into the tak server (Whitehead server) and then run it. All my BAM files are hosted there and I can't think of a more convenient solution. I've tried using Chicken and various VPN clients for Mac OS X to directly login to the Whitehead Linux servers, but those clients are very clunky and have all sort of irritating bugs (e.g. virtually impossible to copy and paste from the VPN client to the machine, etc.)

Any thoughts on this? Thanks, Yarden

jrobinso · 2014-02-11T17:13:09Z

I'm surprised the whitehead doesn't have a a means to mount linux drives
on a Mac and a vpn solution.

We don't have resources to do X11 debugging right now, if you can
identify the source of the slowness we are happy to make changes you can
test. Have you tried disabling anti-aliasing in the "advanced"
preference tab? The problem with X11 is there is lots of traffic
between client and server for mouse movement and graphics. If we can
identify a particular track that is the culprit we could try things like
limiting the color palette used for that track.

Jim

Hi Jim,

That's definitely how most people run it, I believe, and at least me.
I ssh into the tak server (Whitehead server) and then run it. All my
BAM files are hosted there and I can't think of a more convenient
solution. I've tried using Chicken and various VPN clients for Mac OS
X to directly login to the Whitehead Linux servers, but those clients
are very clunky and have all sort of irritating bugs (e.g. virtually
impossible to copy and paste from the VPN client to the machine, etc.)

Any thoughts on this? Thanks, Yarden

—
Reply to this email directly or view it on GitHub
#60 (comment).

yarden · 2014-02-11T23:26:03Z

I don't think this is at all a limitation on Whitehead end. The limitation I faced is that all the VPN clients for Mac OS X just don't work very well; they have bugs and don't seem to allow simple copy and pasting between the remote environments. For example, I tried JollyFastVNC. Maybe I am overlooking some better clients, so if you have suggestions, I'd love to know.

Whitehead does allow you to remotely mount the Linux filesystem. Are you suggesting that the best approach is to install IGV locally (e.g. on my Mac OS X) and then simply load the BAMs via remote mounting of the Whitehead servers? Perhaps that's the best solution?

Thanks, Yarden

trutane · 2014-08-02T02:05:02Z

I can confirm the performance issue using recent versions of IGV over X11 (v2.3.31 and v2.3.34) which was not a problem as of v2.1.2. Using VNC with the recent IGV versions, the performance issue goes away.

To give a sense of the magnitude of the performance hit, I have a batch script that loads a BAM containing WGS data, goes to a ~20kb region, sorts, collapses, and takes a snapshot. For IGV v2.3.x, the process took ~24 minutes over X11 (wall clock) but only ~30 seconds over VNC. Using IGV v2.1.2, the performance was not impacted when running over X11.

Perusing the IGV git logs between v2.1.x and v2.2.x may provide some useful clues as to what change (feature addition or bug fix) caused this regression.

jrobinso · 2014-08-02T02:07:58Z

Did you try disabling anti-aliasing in the advanced preferences tab.

On Friday, August 1, 2014, Steve Chervitz Trutane notifications@github.com
wrote:

I can confirm the performance issue using recent versions of IGV over X11
(v2.3.31 and v2.3.34) which was not a problem as of v2.1.2. Using VNC with
the recent IGV versions, the performance issue goes away.

To give a sense of the magnitude of the performance hit, I have a batch
script that loads a BAM containing WGS data, goes to a ~20kb region, sorts,
collapses, and takes a snapshot. For IGV v2.3.x, the process took ~24
minutes over X11 (wall clock) but only ~30 seconds over VNC. Using IGV
v2.1.2, the performance was not impacted when running over X11.

Perusing the IGV git logs between v2.1.x and v2.2.x may provide some
useful clues as to what change (feature addition or bug fix) caused this
regression.

—
Reply to this email directly or view it on GitHub
#60 (comment).

trutane · 2014-08-04T22:51:06Z

The advanced preference tab has a "Enable antialiasing" checkbox that is unchecked by default. I verified it was unchecked before testing (IGV version 2.3.34).

The slowest step appears to be the snapshot command. You can seen the image being redrawn on screen one row at a time - slow if you have a deep stack of reads and multiple tracks.

Is there a way to run IGV without any graphics output (e.g. when you just want to generate snapshots)?

jrobinso · 2014-08-05T00:48:33Z

IGV needs a display, there is a Java library that basically let you run
headless by emulating X11 graphics. I don't recall the name of it, if
you do a web search it should be easy to find.

The advanced preference tab has a "Enable antialiasing" checkbox that
is unchecked by default. I verified it was unchecked before testing
(IGV version 2.3.34).

The slowest step appears to be the snapshot command. You can seen the
image being redrawn on screen one row at a time - slow if you have a
deep stack of reads and multiple tracks.

Is there a way to run IGV without any graphics output (e.g. when you
just want to generate snapshots)?

—
Reply to this email directly or view it on GitHub
#60 (comment).

gatoravi · 2014-08-05T17:23:15Z

Just for reference - I think the java library Jim mentioned above is 'xvfb' (http://en.wikipedia.org/wiki/Xvfb), the way to use the library would be

xvfb-run java -jar /pathto/igv.jar -b batch_script.bat

trutane · 2014-08-11T20:27:05Z

Thanks for the xvfb tip. I tested it out and it does indeed work, even with anti-aliasing enabled.

However, the generated snapshots are of MUCH lower quality than when the same script is processed directly via IGV (feeding it in via the GUI Tools menu). The snapshots are about 5x smaller in size and it looks like some commands weren't processed at all (e.g. maxPanelHeight).

Perhaps this is a limitation of xvfb. Maybe Xdummy would be worth a try? http://xpra.org/trac/wiki/Xdummy

jrobinso · 2014-08-11T20:33:20Z

Hi, sorry I don't have time to dig into this myself, but you might try
exporting .svg files rather than .png. Just use an .svg extension
comment here if that helps, or not.

Thanks for the xvfb tip. I tested it out and it does indeed work, even
with anti-aliasing enabled.

However, the generated snapshots are of MUCH lower quality than when
the same script is processed directly via IGV (feeding it in via the
GUI Tools menu). The snapshots are about 5x smaller in size and it
looks like some commands weren't processed at all (e.g. maxPanelHeight).

Perhaps this is a limitation of xvfb. Maybe Xdummy would be worth a
try? http://xpra.org/trac/wiki/Xdummy

—
Reply to this email directly or view it on GitHub
#60 (comment).

trutane · 2014-08-12T01:33:53Z

The .svg images generated by IGV running under xvfb are improved over the .png, but they're still not great. Images are less grainy and axis labels are much better in the SVG, but the overall image size is still small and the maxPanelHeight command is not handled (complete read stack is not visible in the image when it should be).

jrobinso · 2015-03-21T03:13:48Z

Closing X11 issues, given the very limited resources available for IGV development (i.e. me) I just can't pursue this. I would welcome pull requests on the issue. Perhaps Java 1.8 will have made some improvements in this regard.

jrobinso closed this as completed Mar 21, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Performance Regression over X11 #60

Performance Regression over X11 #60

jimhester commented Jul 18, 2013

jrobinso commented Jul 18, 2013

jimhester commented Jul 19, 2013

yarden commented Jul 30, 2013

jrobinso commented Jul 30, 2013

yarden commented Aug 3, 2013

jrobinso commented Aug 4, 2013

jrobinso commented Aug 4, 2013

yarden commented Aug 4, 2013

yarden commented Aug 4, 2013

jrobinso commented Aug 4, 2013

jrobinso commented Aug 4, 2013

yarden commented Feb 11, 2014

jrobinso commented Feb 11, 2014

yarden commented Feb 11, 2014

jrobinso commented Feb 11, 2014

yarden commented Feb 11, 2014

trutane commented Aug 2, 2014

jrobinso commented Aug 2, 2014

trutane commented Aug 4, 2014

jrobinso commented Aug 5, 2014

gatoravi commented Aug 5, 2014

trutane commented Aug 11, 2014

jrobinso commented Aug 11, 2014

trutane commented Aug 12, 2014

jrobinso commented Mar 21, 2015

Performance Regression over X11 #60

Performance Regression over X11 #60

Comments

jimhester commented Jul 18, 2013

jrobinso commented Jul 18, 2013

jimhester commented Jul 19, 2013

yarden commented Jul 30, 2013

jrobinso commented Jul 30, 2013

yarden commented Aug 3, 2013

jrobinso commented Aug 4, 2013

jrobinso commented Aug 4, 2013

yarden commented Aug 4, 2013

yarden commented Aug 4, 2013

jrobinso commented Aug 4, 2013

jrobinso commented Aug 4, 2013

yarden commented Feb 11, 2014

jrobinso commented Feb 11, 2014

yarden commented Feb 11, 2014

jrobinso commented Feb 11, 2014

yarden commented Feb 11, 2014

trutane commented Aug 2, 2014

jrobinso commented Aug 2, 2014

trutane commented Aug 4, 2014

jrobinso commented Aug 5, 2014

gatoravi commented Aug 5, 2014

trutane commented Aug 11, 2014

jrobinso commented Aug 11, 2014

trutane commented Aug 12, 2014

jrobinso commented Mar 21, 2015