Issues related to using ImagePrintFigureOperation

[Phillipus]

I've opened this discussion as one central place to discuss issues around using ImagePrintFigureOperation as some problems are related to SWT and also because the Pull Requests here are getting cluttered with side issues (brought up by me) and this information is scattered in Issues and PRs.

Just a reminder that the reason that ImagePrintFigureOperation was created was to solve the problem of clipped text when creating an Image from a Figure.

For reference:

#1001
#1148

[Phillipus]

A problem I've discovered relates to the parent Shell used in the GraphicalViewer that is created to get the printable layer figure:

Shell shell = new Shell();
GraphicalViewerImpl viewer = new GraphicalViewerImpl();
viewer.createControl(shell);
// (Set EditPartFactory and Root EditPart here)
LayerManager layerManager = LayerManager.Helper.find(viewer);
IFigure figure = layerManager.getLayer(LayerConstants.PRINTABLE_LAYERS);
ImagePrintFigureOperation op = new ImagePrintFigureOperation(shell, figure);
Image image = op.run();

I've been disposing of that Shell each time after creating the Image. But in practice when the Shell is disposed so is the GraphicalViewerImpl and all of its EditParts are de-activated and, in our usage, child Figures are disposed. So when a new zoom is set later and the Figure is re-drawn in ImagePrintFigureOperation.createImageAtScale() there are problems because the child Figures have also been disposed (some objects in our Figures are released).

This means that the Shell can't be disposed as long as the Image exists and further Image.getImageData() calls are made. Creating many Images in this way would require many long lived Shells. I tried using one static Shell for all image creations but the heap memory is not released because I think there are still object references in the Shell.

If the Image is short-lived (for example saving it to file) this is not a problem as the Shell can be disposed after saving the Image, but if the Image is long-lived, as we do in Archi when creating hundreds of Gallery thumbnails of diagrams, this is a problem.

[ptziegler]

Yes, because the print operation is using an ImageDataProvider, which causes the image data is created on-demand. This means that at least right now, the figures must be in a valid state for as the image exist and any related resources mustn't be disposed.

The reason I chose the ImageDataProvider is because I need to call image.getImageData(100), to "force" the GC to paint the image as if at 100% zoom. If I weren't using this provider and if I were to create and paint the image at e.g. 125% zoom, this would result in an empty image, as SWT would look up the data for 125% and not 100%, which is the only one that exists.

To solve both the memory and the lifecycle problem, I think the most practical solution is to find a way to get the image creation without provider working again, so that it doesn't produce clipping and can be painted at different zoom levels. You then obviously get blurry images if the zoom level changes, but that's also how it used when you were creating the image without this operation.

[Phillipus]

Thanks for your insights.

I think the most practical solution is to find a way to get the image creation without provider working again, so that it doesn't produce clipping and can be painted at different zoom levels.

How could that be done? I tried a few things but keep coming back to your solution with the provider.

The actual problem behind all of this is text clipping in Figures in Images only on Windows. If the text clipping could be solved that would be great.

To be clear, in Archi we have the following use cases:

1. Export a diagram as an Image to file (or copy to clipboard) at the current display scale. This is is the easiest case as it is short lived and the parent Shell of the GraphicalViewerImpl can be disposed immediately and it only needs one image.getImageData(zoom) call (actually two with the 100% zoom).
2. The Nebula Gallery of thumbnails of diagrams generated via a GraphicalViewerImpl. This is the long lived case where if an ImageDataProvider is used the parent Shell needs to persist. But for this case I'll probably use the old method of creating an Image once with no ImageDataProvider because the images are small thumbnails so it doesn't matter if some text is clipped.

[ptziegler]

Take the following example:

package gef;

import org.eclipse.swt.graphics.GC;
import org.eclipse.swt.graphics.Image;
import org.eclipse.swt.widgets.Display;

public class FontHeights {

	public static void main(String[] args) {
		Display display = Display.getDefault();

		System.out.println("100%: " + getFontHeight(display, 100));
		System.out.println("125%: " + getFontHeight(display, 125));
		System.out.println("150%: " + getFontHeight(display, 150));
		System.out.println("175%: " + getFontHeight(display, 175));
		System.out.println("200%: " + getFontHeight(display, 200));
		System.out.println("400%: " + getFontHeight(display, 400));
	}

	private static int getFontHeight(Display display, int zoom) {
		Image image = new Image(display, 100, 100);
		GC gc = new GC(image);
		try {
			image.getImageData(zoom);
			return gc.getFontMetrics().getHeight();
		} finally {
			gc.dispose();
			image.dispose();
		}
	}
}

... which on my system, produces the following results:

100%: 15
125%: 16
150%: 17
175%: 17
200%: 16
400%: 16

These font metrics are used to determine e.g. the size of a figure. So at 125% zoom, a Label is more than just 25% larger because it also has to account for the larger font height. The problem is that an image doesn't have a concept of "monitor zoom" (because it can be painted anywhere), but by default, uses the current monitor zoom. The GEF viewer will always use 100% zoom.

So the image and the viewer operate at different zoom levels, which internally work with different font heights. The result is the clipping you observe. Hence why the current solution forces both to work at 100% zoom. I don't think this a miscalculation. It's simply two objects operating in different scopes that somehow have to be brought together.

[Phillipus]

Thanks for the explanation, that makes things clearer.

To solve both the memory and the lifecycle problem, I think the most practical solution is to find a way to get the image creation without provider working again, so that it doesn't produce clipping and can be painted at different zoom levels.

Is that a possibility?

[ptziegler]

Technically, yes. The Image class has a private Image(display, imageData, zoom) constructor. If I copy over the image data and pass the current monitor zoom, the new image is painted correctly. The question is whether there's a clean way to achieve this.

[ptziegler]

I eventually came up with this mess for the ImagePrintFigureOperation, which works at least for the example:

private Image createAutoScaledImage() {
	double scale = InternalDraw2dUtils.calculateScale(control);
	Image imageZoom = createImageAtScale(scale);

	int monitorZoom = (int) (scale * 100);
	if (monitorZoom == 100) {
		return imageZoom;
	}
	// Image is stored at 100% zoom, but is scaled by the monitor zoom
	ImageData imageDataZoom = imageZoom.getImageData(100);
	Image tmp = new Image(display, imageDataZoom);

	int width100 = imageDataZoom.width * 100 / monitorZoom;
	int height100 = imageDataZoom.height * 100 / monitorZoom;
	Image image = new Image(display, width100, height100);

	GC gc = new GC(image);
	image.getImageData(monitorZoom);
	gc.drawImage(tmp, 0, 0, width100, height100);
	gc.dispose();

	tmp.dispose();
	return image;
}

As said before, the problem is that the image data for this image is created at 100% and not the current monitor zoom. Given an image of size (20, 80) at 125% zoom, for example. There I need to construct the image at 100% zoom with size (16, 64). Next I need to paint the pre-scaled image onto this new image but as if at 125% zoom. If I didn't make any mistake, this should then "convert" the image data from 100% to 125% zoom.

[Phillipus]

Thanks for coming up with this, I tested it. It just needs a imageDataZoom.dispose() in there.

I tested at 175% and 200% and as far as I can tell:

• There is no text clipping
• Memory footprint is low and much the same as using the traditional image method

[Phillipus]

I've experimented more with using different variations of ImagePrintFigureOperation and whereas it does fix text clipping the quality of the image lines is not so good. I think that scaling up and down causes this.

This is a figure exported not using ImagePrintFigureOperation and zoomed in x4 for the screenshot so we can see the lines:

And this is the same using ImagePrintFigureOperation: