New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(MacOS / hiDPI) Retina display rendering #94

Open
thomaslima opened this Issue Mar 16, 2018 · 20 comments

Comments

Projects
None yet
2 participants
@thomaslima
Collaborator

thomaslima commented Mar 16, 2018

Dear all,

I've been working on this retina display rendering feature recently. I've scratched my head quite a bit and I think I'm starting to, dare I say, deal with some bugs in Qt. Here's the Qt doc about it: http://doc.qt.io/qt-5/qpainter.html#drawing-high-resolution-versions-of-pixmaps-and-images

When you have a QImage, it suffices to say qimg.setDevicePixelRatio(2) for retina Macs, then the QPixmap created after it respects the DPR and renders correctly. But a QBitmap (which inherits QPixmap) object, such as the ruler, fails to render correctly, even though it should. I've tried many ways to make it render, and the only way i got it to work was to convert the bitmap to an image, and then back to pixmap. This part of the code would need some serious refactoring for that to work. I would need help for that.

I'm working on the macos_retina branch in my fork: https://github.com/lightwave-lab/klayout/tree/macos_retina , feel free to contribute!

Build with ./build4mac.py -p B36 -q Qt5Custom && ./build4mac.py -p B36 -q Qt5Custom -Y && qt5.pkg.macos-HighSierra-release/klayout.app/Contents/MacOS/klayout, with qt and python from brew.

I'm posting some preview pictures here, including failures in the case of 2x oversampling.

retina

no-retina

retina2x

no-retina2x

@klayoutmatthias

This comment has been minimized.

Owner

klayoutmatthias commented Mar 18, 2018

Hi Tomas,

I only have a VirtualBox, no Retina display :-(

I assume you have tried the refactoring already. QBitmap is supposed to be more efficient, but today that should not be a big issue. I understand that you're saying that using a QImage would solve the issue (if you use qimg.setDevicePixelRatio(2) ), right?

Currently the mentioned code works like this:

  • the layout is painted to a QImage and transferred to the screen after subsampling
  • the foreground objects (such as rulers, selection markers, highlights and other markers) are painted to pre-allocated bitmaps ("planes") and transferred individually to the screen as overlays. Subsampling happens on bit-by-bit basis which also explains the uglyness of the rulers in subsampling mode.

So a quick solution would be to add the bitmaps to the QImage before subsampling and only subsample the image as a whole. Should be quick, cleaner and safe for Retina.

I'll propose a patch. Give me a some time. I'll also try to get rid of the #ifdef's as different buffer schemes are no longer required. That's a legacy concept from old X11 times.

Best regards,

Matthias

@thomaslima

This comment has been minimized.

Collaborator

thomaslima commented Mar 18, 2018

hi Matthias, I spent some time dealing with this, and right now the code in my branch simply multiplies the viewport_l's dimensions by 2, which is the devicepixelratio for macs. But this variable can be captured by something like qApp.mainscreen.devicePixelRatio(), so that it would be more compatible to other platforms. I think i've put the *2 everywhere necessary for the canvas in my branch, if you can take a look. But everywhere else we use a QImage the same has to be done. We also need 2x versions of icons.

I would love to help you out but I feel like I need more insight of the code to tackle all the places you use QImage without messing up the build for other platforms.

Let me show you a few examples on how it renders on Macs (but I am sure same would happen in windows):

screen shot 2018-03-18 at 19 11 52

screen shot 2018-03-18 at 19 12 04

screen shot 2018-03-18 at 19 12 08

klayoutmatthias added a commit that referenced this issue Mar 19, 2018

A refactoring attempt for the Retina display issue (#94)
This is what's been done:
 - remove the old double and single buffering /w mask approach
 - modify the bitmap rendering so it's done in a offscreen
   image before subsampling
   (effect: rulers display smoothly in subsampling mode)
 - refactoring the "device pixel ratio" topic:
   Made the DPR a variable, viewport width is scaled up
   to reflect the true image size, inserted #ifdef's for Qt4.

DISCLAIMER: I don't know whether this still works - I don't
have a Retina display :-(
@klayoutmatthias

This comment has been minimized.

Owner

klayoutmatthias commented Mar 19, 2018

Hi Thomas,

I have just created a pull request with some refactoring attempt (lightwave-lab#1).

you did a remarkable job in identifying the root cause and actually your solution was pretty much the correct approach :-) I just had to try to turn it into a generic solution.

I could only verify it's still working on a normal display - I hope high-DPI displays still work.

Here my reasoning:

As I understand the "devicePixelRatio" story, it goes like this: high-DPI displays have much denser pixels but there is still software around which uses pixel dimensions - for example 32x32 icons (me included). These would look pretty small. So the solution is to use a virtual pixel space which is 2 times coarser than the physical one, but as the rendering engines support floating-point coordinates, you can paint on half pixels. The effect will be the same than painting on integer pixels but at twice the size.

In order to support this, we need to establish images which have a bigger dimension (initially), then set the devicePixelRatio to 2. This will make the images shrink to a virtual size which is the half. The image data (such as scanline data etc.) will still be twice. Painting on images would happen with the virtual pixel size, but KLayout does not benefit from that as it has it's own rendering engine.

So we need to do this:

  • For a high-DPI display rulers and other markers need to be drawn twice as thick normal so they don't appear too thin.
  • The width and height of the widget are virtual width/height. To obtain the true width and height we have to multiply their values with 2.
  • The QBitmap issue needs to be solved.

For the first topic, luckly KLayout has a concept similar to the devicePixelRatio: the "resolution". The idea is to define a drawing unit - a "pixel" - and indicate that the canvas has a smaller physical pixel size. The rulers and annotations - whose width is one "pixel" - get painted with n physical pixels if the resolution is a 1/n. In oversampling mode 2x, the resolution would be 0.5. With Retina we can set this to 1/(oversampling*DPR), so for example with 2x oversampling the resolution would become 0.25 and the rulers will be painted 4 physical pixels wide.

The third topic, the solution is to usw an alternative implementation: instead of painting the subsampled bitmaps for the dynamic objects to the screen as bitmaps, I collect them (full resolution) in a QImage and subsample this. This path utilizes the same subsampling code than the normal (static) display. It just needs to be extended to support alpha channel subsampling, because it's drawn over the static content and needs to be transparent.

I turns out that this actually reduces the code and as a side benefit, the rulers look smoother in subsampling mode.

Let me know if the Retina display has issues with this solution.

Best regards,

Matthias

@thomaslima

This comment has been minimized.

Collaborator

thomaslima commented Mar 19, 2018

Thank you so so much! I will try it out right now. The icons loaded from png files actually render in the say you describe already, as my screenshots suggest. The point is if we added a high resolution one, it would look prettier for those with high dpi displays.

@thomaslima

This comment has been minimized.

Collaborator

thomaslima commented Mar 19, 2018

I believe your solution worked well for the rendering, but the mouse events are now trapped on the lower-left corner of my display. That is to say: trying to draw a ruler across the canvas actually draws it half-sized on the lower left side. But the rulers are drawn in high resolution right now, so it's great progress!

screen shot 2018-03-19 at 18 19 34

@thomaslima

This comment has been minimized.

Collaborator

thomaslima commented Mar 19, 2018

The ruler labels also disappeared when I set the oversampling to 2x. I have no idea how to solve the mouse event issue. I have to wait for a hint from you.

screen shot 2018-03-19 at 18 29 54

@thomaslima

This comment has been minimized.

Collaborator

thomaslima commented Mar 19, 2018

Another hint to you: the xy coordinates on the bottom-right, as I move the mouse, are also halved compare to true size.

@thomaslima

This comment has been minimized.

Collaborator

thomaslima commented Mar 19, 2018

@klayoutmatthias I think it has to do with the m_trans variable in the ViewObjectWidget::mouse_event_trans method. Really excited to test this out.

@klayoutmatthias

This comment has been minimized.

Owner

klayoutmatthias commented Mar 19, 2018

Hi Thomas,

Oh yes ... the mouse coordinates are "virtual" too ... what a mess.

I created a pull request, but maybe you're right and it's better (and easier) to change the transformation. Please ignore the pull request if you find the transformation approach is working better.

I have no idea why the ruler labels don't show. Maybe something with the stored fonts.

Matthias

@klayoutmatthias

This comment has been minimized.

Owner

klayoutmatthias commented Mar 20, 2018

Your approach is better :-) - I have created one more pull request modifying the transformation.

@klayoutmatthias

This comment has been minimized.

Owner

klayoutmatthias commented Mar 20, 2018

I got the reason for the missing ruler labels ... it's entirely stupid. There are fonts provided for resolutions 1, 1/2 and 1/3. No more was required so far. Now we also need 1/4 and 1/6.

I'll try to create the fixed font bitmaps. Currently they are compiled in.

Sorry for this confusion.

BR, Matthias

@klayoutmatthias

This comment has been minimized.

Owner

klayoutmatthias commented Mar 20, 2018

I have upgraded the pull request. There should be fonts for all resolutions to 1/6 now.

Matthias

@thomaslima

This comment has been minimized.

Collaborator

thomaslima commented Mar 20, 2018

It works now after lightwave-lab#3. I had to fix the import headers for the new fontgen.cc but it just worked out of the box. Very nice job!

I shall change the title and keep this open for more hiDPI compatibility.

screen shot 2018-03-19 at 22 58 46

@thomaslima thomaslima changed the title from (MacOS / hiDPI) Retina display rendering of canvas to (MacOS / hiDPI) Retina display rendering Mar 20, 2018

thomaslima added a commit to lightwave-lab/klayout that referenced this issue Mar 20, 2018

A refactoring attempt for the Retina display issue (klayoutmatthias#94)
This is what's been done:
 - remove the old double and single buffering /w mask approach
 - modify the bitmap rendering so it's done in a offscreen
   image before subsampling
   (effect: rulers display smoothly in subsampling mode)
 - refactoring the "device pixel ratio" topic:
   Made the DPR a variable, viewport width is scaled up
   to reflect the true image size, inserted #ifdef's for Qt4.

DISCLAIMER: I don't know whether this still works - I don't
have a Retina display :-(
@thomaslima

This comment has been minimized.

Collaborator

thomaslima commented Mar 20, 2018

Dear all:

Recap: The retina rendering of the canvas was figured out in #101 . Next up:

  • Stiple rendering on layer toolbox
  • hi-def icons (need artist or original svg files)
  • Settings->Text-> color (also in Application-> Selection and Display->Background,Context, Appearance)
  • Settings->Color Palette
  • Display->Stipple palette
@klayoutmatthias

This comment has been minimized.

Owner

klayoutmatthias commented Mar 20, 2018

Ah yes, sorry for the fontgen.cc headers ... the fontgen.cc is just a helper tool.

Instead of compiling in the font bitmaps I should generate them on the fly. The I'm going to remove fontgen.cc.

The other QImage objects should not require refactoring. The are used for generating printouts and screenshots. So I don't think they need the DPR scheme.

I have started to use SVG for the icons, but not all of them are translated yet. I can try to do so, but it will take a while.

I'm going to merge this into master (as you did). I would like to release 0.25.2 and we can include the Retina canvas support into this.

Best regards,

Matthias

@thomaslima

This comment has been minimized.

Collaborator

thomaslima commented Mar 20, 2018

Thank you for including that in .25.2. Let me know if we can help with the svg.

About the other QImage/QPixmaps objects, I meant to say the ones that impact the issues highlighted in my comment above.

@thomaslima

This comment has been minimized.

Collaborator

thomaslima commented Mar 20, 2018

@klayoutmatthias

This comment has been minimized.

Owner

klayoutmatthias commented Jun 10, 2018

I think we finally solved this, did we?

@thomaslima

This comment has been minimized.

Collaborator

thomaslima commented Jun 15, 2018

@klayoutmatthias the layout canvas is solved, but there are other places which are not rendered properly still, outlined in my Mar 18 and Mar 20 posts. But these don't affect me much, personally.

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Jul 24, 2018

Update to 0.25.3
* Use Qt 5 instead of Qt 4

Changelog:
0.25.3          (2018-05-03):

* Enhancement: Compile option to use libcurl instead of QtNetwork
       This workaround enables builds on the native CentOS6
       configuration with the package manager functional.
* Enhancement: Pass label (text) layers through DRC script
       The "labels" method now allows copying labels from a layer
       to the output.
* Enhancement: New "split" method for polygons
       This method will fragment the polygons into two or more
       smaller parts for reducing their vertex count.
* Enhancement: New "dtrans" method for RecursiveShapeIterator
       This method delivers the transformation in micron unit space
* Enhancement: New "insert" methods for Shapes
       Shapes can now insert from another Shapes container and
       from a RecursiveShapeIterator. Possible applications are
       fast flattening and fast region selection.
* Enhancement: New method to generate RDB items from shapes
       RdbItem#add_value from a shape, RdbDatabase#create_items
       from a shapes container or recursive shape iterator and
       RdbDatabase#create_item from a shape.
* Enhancement: DXF and CIF "keep layer names"
       If this option is set in the reader options, layer names
       are not translated into GDS layer/datatype pairs.
       Specifically a layer called "L5" for example is not
       translated to 5/0 and to "L5D0" on output correspondingly.
       The buddy script option is "--keep-layer-names".
* Enhancement: DXF contour joining accuracy
       On the DXF reader's options, a contour joining accuracy
       can be specified. Small gaps smaller than this accuracy
       will be closed when forming joined contours from edges.
       The buddy script option is --dxf-contour-accuracy=value.
* Bugfix: DXF display issue
       In some cases, DXF cell instances did not trigger bounding
       box update and the cells only got visible after save and
       load.
* Bugfix: XOR progress is more realistic
       The progress is updated after the layer has been computed,
       not before.
* Bugfix: klayoutmatthias/klayout#117
       DTrans#itype was broken.
* Bugfix: klayoutmatthias/klayout#116
       Fixed a polygon decomposition bug when writing GDS files
       with big polygons with many holes. As a side effect, the
       polygons with many holes computed by a NOT operation for
       example are less complex and spikes in the cutlines are
       avoided.
* Bugfix: klayoutmatthias/klayout#115
       Reader options were not persisted.
* Bugfix: klayoutmatthias/klayout#114
       Custom line styles not loaded from tech's layer properties
       file.
* Enhancement: klayoutmatthias/klayout#113
       The XOR tool has an option now to heal result shapes which
       cross tile boundaries. The result shape count of tiled
       and non-tiled mode should basically be the same then.
* Bugfix: klayoutmatthias/klayout#112
       Salt package repository relative paths have not been working.
* Bugfix: klayoutmatthias/klayout#109
       Issues with Python 3 and shape properties - property
* Bugfix: klayoutmatthias/klayout#108
       Bugfix on Box#enlarge and Box#moved for empty boxes.
       keys generated with Python 3 could not be written to GDS2.
* Bugfix: klayoutmatthias/klayout#107
       Undo not working with shapes.
* Enhancement: klayoutmatthias/klayout#106
       Search & replace help page enhancements.

0.25.2          (2018-03-20):
* Bugfix: klayoutmatthias/klayout#90
      DRC: "extended" was not working as expected with "joined = true"
* Bugfix: klayoutmatthias/klayout#89
      Display issue on MacOS fixed
* Enhancement: klayoutmatthias/klayout#85
      IDE debugger: files can be excluded from showing exceptions when
      they are thrown. To exclude a file press the new "Ignore" button
      when the debugger tells you an exception has been generated.
      To re-able exception reporting, clear the list of the files
      in the IDE settings ("Debugging" tab)
      The macro IDE settings can now be edited in the File/Setup
      dialog.
* Bugfix: klayoutmatthias/klayout#94
      Retina displays are support to some extend on MacOS.
      An open topic is the quality of the icons.
* Enhancement: build system for MacOS

0.25.1          (2018-02-23):
* Enhancements: build compatibility with MacOS and Qt 5.9.
      Qt 4.6 supported now as well with one restriction:
      the package installation feature is not working.
* Bugfixs: Package manager
       - Remove button wasn't enabled if multiple packages were selected
       - A potential crash ob removing packages was fixed
* Enhancement: 64 bit coordinate support enabled on Windows builds
* Further bugfixes: See links
       - klayoutmatthias/klayout#21 (Autorun(-early) doesn't seem to run when lym files are inside a package)
       - klayoutmatthias/klayout#24 (Text insert dialog bug - Ok button isn't working)
       - klayoutmatthias/klayout#26 (Exceptions are reported every time they propagate up in the call chain in the ruby debugger)
       - klayoutmatthias/klayout#28 (CIF format detection failed)
       - klayoutmatthias/klayout#30 (Writer options dialog non-functional on a fresh configuration)
       - klayoutmatthias/klayout#32 (Rounding issue with instance properties)
       - klayoutmatthias/klayout#33 (Plugin factory not working when using with Python)
       - klayoutmatthias/klayout#36 (Hardening against destruction of object inside event handler)
       - klayoutmatthias/klayout#39 (Action cannot be reassigned)
       - klayoutmatthias/klayout#40 (Crash in Python binding)
       - klayoutmatthias/klayout#41 (Polygon#touches? issue)
       - klayoutmatthias/klayout#42 (Headless mode support with Qt5/-zz)
       - klayoutmatthias/klayout#43 (Crash when using Qt specific command line options)
       - klayoutmatthias/klayout#44 (Transformation constructor with x,y not working)
       - klayoutmatthias/klayout#45 (Partial selection does not capture instance)
       - klayoutmatthias/klayout#48 (Cancel does not reset current tool)
       - klayoutmatthias/klayout#51 (Segmentation fault on return to main window and other opportunities)
       - klayoutmatthias/klayout#53 (Unreadable 'about' text)
       - klayoutmatthias/klayout#62 (QXmlSimpleReader#parse cannot be called)
       - klayoutmatthias/klayout#63 (Wrong output on DRC non_interacting with empty second input)
       - klayoutmatthias/klayout#64 (Crash on exit)
       - klayoutmatthias/klayout#68 (OASIS reader issue with degenerated shapes)
       - klayoutmatthias/klayout#69 (DRC: 'inside' does not merge shapes of second input)
       - klayoutmatthias/klayout#71 (Target cell argument is required)
       - klayoutmatthias/klayout#72 (Edges/Region NOT issue)
       - klayoutmatthias/klayout#73 (Allow 'change layers' on PCells which support a single layer parameter)
       - klayoutmatthias/klayout#74 (Small-corner boolean issue)
       - klayoutmatthias/klayout#75 (Python PCell issue when parameters are called 'layer')
       - klayoutmatthias/klayout#79 (Replace function enabled also for read-only macros)
* Further enhancements: see links
       - klayoutmatthias/klayout#29 (Permissive mode for OASIS writer on odd-width paths)
       - klayoutmatthias/klayout#59 (Async download of package index and details)
       - klayoutmatthias/klayout#66 (Authentication dialog indicates retry)
       - klayoutmatthias/klayout#77 (Layout#copy_tree now works in non-editable mode too)

0.25            (2017-11-04):
* Enhancement: Menu customization
      Menu items can be disabled or enabled now. The former
      "Key Bindings" configuration page has been turned into
      "Menu Customization". Check boxes allow enabling or
      disabling of menu items. Disabled menu items are not
      shown.
      A search filter allows quick selection of menu items
      by title or path.
* Enhancement: The cell tree no longer is collapsed on edits
      or when the sorting order changes.
* Enhancement: Using a special menu path, macros can be
      configured to appear in submenus now.
* Enhancement: KLAYOUT_HOME environment variable to
      control the location of the application folder.
      The application folder is the place where KLayout
      keeps the configuration, macros, packages and
      other things.
* Enhancements: Major overhaul of the build environment
      - Qt5 supported now
      - Employs qmake as build system
      - Reorganisation of the sources
      - Continuous integration enabled (unit tests
        integrated in .pro files, JUnit output etc.)
      - Compatible with the clang compiler
* Enhancements: Buddy tools: KLayout now comes with a set of
      additional executables that provide some functionality
      subset in a lean way. These binaries are named "strm..."
      and are installed along with KLayout.
      - strm2cif: converts any format to CIF
      - strm2dxf: converts any format to DXF
      - strm2gds: converts any format to GDS2
      - strm2gdstxt: converts any format to GDS2/text
      - strm2oas: converts any format to OASIS
      - strm2txt: converts any format to plain text (internally)
      - strmclip: creates a clip from a file
      - strmcmp: compares two files (logically)
      - strmxor: compares two files (geometrically)
      For more information, use the "-h" or "--help" option.
* Enhancements: HTTPS support for loading files from URL's directly
* Enhancements: Numerous enhancements and optimizations in the RBA/pya
      area and in the debugger.
* Enhancements: KLayout notifies the user when a file was changed and
      asks whether to reload.
* Enhancements: "File cell or layer": type into the cell or layer list to jump to the
      cell name typed.
* Enhancements: Advanced search features in the filter entry box
      for the cell selection form: Case sensitive/insensitive,
      glob pattern, clear button.
* Enhancements: Tree expansion icons have a better contrast now on Windows
      with dark background.
* Enhancements: Forward and backward navigation buttons in the main window.
      These buttons jump to the previous or next display state:
      zoom window, hierarchy depth, etc..
* Enhancements: A Package manager is provided. Packages are add-ons for
      KLayout and can include macros, script libraries, fonts
      for the TEXT PCell, technologies, DRC scripts etc.
      Packages can be published on GitHub or any server
      supporting WebDAV. After registering a package, users
      can install or update packages with a few clicks.
      For more information see https://github.com/klayoutmatthias/klayout/wiki
      or "About Packages" in the "Various Topics" area of the main
      documentation. The package manager is found in the "Tools"
      menu under "Manage Packages".
* Enhancements: LEF/DEF reader:
       - Compatibility fixes with version 5.8
       - Regions and Groups are supported now
       - LEF/DEF is a standare format now and
         can be used with "File/Open" for example.
* Enhancement: DXR reader supports variable path widths now
* Enhancement: XOR provides graphical progress
* Enhancements: Libraries can be associated with a technology. Only libraries
      associated with the selected technology are shown in the selection
      dialog.
* Enhancements: Overlapping instances are easier to select now (instance
      selection cycling).
* Enhancements: Many more methods in RBA/pya. For example:
       - Ability to directly insert "D" types (such as DBox
         into layouts and cells.
       - RBA::RecursiveShapeIterator now supports complex (non-rectangular)
         search regions
       - A more consistent type framework (consistent complex transformation
         types, vectors)
       - More intuitive interface in certain objects (i.e. CellView,
         LayerPropertiesNode, Annotation etc.). Changing an attribute
         usually has an immediate effect on the respective property now
         ("live objects")
       - Hash functions for certain objects - Box, Edge, Trans, Polygon,
         and other objects can be used as dictionary keys now.
       - New import methods for Region, EdgePairs and Edges
       - New features for LayerMapping and CellMapping
       - Multiple selected cells in LayoutView
       - RBA::TextGenerator to supply the Basic.TEXT PCell's
         functionality as static feature
       - Receivers can be added to signals and removed from
         signals now.
       - Individual instance iterator for CellInstArray
       - ...
* Enhancements: Enhanced layout statistics form with detailed shape
      statistics.
* Enhancements: Line styles (dashed, dotted etc.) available now.
      Works best with manhattan layouts.
* Enhancements: "Marked box mode": boxes are drawn with a cross.
* Enhancements: Arbitrary pattern size for custom fill pattern (no
      limited to multiples of 8). Max size is 32x32.
      Enhanced annotations:
       - Ellipses
       - Position markers
       - Text alignment
       - "Auto measure" rulers (place a ruler measuring the
         distances between adjacent objects with a single
         click)
* Enhancements: DRC enhancements:
       - Text support
       - "smoothed" function
       - "layers" function delivers all layers read
       - Polygon with edge interactions
       - middle and extent_refs methods for center point and other
         references
       - "corners" function
       - "collect...", "select" and "each" methods
* Enhancements: GDS2 paths can now use the MultiXY extension too.
* Enhancements: X2 Support for Gerber import.
* Enhancements: A function to reset the window state after the
      window has been misconfigured (i.e. detaching or closing
      the dockable windows). This feature is found in
      View/Restore Window.
@thomaslima

This comment has been minimized.

Collaborator

thomaslima commented Aug 3, 2018

Made some progress on this. See PR #150

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment