Scalable UI textures #2302

akortunov · 2019-04-02T14:48:02Z

Currently most of our UI textures can have only fixed size because of MyGUI design.
As a result, such textures look bad if player uses UI scaling. Also Morrowind often does not use textures directly - it uses textured meshes instead. For example, it makes impossible to use HD crosshair replacers, which work in Morrowind.

The main idea here is to implement an autosized texture skin widget, which will use the whole texture indifferently from texture resolution.
What I managed to achive so far:

Scalable crosshair.
Scalable texture buttions in the main menu and book windows.
Scalable skins for most of UI.
Scalable background textures for item widgets (we already scale items and spells icons themselves)
Scalable bookmark menu in the journal.
~~Scalable cursors.~~ Removed since may cause driver crashes on some setups.

Unfortunately, I do not know how to handle compound widgets, which do not use the whole texture (e.g. door markers, scrollbars).

akortunov · 2019-04-04T08:04:52Z

An example of upscaled main menu (2x resolution, 2x UI scaling factor, Pelagiad):

psi29a · 2019-04-04T19:06:41Z

Does this have any performance impact?

akortunov · 2019-04-04T19:13:43Z

Does this have any performance impact?

In theory, it should be negligable since we usually scale things only once during game start.

apps/openmw/mwgui/mainmenu.cpp

akortunov · 2019-04-05T06:30:24Z

Basically, should be finished now, but there are some minor issues left:

~~I still have no idea why I can not resize a texture from BSA file. @AnyOldName3, any ideas?~~
It is unclear if cursors should be affected by UI scaling or they should not.

akortunov · 2019-04-05T08:47:14Z

And a separate question - what to do with scrollbars? Currently they use completely unrelated textures (e.g. bookmark one). As a result, they look alien even in clean game, and with UI replacers situation is even worse.

So I suggest to provide custom textures for scrollbars arrows and track button. In this case:

We can make scrollbars scalable too.
They will look more consistent with other UI in clean game.
Texture-only UI replacers can be updated to work with OpenMW properly.

A new/current design comparison:

akortunov · 2019-04-05T10:35:02Z

OK, I managed to solve the issue with cursor scaling, but this part still will need a review from AnyOldName3.

AnyOldName3 · 2019-04-05T13:37:43Z

I'm not seeing anything immediately obviously stupid, but I don't really know anything about how MyGUI works and very little with SDL.

What I'm getting from looking at this is that MyGUI is being told how many pixels to make UI elements and how many pixels of each texture to put into those UI elements and we're now telling it to not assume the latter is a fixed amount. I'd have thought that there'd also be a way to specify which parts of a texture to use using floating point coordinates in the range 0 to 1 (seeing as that's how textures are actually sampled), and then we wouldn't have to deal with lots of what this is doing, as something that uses the top left quarter of a texture is going to use the top left quarter of the texture no matter how big it is.

akortunov · 2019-04-05T13:53:01Z

What I'm getting from looking at this is that MyGUI is being told how many pixels to make UI elements and how many pixels of each texture to put into those UI elements and we're now telling it to not assume the latter is a fixed amount.

A sort of it - we autocalculate skin size based on actual target texture resolution instead of taking it from layout files. As a result, if a widget which uses it has a different size, it will rescale texture to fit its size.
In the upsteram master we take only given part of texture (32x32 for crosshair, for example), which leads with odd bugs.

I'd have thought that there'd also be a way to specify which parts of a texture to use using floating point coordinates in the range 0 to 1 (seeing as that's how textures are actually sampled)

I tried to find something useful, but did not find anything. Without the offset attribute MyGUI assumes that the skin has the 0x0 size.
As I understood, in the MyGUI textures are supposed to have the fixed size, and a common use case implies many skins in the one large texture file.

akortunov · 2019-04-06T05:20:21Z

Well, just for reference - I managed to add the Resources folder to the VFS and feed it to MyGUI, so we can provide overridable textures for scrollbars, but I suppose it is for another PR.

akortunov · 2019-04-06T06:18:47Z

All right, it is a time for a proper testing and review.

elsid · 2019-04-07T15:16:29Z

Did some testing. When [GUI] scaling factor is 2.75 and more, game start leads to graphics driver crash and complete graphical and keyboard OS freeze (sound is still playing). The problem reproduces only on 32d2189. It is somehow related to this and this lines (depends if I use OPENMW_DECOMPRESS_TEXTURES env). If I comment those, there is no problem except cursor is not scaled.

I have Arch Linux with kernel version 5.0.6-arch1-1-ARCH and nvidia driver version 418.56-6. journalctl may have one of these errors:

Apr 07 16:44:11 es-nbd kernel: NVRM: GPU at PCI:0000:01:00: GPU-6a7b4faf-be8f-4ef1-e554-67a5601b86f0
Apr 07 16:44:11 es-nbd kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: Class 0x0 Subchannel 0x0 Mismatch
Apr 07 16:44:11 es-nbd kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x4041b0=0x200000
Apr 07 16:44:11 es-nbd kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x404000=0x80000002
Apr 07 16:44:11 es-nbd kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ChID 0018, Class 0000902d, Offset 00000860, Data 00000000
Apr 07 16:44:11 es-nbd kernel: NVRM: Xid (PCI:0000:01:00): 32, Channel ID 00000018 intr 02000000
Apr 07 16:44:11 es-nbd kernel: NVRM: Xid (PCI:0000:01:00): 31, Ch 00000019, intr 10000000. MMU Fault: ENGINE GRAPHICS GPCCLIENT_PROP_0 faulted @ 0x1_00430000. Fault is of type FAULT_PTE ACCESS_TYPE_WRITE

Apr 07 16:49:44 es-nbd kernel: NVRM: GPU at PCI:0000:01:00: GPU-6a7b4faf-be8f-4ef1-e554-67a5601b86f0
Apr 07 16:49:44 es-nbd kernel: NVRM: Xid (PCI:0000:01:00): 12, COCOD 00000018 80019700 0000c197 00000830 60403825
Apr 07 16:49:44 es-nbd kernel: NVRM: Xid (PCI:0000:01:00): 31, Ch 00000019, intr 30000000. MMU Fault: ENGINE GRAPHICS GPCCLIENT_PROP_0 faulted @ 0x1_00430000. Fault is of type FAULT_PTE ACCESS_TYPE_WRITE

Apr 07 16:57:13 es-nbd kernel: Asynchronous wait on fence NVIDIA:nvidia.prime:275c timed out (hint:intel_atomic_commit_ready+0x0/0x58 [i915])

Probably it's just my specific setup problem. But usually I play with scaling factor = 3.0. Anyone else can reproduce?

Capostrophic · 2019-04-07T15:26:59Z

Can't reproduce.

psi29a · 2019-04-08T08:07:03Z

@elsid you using a hybrid graphics setup that makes use of nvidia:prime implementation?

elsid · 2019-04-08T15:47:56Z

@elsid you using a hybrid graphics setup that makes use of nvidia:prime implementation?

yes

akortunov · 2019-04-08T18:47:34Z

Too bad. Are there alternative ways to rescale crosshair textures properly?

akortunov · 2019-04-09T06:41:39Z

@elsid, can you check if the hardware decompression works with this function?

    SDLUtil::SurfaceUniquePtr hardwareDecompress (osg::ref_ptr<osg::Image> source, float rotDegrees, int cursorWidth, int cursorHeight)
    {
        MyGraphicsContext context(cursorWidth, cursorHeight);

        osg::ref_ptr<osg::State> state = context.getContext()->getState();

        osg::ref_ptr<osg::Texture2D> texture = new osg::Texture2D;
        texture->setImage(source);
        texture->setWrap(osg::Texture::WRAP_S, osg::Texture::CLAMP_TO_BORDER);
        texture->setWrap(osg::Texture::WRAP_T, osg::Texture::CLAMP_TO_BORDER);
        texture->setBorderColor(osg::Vec4f(0.f, 0.f, 0.f, 0.f));

        osg::ref_ptr<osg::TexMat> texmat = new osg::TexMat;
        osg::Matrix texRot (osg::Matrix::identity());
        float theta ( osg::DegreesToRadians(-rotDegrees) );
        float cosTheta = std::cos(theta);
        float sinTheta = std::sin(theta);

        texRot(0,0) = cosTheta;
        texRot(1,0) = -sinTheta;
        texRot(0,1) = sinTheta;
        texRot(1,1) = cosTheta;
        // Offset center of rotation to center of texture
        texRot(3,0) = 0.5f + ( (-0.5f * cosTheta) - (-0.5f * sinTheta) );
        texRot(3,1) = 0.5f + ( (-0.5f * sinTheta) + (-0.5f * cosTheta) );

        texmat->setMatrix(texRot);

        state->applyTextureAttribute(0, texmat);

        osg::ref_ptr<osg::RefMatrix> identity (new osg::RefMatrix(osg::Matrix::identity()));
        state->applyModelViewMatrix(identity);
        state->applyProjectionMatrix(identity);

        state->applyMode(GL_TEXTURE_2D, true);
        state->applyTextureAttribute(0, texture);

        osg::ref_ptr<osg::Image> resultImage = new osg::Image;
        resultImage->allocateImage(cursorWidth, cursorHeight, 1, GL_RGBA, GL_UNSIGNED_BYTE);

        osg::RenderInfo renderInfo;
        renderInfo.setState(state);

        glViewport(0, 0, cursorWidth, cursorHeight);

        osg::ref_ptr<osg::Geometry> geom;

        geom = osg::createTexturedQuadGeometry(osg::Vec3(-1,-1,0), osg::Vec3(2,0,0), osg::Vec3(0,2,0));

        geom->drawImplementation(renderInfo);

        glReadPixels(0, 0, cursorWidth, cursorHeight, GL_RGBA, GL_UNSIGNED_BYTE, resultImage->data());

        geom->releaseGLObjects();
        source->releaseGLObjects();
        texture->releaseGLObjects();

        return SDLUtil::imageToSurface(resultImage, true);
    }

elsid · 2019-04-09T20:32:48Z

Still have freeze, but it happens only when I open inventory.
first launch:

Apr 09 21:56:26 es-nbd /usr/lib/gdm-x-session[1593]: (EE) NVIDIA(0): The NVIDIA X driver has encountered an error; attempting to
Apr 09 21:56:26 es-nbd /usr/lib/gdm-x-session[1593]: (EE) NVIDIA(0):     recover...
Apr 09 21:56:26 es-nbd kernel: NVRM: GPU at PCI:0000:01:00: GPU-6a7b4faf-be8f-4ef1-e554-67a5601b86f0
Apr 09 21:56:26 es-nbd kernel: NVRM: Xid (PCI:0000:01:00): 69, Class Error: ChId 0018, Class 0000c197, Offset 00002388, Data 20040004, ErrorCode 00000004
Apr 09 21:56:26 es-nbd /usr/lib/gdm-x-session[1593]: (II) NVIDIA(0): Error recovery was successful.
Apr 09 21:56:30 es-nbd /usr/lib/gdm-x-session[1593]: (EE) NVIDIA(0): The NVIDIA X driver has encountered an error; attempting to
Apr 09 21:56:30 es-nbd /usr/lib/gdm-x-session[1593]: (EE) NVIDIA(0):     recover...
Apr 09 21:56:30 es-nbd kernel: NVRM: Xid (PCI:0000:01:00): 32, Channel ID 0000001b intr 00800000
Apr 09 21:56:30 es-nbd kernel: NVRM: Xid (PCI:0000:01:00): 32, Channel ID 0000001b intr 00800000
Apr 09 21:56:34 es-nbd /usr/lib/gdm-x-session[1593]: (II) NVIDIA(0): Error recovery was successful.
Apr 09 21:56:34 es-nbd /usr/lib/gdm-x-session[1593]: (II) event20 - COMPANY USB Device: SYN_DROPPED event - some input events have been lost.
Apr 09 21:56:34 es-nbd /usr/lib/gdm-x-session[1593]: (EE) client bug: timer event20 debounce: offset negative (-2004ms)
Apr 09 21:56:34 es-nbd /usr/lib/gdm-x-session[1593]: (EE) client bug: timer event20 debounce short: offset negative (-2017ms)
Apr 09 21:56:34 es-nbd /usr/lib/gdm-x-session[1593]: (EE) NVIDIA(0): The NVIDIA X driver has encountered an error; attempting to
Apr 09 21:56:34 es-nbd /usr/lib/gdm-x-session[1593]: (EE) NVIDIA(0):     recover...
Apr 09 21:56:34 es-nbd kernel: NVRM: Xid (PCI:0000:01:00): 69, Class Error: ChId 001b, Class 0000c197, Offset 0000238c, Data 20040004, ErrorCode 0000000c
Apr 09 21:56:34 es-nbd /usr/lib/gdm-x-session[1593]: (II) NVIDIA(0): Error recovery was successful.

second launch:

Apr 09 21:56:50 es-nbd /usr/lib/gdm-x-session[1593]: (EE) NVIDIA(0): The NVIDIA X driver has encountered an error; attempting to
Apr 09 21:56:50 es-nbd /usr/lib/gdm-x-session[1593]: (EE) NVIDIA(0):     recover...
Apr 09 21:56:50 es-nbd kernel: NVRM: Xid (PCI:0000:01:00): 69, Class Error: ChId 001b, Class 0000c197, Offset 00001614, Data 00000000, ErrorCode 0000000d
Apr 09 21:56:50 es-nbd /usr/lib/gdm-x-session[1593]: (II) NVIDIA(0): Error recovery was successful.
Apr 09 21:56:53 es-nbd /usr/lib/gdm-x-session[1593]: (EE) NVIDIA(0): The NVIDIA X driver has encountered an error; attempting to
Apr 09 21:56:53 es-nbd /usr/lib/gdm-x-session[1593]: (EE) NVIDIA(0):     recover...
Apr 09 21:56:53 es-nbd kernel: NVRM: Xid (PCI:0000:01:00): 9, Channel 0000001b Instance 00000000 Intr 00000010
Apr 09 21:56:57 es-nbd kernel: NVRM: Xid (PCI:0000:01:00): 12, Ch 0000001b Cl 0000a140 Off 00002420 Data 0b800000
Apr 09 21:56:57 es-nbd /usr/lib/gdm-x-session[1593]: (II) NVIDIA(0): Error recovery was successful.
Apr 09 21:56:57 es-nbd /usr/lib/gdm-x-session[1593]: (EE) client bug: timer event20 debounce: offset negative (-3920ms)
Apr 09 21:56:57 es-nbd /usr/lib/gdm-x-session[1593]: (EE) client bug: timer event20 debounce short: offset negative (-3933ms)
Apr 09 21:57:10 es-nbd kernel: Asynchronous wait on fence NVIDIA:nvidia.prime:1109c timed out (hint:intel_atomic_commit_ready+0x0/0x58 [i915])

akortunov · 2019-04-10T05:54:46Z

All right, it seems I will have to drop the cursor scaling at all. Can you check again if the UI (including inventory) works fine with commented scaleImage calls?

…#4968)

elsid · 2019-04-11T18:42:12Z

UI with commented scaleImage works fine.

akortunov · 2019-04-12T03:53:13Z

Ok, I dropped the cursor scaling.

Capostrophic reviewed Apr 4, 2019

View reviewed changes

apps/openmw/mwgui/mainmenu.cpp Outdated Show resolved Hide resolved

akortunov changed the title ~~[WIP] Scalable UI textures~~ Scalable UI textures Apr 6, 2019

akortunov changed the title ~~Scalable UI textures~~ [Testing needed] Scalable UI textures Apr 6, 2019

Allow to calculate UI skin size based on texture resolution (feature …

31ddb0a

…#4968)

akortunov force-pushed the guifixes branch from 32d2189 to 31ddb0a Compare April 12, 2019 03:52

akortunov changed the title ~~[Testing needed] Scalable UI textures~~ Scalable UI textures Apr 12, 2019

psi29a merged commit 785a667 into OpenMW:master Apr 12, 2019

akortunov mentioned this pull request Apr 20, 2019

Update background texture if a new one is empty (regression #4986) #2334

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Scalable UI textures #2302

Scalable UI textures #2302

akortunov commented Apr 2, 2019 •

edited

akortunov commented Apr 4, 2019 •

edited

psi29a commented Apr 4, 2019

akortunov commented Apr 4, 2019

akortunov commented Apr 5, 2019 •

edited

akortunov commented Apr 5, 2019 •

edited

akortunov commented Apr 5, 2019 •

edited

AnyOldName3 commented Apr 5, 2019

akortunov commented Apr 5, 2019 •

edited

akortunov commented Apr 6, 2019

akortunov commented Apr 6, 2019

elsid commented Apr 7, 2019

Capostrophic commented Apr 7, 2019

psi29a commented Apr 8, 2019

elsid commented Apr 8, 2019

akortunov commented Apr 8, 2019 •

edited

akortunov commented Apr 9, 2019

elsid commented Apr 9, 2019

akortunov commented Apr 10, 2019

elsid commented Apr 11, 2019

akortunov commented Apr 12, 2019

Scalable UI textures #2302

Scalable UI textures #2302

Conversation

akortunov commented Apr 2, 2019 • edited

akortunov commented Apr 4, 2019 • edited

psi29a commented Apr 4, 2019

akortunov commented Apr 4, 2019

akortunov commented Apr 5, 2019 • edited

akortunov commented Apr 5, 2019 • edited

akortunov commented Apr 5, 2019 • edited

AnyOldName3 commented Apr 5, 2019

akortunov commented Apr 5, 2019 • edited

akortunov commented Apr 6, 2019

akortunov commented Apr 6, 2019

elsid commented Apr 7, 2019

Capostrophic commented Apr 7, 2019

psi29a commented Apr 8, 2019

elsid commented Apr 8, 2019

akortunov commented Apr 8, 2019 • edited

akortunov commented Apr 9, 2019

elsid commented Apr 9, 2019

akortunov commented Apr 10, 2019

elsid commented Apr 11, 2019

akortunov commented Apr 12, 2019

akortunov commented Apr 2, 2019 •

edited

akortunov commented Apr 4, 2019 •

edited

akortunov commented Apr 5, 2019 •

edited

akortunov commented Apr 5, 2019 •

edited

akortunov commented Apr 5, 2019 •

edited

akortunov commented Apr 5, 2019 •

edited

akortunov commented Apr 8, 2019 •

edited