Adds support for invalidating a single rectangle. #817

jneem · 2020-04-08T17:00:25Z

This adds support in druid-shell for invalidating a single rectangle.

Caveats/questions:

I checked that the win/mac changes compile, but I can't test them. Mac is particularly likely to be wrong, since I have no idea how rust-objc works, and how it maps to the apple docs online.
Should the plain invalidate methods be deprecated? Or should I just keep the name and introduce the rect parameter?

raphlinus

Overall this looks like a good direction, a couple comments inline.

My biggest concern is around scrolling; the logic for propagating invalidations is different for that than for most other containment.

Apologies my bandwidth for reviewing this in more detail is more limited than usual.

druid/src/contexts.rs

druid/src/widget/invalidation.rs

jneem · 2020-04-13T22:12:02Z

So I had a first stab at getting scrolling working. I'm not super happy with it (which is why I only implemented EventCtx so far), but it's the best I have right now.

As far as I understand, the way scrolling eventually works is the following: the scroll widget will maintain a buffer that contains the painted contents of its children. That buffer will be somewhat bigger than the viewport. When the Scroll is scrolled, it just blits from the buffer. It can handle asking its child to rerender the buffer on its own time, possibly even as an idle callback.

This means that:

if a child of the scroll requests repainting some visible rect, the scroll needs to propagate that back up to the system, properly translated; but also,
the scroll needs to store the invalidated regions somewhere, in order do do its own repainting.

AFAICT, these two constraints (and especially the second) means that the Scroll really needs to insert a callback somewhere. And this was the least-ugly place I could find to put it.

jneem · 2020-04-13T22:16:17Z

By the way, the current implementation has a buglet, in that there is a situation where it invalidates too much (and I don't know how to fix it without more drastic changes): if the scroll has a Flex child, and that child invalidates something in the viewport and something out of the viewport, then the bounding box of those two invalidations will be quite large (but this isn't the bug yet). The bug is that the part outside the viewport won't get painted, because Flex doesn't paint its invisible children. That means it will never get uninvalidated (validated?), and so that large bounding box will keep getting repainted.

I'm not planning to do anything about this right away, I just wanted to right it down so I don't forget...

cmyr

Some notes and questions, but I think this is in good shape should should be ready to go soon!

druid-shell/src/platform/mac/window.rs

druid-shell/src/window.rs

druid/examples/scroll.rs

druid/src/contexts.rs

druid/src/core.rs

cmyr · 2020-04-17T15:09:08Z

druid/src/widget/invalidation.rs

@@ -0,0 +1,67 @@
+// Copyright 2020 The xi-editor Authors.
+//


the way I might consider implementing this is not as a separate widget, but as part of WidgetPod's paint method? It's possible this doesn't make sense, but it's my first instinct.

I originally tried to imitate the implementation of debug_paint_layout (i.e. using EnvScope and WidgetPod::paint instead of a new widget). The problem is that, unlike drawing layout-rect boundaries, we don't want to recursively draw the invalidation regions for the children. It just occurred to me, though, that I can probably use Painter instead of a new widget.

I see what you mean.

actually thinking about this more (and further into my review), maybe it does make sense to draw them recursively? This would clarify where the invalidation calls that produced the final result came from?

In this world, I might also just draw outlines of the invalidated regions, so that we don't end up stacking up the alpha channel and obscuring the view.

druid/src/window.rs

druid/src/widget/invalidation.rs

jneem · 2020-04-17T16:49:26Z

Thanks for the review! I'll do a round or two of clean-up and then remove the [WIP]. I'd still feel better if someone briefly tested windows/mac, though...

jneem · 2020-04-17T22:19:02Z

Thanks for the review! I'll do a round or two of clean-up and then remove the [WIP]. I'd still feel better if someone briefly tested windows/mac, though...

jneem · 2020-04-20T15:00:18Z

The remaining CI failures appear to be linking errors that I can't reproduce locally...

Fixes #402. Widgets can request partial invalidation using `request_paint_rect`. All of the partial invalidation requests will be combined into a single bounding rectangle, and passed to the system. When painting widgets, they have access to the dirty region. With this commit, widgets that paint outside their paint rects can cause graphical glitches. These widgets are buggy, but their bugs weren't so visible before (see #628).

jneem · 2020-04-21T04:42:24Z

Ok, seems like a rebase resolved it.

cmyr

Okay the code looks generally good but playing with the example I'm not sure I'm seeing the expected behaviour (on mac).

druid-shell/examples/invalidate.rs

cmyr · 2020-04-21T15:22:05Z

druid/src/widget/invalidation.rs

+        let color = env.get_debug_color(self.debug_color).with_alpha(0.5);
+        let rect = ctx.region().to_rect();
+        ctx.fill(rect, &color);
+        self.debug_color += 1;


what's the rationale for changing the color, here? to me it makes sense to have the color be unchanged between paint cycles.

Without the color change, every time a region gets invalidated it gets painted with a blue (say) overlay. The overall effect is that the entire window is covered with a blue overlay all the time, so you can't see what was just invalidated.

I'm confused; wouldn't that only be the case if the entire view was invalidated?

druid/src/widget/mod.rs

cmyr · 2020-04-21T15:24:14Z

druid/examples/invalidation.rs

+            ctx.fill(Circle::new(*data, RADIUS), &Color::BLACK);
+        })
+    }
+}


so on macOS, this example is not displaying any visible minimal invalidation; on click the whole screen is tinted.

Ok, that's not right. Does the invalidation test in druid_shell work? It should look something like the attached gif.

no, on mac it is just a window with a single solid background color. I'll dig in a little bit...

cmyr · 2020-04-21T15:26:08Z

druid/src/widget/invalidation.rs

@@ -0,0 +1,67 @@
+// Copyright 2020 The xi-editor Authors.
+//


actually thinking about this more (and further into my review), maybe it does make sense to draw them recursively? This would clarify where the invalidation calls that produced the final result came from?

In this world, I might also just draw outlines of the invalidated regions, so that we don't end up stacking up the alpha channel and obscuring the view.

jneem · 2020-04-21T19:24:34Z

I'll try out a few options for the debug visualization. I don't think recursive drawing will work well, because there's just one single invalidation rect for the entire window, and so any individual widget's invalidation rect is just its paint rect intersected with the global rect -- there's no interesting recursive structure like there is for the layout rects.

I'll also try doing outlines.

jneem · 2020-04-21T20:06:18Z

By the way, this is what the (druid) invalidation test looks like for me.

jneem · 2020-04-21T21:26:41Z

Here are two other options: recursive outlines and non-recursive outlines. I think my favorite is the non-recursive outlines.

cmyr

Okay, I found two small issues keeping the examples from working on mac.

With these changes, this works as intended.

One other thing I noticed digging in a bit more deeply is that animation always invalidates everything; I think this is okay for now but we should open an issue about it when this gets merged.

druid-shell/src/platform/mac/window.rs

cmyr · 2020-04-22T14:33:48Z

druid-shell/examples/invalidate.rs

+    fn connect(&mut self, handle: &WindowHandle) {
+        self.handle = handle.clone();
+    }
+
+    fn paint(&mut self, piet: &mut Piet, rect: Rect) -> bool {
+        self.update_color_and_rect();
+        piet.fill(rect, &self.color);
+
+        self.handle.invalidate_rect(self.rect);
+        false
+    }


on mac, invalidation calls that happen during drawing are ignored.

I would suggest changing this to use a timer, something like:

fn connect(&mut self, handle: &WindowHandle) { self.handle = handle.clone(); self.handle.request_timer(std::time::Duration::from_millis(60)); } fn timer(&mut self, _id: TimerToken) { self.update_color_and_rect(); self.handle.invalidate_rect(self.rect); self.handle.request_timer(std::time::Duration::from_millis(60)); } fn paint(&mut self, piet: &mut Piet, rect: Rect) -> bool { piet.fill(rect, &self.color); false }

jneem · 2020-04-22T15:20:13Z

Thanks! I fixed those, and also some other issues with propagating a too-large invalid region in druid (in cairo, the drawing context is automatically clipped, so I wasn't seeing the issue).

Regarding the animation, there was a reason I hadn't done it yet, but I can't remember why...

cmyr · 2020-04-22T17:05:41Z

I wouldn't worry about the animation, it will be sort of tricky to get right with the current API. The longer-term plan is to build a higher-level API on top of what we currently have, and when we do that we can revisit.

I guess it would be possible to make some specific improvements now, though; but this would require turning request_anim into a Region or something.

cmyr · 2020-04-22T17:07:11Z

Also I'm not sure I can tell the difference between the recursive v. non-recursive outlines; I defer to your judgement.

jneem · 2020-04-22T17:22:31Z

Ok, so I've gone with non-recursive outlines then. It's easy enough to change later anyway.

Also, I'll open an issue about request_anim.

cmyr

Great, thanks for your patience on this, I'm satisfied. I'll ping @raphlinus and see if he wants to take a look, I think we're looking good though.

Just out of curiousity, have you used this and found a measurable improvement in performance?

cmyr · 2020-04-22T17:54:43Z

Oh, one last thing: it might be worth adding this to the wasm examples? in druid/examples/wasm. (forgive me if this is don't, I skimmed and didn't see i)

jneem · 2020-04-22T18:36:20Z

I think it's automatic, right? druid/examples/wasm/src/examples seems to be symlinked to druid/examples.

Edit: although now that I'm testing the wasm example, the whole thing seems to get invalidated...

jneem · 2020-04-22T20:42:06Z

Ok, I think I may have fixed the wasm example. I am a bit puzzled by the two calls to piet_ctx.finish() in druid_shell::platform::web::WindowState::render() -- is that just an oversight?

I had a toy example showing improved performance for small invalidations when drawing is expensive. I can try cleaning it up and adding it as an example. But my real motivation for doing this needs more work before it can benefit...

cmyr · 2020-04-23T01:21:18Z

I can't speak to the wasm stuff; maybe @elrnv can. In any case I'm happy to merge this now, thanks again for your patience!

elrnv · 2020-04-23T05:12:17Z

I am not sure why we need to finish calls. For reference, I believe the code @jneem is referring to is in the render function

fn render(&self) -> bool {
    self.context
        .clear_rect(0.0, 0.0, self.get_width() as f64, self.get_height() as f64);
    let mut piet_ctx = piet_common::Piet::new(self.context.clone(), self.window.clone());
    let want_anim_frame = self.handler.borrow_mut().paint(&mut piet_ctx);
    if let Err(e) = piet_ctx.finish() {
        log::error!("piet error on render: {:?}", e);
    }
    let res = piet_ctx.finish();
    if let Err(e) = res {
        log::error!("EndDraw error: {:?}", e);
    }
    want_anim_frame
}

This part is over a year old. It does look redundant to me. After trying the examples without the second piet_ctx.finish(), I couldn't find any difference. I think it's safe to remove the second .finish() call.

raphlinus reviewed Apr 10, 2020

View reviewed changes

druid/src/contexts.rs Outdated Show resolved Hide resolved

druid/src/widget/invalidation.rs Show resolved Hide resolved

cmyr reviewed Apr 17, 2020

View reviewed changes

cmyr added the S-waiting-on-author waits for changes from the submitter label Apr 17, 2020

jneem changed the title ~~[WIP] Adds support for invalidating a single rectangle.~~ Adds support for invalidating a single rectangle. Apr 19, 2020

cmyr reviewed Apr 21, 2020

View reviewed changes

Fix the small issues from code review.

5adab13

Fix propagation of invalid region to children.

efd6521

cmyr requested changes Apr 22, 2020

View reviewed changes

Hopefully fix invalidation on mac.

8959bc5

Use rectangle outlines for debug_invalidation.

fed49b1

jneem mentioned this pull request Apr 22, 2020

request_anim invalidates the entire window #867

Open

cmyr approved these changes Apr 22, 2020

View reviewed changes

jneem added 2 commits April 22, 2020 15:22

Propagate the invalid rects properly in wasm.

0f8b861

Invalidate more conservatively when animating.

dc9b5e9

cmyr merged commit fec0f0e into linebender:master Apr 23, 2020

cmyr mentioned this pull request Apr 23, 2020

redundant calls to finish() in wasm render method #868

Closed

This was referenced Apr 24, 2020

The invalidation example behaves differently with wasm #872

Closed

Intense black flickering and artifacts on Windows #875

Closed

xStrom mentioned this pull request Apr 28, 2020

Improve Windows invalidation logging. #882

Merged

xStrom removed the S-waiting-on-author waits for changes from the submitter label May 15, 2020

Adds support for invalidating a single rectangle. #817

Adds support for invalidating a single rectangle. #817

Conversation

jneem commented Apr 8, 2020

raphlinus left a comment

Choose a reason for hiding this comment

jneem commented Apr 13, 2020

jneem commented Apr 13, 2020

cmyr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jneem commented Apr 17, 2020

jneem commented Apr 17, 2020

jneem commented Apr 20, 2020

jneem commented Apr 21, 2020

cmyr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jneem commented Apr 21, 2020

jneem commented Apr 21, 2020

jneem commented Apr 21, 2020

cmyr left a comment

Choose a reason for hiding this comment

cmyr Apr 22, 2020 • edited Loading

Choose a reason for hiding this comment

jneem commented Apr 22, 2020

cmyr commented Apr 22, 2020

cmyr commented Apr 22, 2020

jneem commented Apr 22, 2020

cmyr left a comment

Choose a reason for hiding this comment

cmyr commented Apr 22, 2020

jneem commented Apr 22, 2020 • edited Loading

jneem commented Apr 22, 2020

cmyr commented Apr 23, 2020

elrnv commented Apr 23, 2020 • edited Loading

cmyr Apr 22, 2020 •

edited

Loading

jneem commented Apr 22, 2020 •

edited

Loading

elrnv commented Apr 23, 2020 •

edited

Loading