Can't do a glyph-transparent varying-color plane

[dankamongmen]

While working on the book, I realized a major oversight: you can't do a glyph-transparency having more than one color!

In the uniblock demo and trans demo, we do a plane which is glyph-transparent (all gcluster values are 0), but has a base cell color, thus changing the color of the glyphs underneath (but not the glyphs themselves). this is well and good.

but let's say you want to apply a gradient in the same way, using a glyph-transparent plane (i.e. not simply staining the plane with the glyphs). you'd set each cell's color, set it to BLEND or OPAQUE, and set each gcluster to 0, right? wrong. any cell with a 0 gcluster uses the base cell in its entirety. and sometimes this is what we want -- we want them all to have the same color and glyph, easy, set the base cell.

you can't just set the gclusters to space, because that would block the glyphs underneath (we can't change that semantic -- you want it, or printing becomes crazy). i don't like the idea of getting rid of the base cell (useful for automatic handling of resizes) or anything like that.

ugh. we could maybe do something horrible like "if gcluster==1, apply only glyph". that's just ghastly :(. think about it some more.

[dankamongmen]

maaaaaaaaaaaaaaybe we say "we apply the color on a null glyph, so long as the base cell is also a null glpyh?" it's not particularly nice, but it would work for some cases...

[dankamongmen]

I think we can actually work around this pretty easily by taking a bit from the 6 reserved bits we have left in the channels variable. If you want a hard passthrough, indicate it there. Alternatively, we could employ some control character for this... yeah, either of those solutions ought work.

[dankamongmen]

We need to work this out before the 2.0 API freeze.

[dankamongmen]

What if we said "to get the base character, you need to be all 0s not just in your glyph, but in your colors as well"? would that work? i don't see any immediate problems with it, it avoids overloading control characters, and it uses no new channel bits.

there might be some case i'm missing, though.

i think i'll make this change, and see if the demos are at all affected by it. if so, we might take another look. if not, i'm going to assume this is fine, and keep the change. i'm just worried there might be a technique i'm overlooking. this is just as powerful as the other two solutions, without any weird overloads, and i think it even logically makes sense: if i set a color, i intend for that color to be used! the only exception would be if i've e.g. stained a large area. but resetting the colors is just as easy as resetting the glyphs in such a case...yeah, i think i like this solution.

[dankamongmen]

The other thing to consider is...what if I want the default to stain, as well?

I suppose in that case you can always set the base char to default, but that kills the base char for other purposes. Maybe we do want to go ahead and use those two channel bits? But then what about the nonsense case of glyph set + force bit?

[dankamongmen]

I think this is almost the right approach, but with one small tweak: rather than checking for the default color, check for whether we're non-transparent. If the channel is CELL_ALPHA_OPAQUE, CELL_ALPHA_BLEND, or CELL_ALPHA_TRANSPARENT, use it. Let's see how that looks. Do remember, of course, that the default is CELL_ALPHA_OPAQUE...

[dankamongmen]

Either way, we're left with some value that we can't override the base cell with...but that's still better than the current situation, where we can't override the base cell with any colors. I think we'll want to set certain areas transparent much more than we'll want set certain areas default. Let's go with the plan from earlier, where you must be the default color to fall through to base cell. That fits in better with the idea of a zero-default, after all. Neither way is particularly elegant.

[dankamongmen]

OK, I have implemented this via deciding between the viscell and the basecell at each of the three steps of paint() -- glyph, bgcolor, and fgcolor. It pretty much works, except we're getting bleedthrough on qrdemo.

[dankamongmen]

chasing down this qrcode weirdness i'm seeing a very strange thing. let's look at the target coordinate y=77 x=103.

here's what we see from paint():

077/103 -> 077/103 switch to vis
077/103 -> 077/103 vis foreground 40000000 targc foreground 60000000
077/103 targc 0x55bb66fadde0 foreground 40000000
005/103 -> 077/103 switch to vis
005/103 -> 077/103 vis foreground 50201020 targc foreground 40000000
077/103 targc 0x55bb66fadde0 foreground 40000000
045/065 -> 077/103 vis foreground 40c25671 targc foreground 60000000
077/103 targc 0x55bb6701eca0 foreground 40c25671
077/103 -> 077/103 switch to vis
077/103 -> 077/103 vis foreground 40000000 targc foreground 40c25671
077/103 targc 0x55bb6701eca0 foreground 40c25671
005/103 -> 077/103 switch to vis
005/103 -> 077/103 vis foreground 50201020 targc foreground 40c25671
077/103 targc 0x55bb6701eca0 foreground 40c25671
*************************** notcurses debug state *****************************
0000 off y:   0 x:   0 geom y:   8 x: 104 curs y:   0 x: 104     0x55bb66f6b300
0001 off y:  32 x:  38 geom y:  78 x: 104 curs y:   0 x:   0     0x55bb66f63600
0002 off y:   0 x:   0 geom y:  78 x: 104 curs y:   0 x:   0 std 0x55bb66f43700
0003 off y:  13 x:   1 geom y:   5 x:  32 curs y:   3 x:  30     0x55bb66f658c0
0004 off y:  72 x:   0 geom y:   6 x: 104 curs y:   0 x:   0     0x55bb66f68390
*******************************************************************************

Why on earth is the target address changing? That's what's giving rise to our problem -- we suddenly are targeting this other address, which is freshly initialized to 0x60000000. I don't at all understand why the target address is changing.

The first two planes (menu and shifted dup plane, the latter containing our output) hit the same place (0x55bb66fadde0), and leave targc prepared as we would expect. The next three planes (stdplane, HUD, and plot) hit a different one (0x55bb6701eca0).

[dankamongmen]

Wait, that interpretation is flawed. menu plane isn't overlapping this coordinate. Let me get another set....

Ahhh, ok, the initial output was from a prior render. Here we go:

0x55f89804d600 051/077 -> 077/103 vis foreground 4094705e targc foreground 60000000
0x55f89804d600 077/103 targc 0x55f898108ca0 foreground 4094705e
0x55f89802d700 077/103 -> 077/103 switch to vis
0x55f89802d700 077/103 -> 077/103 vis foreground 40000000 targc foreground 4094705e
0x55f89802d700 077/103 targc 0x55f898108ca0 foreground 4094705e
0x55f898052390 005/103 -> 077/103 switch to vis
0x55f898052390 005/103 -> 077/103 vis foreground 50201020 targc foreground 4094705e
0x55f898052390 077/103 targc 0x55f898108ca0 foreground 4094705e
*************************** notcurses debug state *****************************
0000 off y:   0 x:   0 geom y:   8 x: 104 curs y:   0 x: 104     0x55f898055300
0001 off y:  26 x:  26 geom y:  78 x: 104 curs y:   0 x:   0     0x55f89804d600
0002 off y:   0 x:   0 geom y:  78 x: 104 curs y:   0 x:   0 std 0x55f89802d700
0003 off y:  13 x:   1 geom y:   5 x:  32 curs y:   3 x:  30     0x55f89804f8c0
0004 off y:  72 x:   0 geom y:   6 x: 104 curs y:   0 x:   0     0x55f898052390
*******************************************************************************

All three writes are now to the correct (same, at least) address. It would appear that our viscell on the target plane has foreground 0x4094705e, which is our true problem -- this is picked up and used, staining the FPS plot. So our problem is presumably just overeager staning within qrcode, not a general one.

[dankamongmen]

Yep, that was it, got it.