Curses doesn't display octants

[cool-RR]

Bug report

Bug description:

I would like to get Unicode octants to display correctly in VisiData. saulpw/visidata#2626

For some reason these characters don't show correctly.

After midichef debugged it, he found that the curses module doesn't display these characters correctly:

#!/usr/bin/python
import curses

octants = ''.join(chr(0x1cd00+i) for i in range(230))
bracketed = '[' + octants + ']'
def main(stdscr):
    stdscr.addstr(f'octants drawn in curses: {bracketed}')
    stdscr.refresh()
    stdscr.getch()

try:
    curses.wrapper(main)
except KeyboardInterrupt:
    pass
print(f'octants printed to stdout: {bracketed}')

Output:

Is this a bug in curses or maybe I misunderstood something about how to operate it?

CPython versions tested on:

3.12

Operating systems tested on:

Linux

[terryjreedy]

Its nice that your linux distribution supports these newish chars. Win 10 does not seem tocurrently, as they are printed as rectangles either blank (CommandPrompt), crossed (tk), or with 6 hex chars (Firefox).

[cool-RR]

@terryjreedy These octants work for me on Windows 10 as well. You just need to download a supporting font, like the latest version of Cascadia.

[midichef]

(summary: the problem here appears not to be specific to CPython. It shows up with curses in C too)

To clarify: the problem is how curses displays octant characters, introduced in Unicode 16.0, with the values in the range '\U0001cd00' through \U0001cde5, inclusive. curses draws them as empty spaces, whereas Python prints them properly as squares with hex codes (if the OS does not have these characters in the system font). And the problem is not specific to the octant characters. It may be all the new characters introduced in Unicode 16.0.

What I get from printf() in C, and what I hope to see from curses in Python is:

What I actually see from curses in Python and from C using libncursesw5 with no Python is, the last two characters, for '\U0001cd00' and '\U0001fac6', are drawn as empty spaces:

I can reproduce the problem even without installing any extra fonts, as long as my system draws some non-empty symbol for the characters when a font is not installed. In Ubuntu, using gnome-terminal, such characters are drawn as a hex code in a square, as seen in the screenshots above.

As seen in the screenshot above, I can reproduce the same effect even when I have installed a font that includes the octant characters: the Cascadia Code fonts version 2407.24, using the specific Cascadia Mono font.

However, I do not think this issue is specific to CPython. The problem occurs with curses in plain C as well, with no Python. But do note that my understanding of C and curses is quite limited. Here is the demonstration code I used to generate the images above. Note that curses properly handles the characters introduced in Unicode 15.0 and 15.1. It only demonstrates the problem of "empty space" with the characters that are new to 16.0.

#include <stdio.h>
#include <ncurses.h>
#include <locale.h>

/* contrast the display of characters in curses addwstr() vs printf */
int main(int argc, char *argv[]) 
{ 
    setlocale(LC_ALL, "");
    initscr();
    const wchar_t *ch_15_0_wifi   = L"[\U0001f6dc]"; /* introduced in Unicode 15.0 🛜 */
    const wchar_t *ch_15_1_hex    = L"[\U0002ebf0]"; /* introduced in Unicode 15.1 𮯰 */
    const wchar_t *ch_16_0_octant = L"[\U0001cd00]"; /* introduced in Unicode 16.0 𜴀  */
    const wchar_t *ch_16_0_misc   = L"[\U0001fac6]"; /* introduced in Unicode 16.0 🫆 - "fingerprint", a randomly chosen character of the new characters in Unicode 16.0, to show that the problem is not specific to the block octants  */
    int y=0;
    mvaddwstr(y, 0, ch_15_0_wifi);
    mvaddwstr(y, 3, ch_15_1_hex);
    mvaddwstr(y, 6, ch_16_0_octant);
    mvaddwstr(y, 9, ch_16_0_misc);
    mvprintw(y+1, 0, "Press a key to exit");
    refresh(); getch(); endwin();
    printf("printf() output:  %ls%ls%ls%ls\n", ch_15_0_wifi, ch_15_1_hex, ch_16_0_octant, ch_16_0_misc);
    return 0;
}

I compile this code with the command g++ so.c -o so -lncursesw -Werror -D_GNU_SOURCE -DUNICODE
using the libncurses-dev package, Version: 6.4+20240113-1ubuntu2, on Ubuntu 24.04.1,
gnome-terminal, # GNOME Terminal 3.52.0 using VTE 0.76.0 +BIDI +GNUTLS +ICU +SYSTEMD,
with the environment variable LANG set to en_US.UTF-8 and LC_COLLATE set to C.

Please let me know if this is not the correct way to compile or run such code. But if I have not made any errors, then the problem is not a Python problem at all, and this issue should be closed.

[ZeroIntensity]

Yeah, this looks like an issue with ncurses upstream. You'll have to report this to the bug-ncurses mailing list.

[cool-RR]

Yeah, this looks like an issue with ncurses upstream. You'll have to report this to the bug-ncurses mailing list.

Reported, thank you.