New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] Reimplement builtin terminal UI with termkey/unibilium #1820

Merged
merged 14 commits into from Feb 17, 2015

Conversation

Projects
None yet
@tarruda
Member

tarruda commented Jan 15, 2015

This PR will throw a lot of old code into the garbage can.

@justinmk

This comment has been minimized.

Show comment
Hide comment
@justinmk
Member

justinmk commented Jan 15, 2015

:)

Show outdated Hide outdated .gitignore Outdated

@marvim marvim added the WIP label Jan 15, 2015

@bfredl

This comment has been minimized.

Show comment
Hide comment
@bfredl

bfredl Jan 15, 2015

Member

Nice line count of this PR. 👍 Segfaults for the moment with my nvimrc tho, but with nvim -u NONE it works just fine :)

Member

bfredl commented Jan 15, 2015

Nice line count of this PR. 👍 Segfaults for the moment with my nvimrc tho, but with nvim -u NONE it works just fine :)

@bfredl

This comment has been minimized.

Show comment
Hide comment
@bfredl

bfredl Jan 15, 2015

Member

Not sure if helpful, but all craches ends with

Program received signal SIGABRT, Aborted.
0x00007fa72c06ba97 in raise () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007fa72c06ba97 in raise () from /usr/lib/libc.so.6
#1  0x00007fa72c06ce6a in abort () from /usr/lib/libc.so.6
#2  0x00007fa72c0ab2b3 in __libc_message () from /usr/lib/libc.so.6
#3  0x00007fa72c0b072e in malloc_printerr () from /usr/lib/libc.so.6
#4  0x00007fa72c0b0eeb in _int_free () from /usr/lib/libc.so.6
#5  0x00000000005f48b5 in destroy_screen (ui=0x170aff0) at /home/bjorn/Programming/neovim/src/nvim/tui/tui.c:315
#6  0x00000000005f448c in try_resize (ev=...) at /home/bjorn/Programming/neovim/src/nvim/tui/tui.c:241
#7  0x00000000005e328d in process_events_from (queue=0x16fac90) at /home/bjorn/Programming/neovim/src/nvim/os/event.c:174
#8  0x00000000005e3189 in event_poll (ms=0) at /home/bjorn/Programming/neovim/src/nvim/os/event.c:135
#9  0x00000000005e59d2 in input_poll (ms=0) at /home/bjorn/Programming/neovim/src/nvim/os/input.c:25

It crashes on resize and at random times it decides to resize itself (e.g. when doing cmdline completion) with the aforementioned result.

Member

bfredl commented Jan 15, 2015

Not sure if helpful, but all craches ends with

Program received signal SIGABRT, Aborted.
0x00007fa72c06ba97 in raise () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007fa72c06ba97 in raise () from /usr/lib/libc.so.6
#1  0x00007fa72c06ce6a in abort () from /usr/lib/libc.so.6
#2  0x00007fa72c0ab2b3 in __libc_message () from /usr/lib/libc.so.6
#3  0x00007fa72c0b072e in malloc_printerr () from /usr/lib/libc.so.6
#4  0x00007fa72c0b0eeb in _int_free () from /usr/lib/libc.so.6
#5  0x00000000005f48b5 in destroy_screen (ui=0x170aff0) at /home/bjorn/Programming/neovim/src/nvim/tui/tui.c:315
#6  0x00000000005f448c in try_resize (ev=...) at /home/bjorn/Programming/neovim/src/nvim/tui/tui.c:241
#7  0x00000000005e328d in process_events_from (queue=0x16fac90) at /home/bjorn/Programming/neovim/src/nvim/os/event.c:174
#8  0x00000000005e3189 in event_poll (ms=0) at /home/bjorn/Programming/neovim/src/nvim/os/event.c:135
#9  0x00000000005e59d2 in input_poll (ms=0) at /home/bjorn/Programming/neovim/src/nvim/os/input.c:25

It crashes on resize and at random times it decides to resize itself (e.g. when doing cmdline completion) with the aforementioned result.

@BlackEdder

This comment has been minimized.

Show comment
Hide comment
@BlackEdder

BlackEdder Jan 16, 2015

This ui doesn't seem to work correctly with terminology (https://www.enlightenment.org/p.php?p=about/terminology) and from my limited testing neither did pynvim. The problem seem to have to do with getting the correct terminal width. The first letter of every sentence starts at the end of the line and then wraps around. Might be a tickit issue I guess (@leonerd)?

BlackEdder commented Jan 16, 2015

This ui doesn't seem to work correctly with terminology (https://www.enlightenment.org/p.php?p=about/terminology) and from my limited testing neither did pynvim. The problem seem to have to do with getting the correct terminal width. The first letter of every sentence starts at the end of the line and then wraps around. Might be a tickit issue I guess (@leonerd)?

Show outdated Hide outdated src/nvim/CMakeLists.txt Outdated
@tarruda

This comment has been minimized.

Show comment
Hide comment
@tarruda

tarruda Jan 16, 2015

Member

Nice line count of this PR. 👍 Segfaults for the moment with my nvimrc tho, but with nvim -u NONE it works just fine :)

I'm just getting started, I think this PR will remove more than 7k lines :)

It crashes on resize and at random times it decides to resize itself (e.g. when doing cmdline completion) with the aforementioned result.

Could be what @oni-link spotted, I will investigate.

Member

tarruda commented Jan 16, 2015

Nice line count of this PR. 👍 Segfaults for the moment with my nvimrc tho, but with nvim -u NONE it works just fine :)

I'm just getting started, I think this PR will remove more than 7k lines :)

It crashes on resize and at random times it decides to resize itself (e.g. when doing cmdline completion) with the aforementioned result.

Could be what @oni-link spotted, I will investigate.

@tarruda

This comment has been minimized.

Show comment
Hide comment
@tarruda

tarruda Jan 17, 2015

Member

This ui doesn't seem to work correctly with terminology (https://www.enlightenment.org/p.php?p=about/terminology) and from my limited testing neither did pynvim. The problem seem to have to do with getting the correct terminal width. The first letter of every sentence starts at the end of the line and then wraps around. Might be a tickit issue I guess (@leonerd)?

👍 Will investigate.

Some legacy functional tests are failing, but the UI should be as stable as the python version.

Member

tarruda commented Jan 17, 2015

This ui doesn't seem to work correctly with terminology (https://www.enlightenment.org/p.php?p=about/terminology) and from my limited testing neither did pynvim. The problem seem to have to do with getting the correct terminal width. The first letter of every sentence starts at the end of the line and then wraps around. Might be a tickit issue I guess (@leonerd)?

👍 Will investigate.

Some legacy functional tests are failing, but the UI should be as stable as the python version.

@fwalch

This comment has been minimized.

Show comment
Hide comment
@fwalch

fwalch Jan 17, 2015

Member

Is it possible to get the libtickit/libtermkey changes into upstream before this is merged into master? I ask because e.g. Arch Linux will need to package these libraries, and first introducing a special libtickit-unstable-neovim and then replacing it with libtickit-unstable at some point will be a bit more work.

Member

fwalch commented Jan 17, 2015

Is it possible to get the libtickit/libtermkey changes into upstream before this is merged into master? I ask because e.g. Arch Linux will need to package these libraries, and first introducing a special libtickit-unstable-neovim and then replacing it with libtickit-unstable at some point will be a bit more work.

@tarruda

This comment has been minimized.

Show comment
Hide comment
@tarruda

tarruda Jan 17, 2015

Member

Is it possible to get the libtickit/libtermkey changes into upstream before this is merged into master? I ask because e.g. Arch Linux will need to package these libraries, and first introducing a special libtickit-unstable-neovim and then replacing it with libtickit-unstable at some point will be a bit more work.

👍. Don't worry as this will only be merged when all tests are fixed and the changes are sent back to @leonerd.

Did you get a chance to test this branch? How stable is it for you?

Member

tarruda commented Jan 17, 2015

Is it possible to get the libtickit/libtermkey changes into upstream before this is merged into master? I ask because e.g. Arch Linux will need to package these libraries, and first introducing a special libtickit-unstable-neovim and then replacing it with libtickit-unstable at some point will be a bit more work.

👍. Don't worry as this will only be merged when all tests are fixed and the changes are sent back to @leonerd.

Did you get a chance to test this branch? How stable is it for you?

@fwalch

This comment has been minimized.

Show comment
Hide comment
@fwalch

fwalch Jan 17, 2015

Member

Did you get a chance to test this branch? How stable is it for you?

I had to check nvim --version to make sure I'm really using this branch, because I didn't notice any differences. Didn't have any problems so far :-)

Member

fwalch commented Jan 17, 2015

Did you get a chance to test this branch? How stable is it for you?

I had to check nvim --version to make sure I'm really using this branch, because I didn't notice any differences. Didn't have any problems so far :-)

@tarruda

This comment has been minimized.

Show comment
Hide comment
@tarruda

tarruda Jan 17, 2015

Member

I had to check nvim --version to make sure I'm really using this branch, because I didn't notice any differences. Didn't have any problems so far :-)

Nice, if you don't mind please keep using it. This PR introduces some serious architectural changes and the more people are testing it the better.

I'm interested in knowing this branch deals with troubling key mappings. One of the compelling reasons I choose libtickit/libtermkey is because it is capable of understanding an encoding scheme that uniquely identifies each key(though I'm not sure if some special terminal configuration is required for that)

Member

tarruda commented Jan 17, 2015

I had to check nvim --version to make sure I'm really using this branch, because I didn't notice any differences. Didn't have any problems so far :-)

Nice, if you don't mind please keep using it. This PR introduces some serious architectural changes and the more people are testing it the better.

I'm interested in knowing this branch deals with troubling key mappings. One of the compelling reasons I choose libtickit/libtermkey is because it is capable of understanding an encoding scheme that uniquely identifies each key(though I'm not sure if some special terminal configuration is required for that)

Show outdated Hide outdated src/nvim/tui/tui.c Outdated
Show outdated Hide outdated src/nvim/tui/tui.c Outdated

@tarruda tarruda referenced this pull request Jan 17, 2015

Closed

Pynvim segfaults #1837

@Tranquility

This comment has been minimized.

Show comment
Hide comment
@Tranquility

Tranquility Jan 17, 2015

Contributor

I realized a small problem with the shell output. Unlike the pynvim ui it is extremely fast which is awesome but the the background color is wrong. Only the text has background color of the colorscheme but the part without text is black.

Contributor

Tranquility commented Jan 17, 2015

I realized a small problem with the shell output. Unlike the pynvim ui it is extremely fast which is awesome but the the background color is wrong. Only the text has background color of the colorscheme but the part without text is black.

@aktau

This comment has been minimized.

Show comment
Hide comment
@aktau

aktau Jan 18, 2015

Member

I'm not entirely through the code yet, but where did input encoding go? (the removed lines in os/input.c).

Member

aktau commented Jan 18, 2015

I'm not entirely through the code yet, but where did input encoding go? (the removed lines in os/input.c).

@tarruda

This comment has been minimized.

Show comment
Hide comment
@tarruda

tarruda Jan 19, 2015

Member

I'm not entirely through the code yet, but where did input encoding go? (the removed lines in os/input.c).

I've removed it, the input_enqueue function should always receive utf-8 strings as input. I'm gonna investigate if that input conversion code is really necessary, and if so it will be done in the tickit UI layer.

Member

tarruda commented Jan 19, 2015

I'm not entirely through the code yet, but where did input encoding go? (the removed lines in os/input.c).

I've removed it, the input_enqueue function should always receive utf-8 strings as input. I'm gonna investigate if that input conversion code is really necessary, and if so it will be done in the tickit UI layer.

@oni-link

This comment has been minimized.

Show comment
Hide comment
@oni-link

oni-link Jan 19, 2015

Contributor

Working in a xterm terminal.

  • nvim -u NONE src/nvim/eval.c only shows the text area grey on black background.
  • Pressing Delete in insert mode inserts ^?.
  • With the solarized colorscheme: pressing 'o' in a block of text inserts
    empty lines that don't have the background color.
Contributor

oni-link commented Jan 19, 2015

Working in a xterm terminal.

  • nvim -u NONE src/nvim/eval.c only shows the text area grey on black background.
  • Pressing Delete in insert mode inserts ^?.
  • With the solarized colorscheme: pressing 'o' in a block of text inserts
    empty lines that don't have the background color.
@tarruda

This comment has been minimized.

Show comment
Hide comment
@tarruda

tarruda Jan 20, 2015

Member

nvim -u NONE src/nvim/eval.c only shows the text area grey on black background.
Pressing Delete in insert mode inserts ^?.
With the solarized colorscheme: pressing 'o' in a block of text inserts empty lines that don't have the background color.

👍

Member

tarruda commented Jan 20, 2015

nvim -u NONE src/nvim/eval.c only shows the text area grey on black background.
Pressing Delete in insert mode inserts ^?.
With the solarized colorscheme: pressing 'o' in a block of text inserts empty lines that don't have the background color.

👍

@oni-link

This comment has been minimized.

Show comment
Hide comment
@oni-link

oni-link Jan 21, 2015

Contributor

My last problems are fixed with the new commits. Here are some new:

  • Typing CTRL-h is registered in variable str in tui.c:key_event() as Backspace. Is this a libtickit error?
  • With the solarized colorscheme: Searching for a term highlights it until Enter is pressed.
    After that only the text color for the found matches is changed, no highlighting anymore.
Contributor

oni-link commented Jan 21, 2015

My last problems are fixed with the new commits. Here are some new:

  • Typing CTRL-h is registered in variable str in tui.c:key_event() as Backspace. Is this a libtickit error?
  • With the solarized colorscheme: Searching for a term highlights it until Enter is pressed.
    After that only the text color for the found matches is changed, no highlighting anymore.
Show outdated Hide outdated src/nvim/tui/tui.c Outdated
Show outdated Hide outdated src/nvim/tui/tui.c Outdated
static bool attrs_differ(HlAttrs a1, HlAttrs a2)
{
return a1.foreground != a2.foreground || a1.background != a2.background

This comment has been minimized.

@oni-link

oni-link Jan 21, 2015

Contributor

Shorter would be return memcmp(&a1,&a2,sizeof(a1));

@oni-link

oni-link Jan 21, 2015

Contributor

Shorter would be return memcmp(&a1,&a2,sizeof(a1));

This comment has been minimized.

@tarruda

tarruda Jan 21, 2015

Member

will that work if the compiler pads the struct?

@tarruda

tarruda Jan 21, 2015

Member

will that work if the compiler pads the struct?

This comment has been minimized.

@jszakmeister

jszakmeister Jan 21, 2015

Member

Answer: no. :-( It's a real tempting optimization to make, but it doesn't work unless you can be certain that the padding is zeroized--which is probably not a good assumption to make, even if our current paths all do it.

@jszakmeister

jszakmeister Jan 21, 2015

Member

Answer: no. :-( It's a real tempting optimization to make, but it doesn't work unless you can be certain that the padding is zeroized--which is probably not a good assumption to make, even if our current paths all do it.

@tarruda

This comment has been minimized.

Show comment
Hide comment
@tarruda

tarruda Jan 21, 2015

Member

With the solarized colorscheme: Searching for a term highlights it until Enter is pressed. After that only the text color for the found matches is changed, no highlighting anymore.

Could you post a screenshot of this? Either I dont understand or I'm unable to reproduce locally.

Member

tarruda commented Jan 21, 2015

With the solarized colorscheme: Searching for a term highlights it until Enter is pressed. After that only the text color for the found matches is changed, no highlighting anymore.

Could you post a screenshot of this? Either I dont understand or I'm unable to reproduce locally.

@elmart

This comment has been minimized.

Show comment
Hide comment
@elmart

elmart Feb 16, 2015

Member

Just to be clear, the freeze you are experiencing only happens with this branch right?

Now I think so again, but as it's something random, I can't confirm yet. Ignore it until I can provide more info on that. What I find strange is that nobody else is experiencing it.

Member

elmart commented Feb 16, 2015

Just to be clear, the freeze you are experiencing only happens with this branch right?

Now I think so again, but as it's something random, I can't confirm yet. Ignore it until I can provide more info on that. What I find strange is that nobody else is experiencing it.

@fmoralesc

This comment has been minimized.

Show comment
Hide comment
@fmoralesc

fmoralesc Feb 16, 2015

Contributor
Contributor

fmoralesc commented Feb 16, 2015

Show outdated Hide outdated src/nvim/main.c Outdated
@tarruda

This comment has been minimized.

Show comment
Hide comment
@tarruda

tarruda Feb 17, 2015

Member

Ok, this is as ready as its gonna get, merging after the tests.

@elmart as soon as you identify a pattern for reproducing the YCM slowdown, I will investigate and hopefully come up with a fix soon

Member

tarruda commented Feb 17, 2015

Ok, this is as ready as its gonna get, merging after the tests.

@elmart as soon as you identify a pattern for reproducing the YCM slowdown, I will investigate and hopefully come up with a fix soon

@justinmk

This comment has been minimized.

Show comment
Hide comment
@justinmk

justinmk Feb 17, 2015

Member

Great job!

Member

justinmk commented Feb 17, 2015

Great job!

tarruda added some commits Feb 12, 2015

test: Add synchronization helper for functional tests
The `wait` function will only return after all input has been processed by nvim.
It is useful to time assertions correctly.
test: Fix race condition in window_spec.lua
Without waiting for the 'gg' command to be processed, its possible that the
following assertion will fail.
test: Remove unnecessary tests from viml_system_spec.lua
The `system` function is never executed with these tests because the ctrl+c is
queued with the input string that calls it(The `process_interrupts` function
will destroy all previous input when a ctrl+c is found).
test: Increase default_screen_timeout when running on travis
Some screen tests such as system/ctrl+c(viml_system_spec.lua) can take some time
to respond(default kill timeout is 2 seconds for an interrupted job) and fail
when running under a slow environment such as travis.
input: Prefer keycodes in input_enqueue key translation
This is required to correctly handle certain keys such as <delete>
input: Fix handle_mouse_event function
Ignore all keys that aren't prefixed with KS_EXTRA.
input: Remove input_buffer_{save,restore}
The input buffer is only used for data that really came from another process and
is only visible to os/input.c. Remove the input_buffer_{save,restore} functions,
they are not necessary(Also can result in problems if data comes while the
typeahead is saved).
ui: Remove/adapt some old code for a big UI refactor
- Remove abstract_ui global, now it is always active
- Remove some terminal handling code
- Remove unused functions
- Remove HAVE_TGETENT/TERMINFO/TERMIOS/IOCTL #ifdefs
- Remove tgetent/terminfo from version.c
- Remove curses/terminfo dependencies
- Only start/stop termcap when starting/exiting the program
- msg_use_printf will return true if there are no attached UIs(
  messages will be written to stdout)
- Remove `ex_winpos`(implement `:winpos` with `ex_ni`)
ui: Rewrite the builtin terminal UI
Now all terminal-handling code was moved to src/nvim/tui, which implements a
new terminal UI based on libtermkey and unibilium
syntax: Refactor to store all term and gui attributes independently
Now the attrentry_T structure will store all attributes in separate fields for
cterm and rgb UIs.
test: Fix 051_highlight_spec.lua
- The syntax `gui=` is invalid when setting properties of highlight group.
- Wait for the initial "-- More --" prompt before continuing. Required to avoid
  a race condition

@tarruda tarruda changed the title from [RFC] Reimplement builtin terminal UI using libtermkey/unibilium to [RFC] Reimplement builtin terminal UI with termkey/unibilium Feb 17, 2015

@tarruda tarruda merged commit 40b7990 into neovim:master Feb 17, 2015

1 check was pending

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details

@jszakmeister jszakmeister removed the RFC label Feb 17, 2015

@bigeagle

This comment has been minimized.

Show comment
Hide comment
@bigeagle

bigeagle Feb 17, 2015

Hi, using the merged version, there're still a problem that the terminal's background color cannot be automatically "resetted". To repeat this bug, nvim -u NONE and try to show some error message, e.g. :q on an unsaved file and there would be an error message with red bg color, now press : and the red background is still there.
I'm using xterm-256color.

bigeagle commented Feb 17, 2015

Hi, using the merged version, there're still a problem that the terminal's background color cannot be automatically "resetted". To repeat this bug, nvim -u NONE and try to show some error message, e.g. :q on an unsaved file and there would be an error message with red bg color, now press : and the red background is still there.
I'm using xterm-256color.

@neovim neovim locked and limited conversation to collaborators Feb 17, 2015

@justinmk

This comment has been minimized.

Show comment
Hide comment
@justinmk

justinmk Feb 17, 2015

Member

Please report in a new issue.

Member

justinmk commented Feb 17, 2015

Please report in a new issue.

@elmart

This comment has been minimized.

Show comment
Hide comment
@elmart

elmart Feb 18, 2015

Member

as soon as you identify a pattern for reproducing the YCM slowdown, I will investigate and hopefully come up with a fix soon

Ok.

Really great work, @tarruda. Thumbs up! 👍

Member

elmart commented Feb 18, 2015

as soon as you identify a pattern for reproducing the YCM slowdown, I will investigate and hopefully come up with a fix soon

Ok.

Really great work, @tarruda. Thumbs up! 👍

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.