Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter ANSI escape sequences for terminal colors as a new term type #180

Closed
eldipa opened this issue Jun 25, 2021 · 2 comments · Fixed by #270
Closed

Filter ANSI escape sequences for terminal colors as a new term type #180

eldipa opened this issue Jun 25, 2021 · 2 comments · Fixed by #270
Labels
enhancement something nice to have but it is not neither critical nor urgent
Milestone

Comments

@eldipa
Copy link
Collaborator

eldipa commented Jun 25, 2021

Describe the feature you'd like
byexample uses pyte to interpret all the ANSI escape sequences coming from the interpreter. But pyte is not cheap and introduces some artifacts in the output like truncated lines.

Some interpreters however doesn't use the full set of possible escape sequences. In the happier case, they would use only the escape sequences to add some coloring to their output.

These sequences don't change the final layout so filter them will yield the same result except colorless. This is what pyte does and it could be implemented directly in byexample with a simple regex.

To be backward compatible the new filter-the-colors mode could be added as a new terminal type

if options['term'] == 'dumb':
out = self._emulate_dumb_terminal(self._output_between_prompts)
elif options['term'] == 'ansi':
out = self._emulate_ansi_terminal(self._output_between_prompts)
elif options['term'] == 'as-is':
out = self._emulate_as_is_terminal(self._output_between_prompts)
, as an intermediate level between dump and ansi.

We could make the change non-backward compatible and add the filtering in dump as it is unclear how many people could be really testing ANSI escape sequences in their examples.

@eldipa eldipa added enhancement something nice to have but it is not neither critical nor urgent good first issue issues that should be easy to get done labels Jun 25, 2021
@eldipa eldipa mentioned this issue Jun 25, 2021
@eldipa eldipa added this to the 11.0.0 milestone Jun 16, 2022
@eldipa eldipa removed the good first issue issues that should be easy to get done label Sep 12, 2022
@eldipa
Copy link
Collaborator Author

eldipa commented Sep 14, 2022

The use of a filter/terminal emulator by default or extending +dumb is not a good idea.

First, processing the escape sequences is required only in a few cases. Do the filtering by default will slow down all the executions with a minimal of benefits.

Moreover, the terminal emulator may change slightly the way the output is obtained and the whole thing does not pay the price of breaking examples.

@eldipa
Copy link
Collaborator Author

eldipa commented Oct 28, 2022

A way to speed up the things and get no negative side effect is to use a specialized termscrapper screen/stream to mostly do nothing except ignore the escape/control sequences.
Internally the screen will just store the feed data in a list without any concept of a 2D screen working indeed as a byte linear filter.
Virtually this does not add any overhead and it should be safe to enable it by default.

eldipa added a commit that referenced this issue Oct 28, 2022
We use a simplified WSPassthroughStream stream to handle any
non-trivial escape sequence (whitespace like newlines and tabs are
passed through).

Then, the stream will call to the event handlers of a simplified
LinearScreen.

This screen will not emulate a 2D screen but just a linear list
of data chunks received from the stream and it will emulate only the
most basic escape sequences and ignore the rest.

The combination of WSPassthroughStream and LinearScreen in processing
the data read from the interpreter *before* being processed by pexpect
makes possible the filtering without breaking the pexpect regex/prompt
matching, nor breaking pre-existent examples that depend on whitespace
nor adds any significant overhead to the runtime.

This filtering can be optionally disabled with `-filter-esc-seqs`. For
terminals other than `dumb`, no filtering is applied.

See #180
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement something nice to have but it is not neither critical nor urgent
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant