Display ascii art with ansi colors.

[uttarayan21]

This is a optional feature that will add option to use jp2a as the ascii backend.
This will add auto color support as well as a way to use jpg/png images for the ascii art.
This will add color support to ascii images made using jp2a or similar tools.

This is an initial draft, will need more work.

Screenshot

[grtcdr]

This is a good feature, but one thing to consider is the speed at which the file is converted from a bitmap to characters we can print, so how's the performance looking?

Another major drawback is having to bring termwiz as a dependency, which in and of itself, brings an absurd amount of dependencies.

I think a major struggle for Macchina and libmacchina right now is the annoying build time that can probably be reduced if we didn't use as many dependencies as we currently do.

After merging the custom-ascii PR (#78), I do not see this as a necessity since the user can import their own ASCII art after having converted it from image to text using jp2a or any other program, we also already provide them with basic color customization and I think a key feature is to extend that customization with ANSI color code parsing.

[uttarayan21]

Performance definitely takes a hit.
Currently it runs at ~120ms.
However only running jp2a takes around ~3ms. So the performance can definitely be improved.

[uttarayan21]

Another major drawback is having to bring termwiz as a dependency, which in and of itself, brings an absurd amount of dependencies.

Yes It's definitely a drawback.
This draft is only a proof of concept.
I'm using this crate ansi4tui which is what brings most dependencies.
I'll try writing my own parser since this seems to be the main bottleneck.

[grtcdr]

I'll try writing my own parser since this seems to be the main bottleneck.

That would be a lovely contribution 😄

[uttarayan21]

So I tried using the ansi_parser crate but It adds 15-20ms of latency just to parse the ascii with colors and doesn't even convert colors to any higher type so I'm writing a parser from scratch.

It should have 4-bit / 8-bit / TrueColor support. It's probably going to take a while since I'm going in blind.
I found a extensive list of Ansi Color codes here for future reference.

[grtcdr]

So I tried using the ansi_parser crate but It adds 15-20ms of latency just to parse the ascii with colors and doesn't even convert colors to any higher type so I'm writing a parser from scratch.

That stucks!

It should have 4-bit / 8-bit / TrueColor support. It's probably going to take a while since I'm going in blind.
I found a extensive list of Ansi Color codes here for future reference.

Make a separate crate for this, as other people might find it super helpful too :)

[uttarayan21]

Okay so I have completed a draft of the parser.
Tested with True color ( rgb) output from jp2a.
Tested with 4 - bit colors.
I haven't tested on windows/mac yet but it should be compatible.
I haven't tested 8-bit colors but they should also be compatible.

Here is the parser ansi-to-tui

Benchmarks
old benchmarks

[grtcdr]

Absolutely terrific!

Benchmarks were almost untouched. So, is the file converting say {c1} to a certain ANSI value or a converting ANSI directly to tui::text::Text

[uttarayan21]

Currently its parsing a byte sequence like
\x1b[38;2;100;200;50m
-> set foreground color (38), with RGB (2), with colors r (100) g (200) b (50).
Then converting them to tui::style::Style and then converting each sequence of text between two format byte sequence as the span text.
And pushing them into a buffer (vector of spans).

[uttarayan21]

However this currently assumes that only there are only ansi colors and not any other ansi byte sequence.
If other types of ansi byte sequences are present it will cause undefined behavior.

However jp2a should always give ansi color only outputs.

Performance can probably be improved further with a bit of tweaking.

[grtcdr]

I was taking a look at the archlinux.ascii file and noticed that every character was being color-coded.

Is it currently not possible to color-code multiple characters at once?

[grtcdr]

@CuriouslyCurious might be interested in this lovely work you've made as he's done some research to try and make this possible, he might be able to give you a hand so you can maybe work on this together and deliver an all-round faster and better solution.

[uttarayan21]

I was taking a look at the archlinux.ascii file and noticed that every character was being color-coded.

Is it currently not possible to color-code multiple characters at once?

Yeah that is one problem I'm facing. jp2a outputs every character with its own styling. I was thinking of possibly improving it using the parser.

[grtcdr]

Yeah that is one problem I'm facing. jp2a outputs every character with its own styling. I was thinking of possibly improving it using the parser.

I'd love to help you on this but I've got a uni project that involves PHP (uugh..) I'm required to make.

[uttarayan21]

Yeah that is one problem I'm facing. jp2a outputs every character with its own styling. I was thinking of possibly improving it using the parser.

I'd love to help you on this but I've got a uni project that involves PHP (uugh..) I'm required to make.

Haha ! I also have a project going on for college, I was thinking of porting it over to rust from python.

[grtcdr]

If it's a small project I'd say go for it!

[uttarayan21]

If it's a small project I'd say go for it!

I would but then people working with me would have to learn rust and that's quite possibly the biggest challenge.

[grtcdr]

I would but then people working with me would have to learn rust and that's quite possibly the biggest challenge.

Oh, you're not solo, yeah it'd be hard to convince them, but give it a try! Maybe they like the things Rust brings to the table.

[CuriouslyCurious]

@CuriouslyCurious might be interested in this lovely work you've made as he's done some research to try and make this possible, he might be able to give you a hand so you can maybe work on this together and deliver an all-round faster and better solution.

Seems like @uttarayan21 has mostly got it under control from what I can see. A bit miffed that he was so fast, as I was making some slow but steady progress on the cansi crate 😛. His code isn't as simplistic as that crate, but it can surely be iterated upon as it seems to handle stuff correctly. I'll keep going with the cansi crate PR though as I think it may be useful for other projects in the future.

Rewriting jp2ain Rust seems simple enough, and then having that as a standalone (or even optional dependency) could be a neat solution.

Great job, @uttarayan21!

[uttarayan21]

Haha, thanks, @CuriouslyCurious

I am actually refactoring the code to make it simpler.
It was a late night endeavor so some stuff definitely doesn't make sense lol.

I, previously, hadn't heard of the cansi crate and figured it might be fun to write a parser from scratch.
There are a few caveats to mine though.

• It doesn't handle broken inputs well.
• No UTF-8 since currently I'm parsing byte by byte It doesn't support unicode text yet, it is ascii only.
• Currently all the bytes are being read to copied to memory before parsing. ( Maybe implement BufReader ?)
• No error handling as of yet but then again writing erroneous bytes to terminal also does nothing.

[uttarayan21]

Okay so I did a bit more work and now it supports UTF-8.
Also bad input should be automatically ignored.

Also utf-8 parsing can be done using String::from_utf8 or simdutf8 which uses simd to speed up parsing.
It can be enabled using the feature flag --features simd.

But since most of the UTF-8 sequences are smaller than 64 ( usually 1 with jp2a ) the std String::from_utf8 should be faster as mentioned in the readme of simdutf8

[grtcdr]

Okay so I did a bit more work and now it supports UTF-8.
Also bad input should be automatically ignored.

Beautiful stuff, you're getting there! :)

[uttarayan21]

Beautiful stuff, you're getting there! :)

Thanks, 😃
I think its done / I can't think of anything else to do.

Examples:

Archlinux ascii

Archlinux UTF-8

Rust ascii

Benchmarks are same as last.

[grtcdr]

I think its done / I can't think of anything else to do.

I think that at the moment, it's not very intuitive, and its very repetitive. We need to make it so users can use {cX} to configure the color of the ASCII instead of the actual ANSI values, and we also need to make it so the parser can apply the colors for not just one character, but multiple characters and entire lines.

This will most likely improve performance too as we're converting entire lines in some cases instead of characters one by one.

[grtcdr]

It's just a script that downloads any video supported by youtube-dl -> ffmpeg outputs the frames at 10 fps -> each frame is converted to ascii by jp2a -> macchina displays the ascii frames as if in a video.

It'd be helpful to have that explanation in the script for new users so they know what the script does :)

Maybe I should rename it to macchina-video.sh

I think that's more fitting, yeah.

[uttarayan21]

It'd be helpful to have that explanation in the script for new users so they know what the script does :)

Okay I added a comment to the script.
Running the script without args will give a "surprise" else the first argument will be the video url and second argument is the time it should wait for a frame in seconds.

Also is there anything else I should clean up before I push ?

[uttarayan21]

Also as of now macchina panics if a file has more than 50 lines / 500 columns since that is the limit of the buffer.

[grtcdr]

I noticed you took out --custom-ascii-color, was it causing issues?

Also as of now macchina panics if a file has more than 50 lines / 500 columns since that is the limit of the buffer.

How about not displaying it if it exceeds 50 lines instead of panicking?

[uttarayan21]

I noticed you took out --custom-ascii-color, was it causing issues?.

Yeah since styling is entirely dependent on the ansi codes and the codes are converted to color in the ansi-to-tui crate so I'd have to make a separate flag If you'd want to keep --custom-ascii-color and --custom-ascii.
Maybe --custom-ascii-with-color for the ansi parsing ?

How about not displaying it if it exceeds 50 lines instead of panicking ?.

Then we'd have to run a line count on the text file before it is passed to ansi-to-tui since it converts all of the lines to tui::text::Text object.

[grtcdr]

Can you merge the latest changes from main? The macOS issue should be fixed :)

Yeah since styling is entirely dependent on the ansi codes and the codes are converted to color in the ansi-to-tui crate so I'd have to make a separate flag If you'd want to keep --custom-ascii-color and --custom-ascii.
Maybe --custom-ascii-with-color for the ansi parsing ?

Well, --custom-ascii-color was put into place because we didn't have an ANSI parser when @CuriouslyCurious implemented reading ASCII from a specific path, I was thinking we keep this in case the user would like to override the coloring applied in that particular file, but this would mean that the parser will have to remove all those special ANSI characters so it doesn't display them. Well it's not necessarily the parser that has to remove the color codes, as its the total opposite of what ansi-to-tui is supposed to do. So I'm thinking you help us implement this feature, if you're okay with that, as you're more knowledgeable in this particular area.

[uttarayan21]

The ansi codes can be pretty easily removed from the color by just using sed / awk, IMO which would be much more efficient than removing them at runtime every single run.

Also removing them is pretty easy in rust too, since they are 0 width unicode characters String::to_utf8 will simply just not render them so just converting each line to a tui::text::Spans will suffice.

So I'm thinking
if --custom-ascii is given without --custom-ascii-color then use the ansi-to-tui
else use the old method.

[grtcdr]

The ansi codes can be pretty easily removed from the color by just using sed / awk, IMO which would be much more efficient than removing them at runtime every single run.

That's true, but we're trying to cover the edge cases so our users don't have to.

Also removing them is pretty easy in rust too, since they are 0 width unicode characters String::to_utf8 will simply just not render them so just converting each line to a tui::text::Spans will suffice.

So pass the buffer to String::to_utf8 before creating Spans?

So I'm thinking
if --custom-ascii is given without --custom-ascii-color then use the ansi-to-tui
else use the old method.

Exactly 😄

[grtcdr]

Compiling libmacchina v0.2.9

It's compiling the old version of libmacchina, libmacchina 0.3.0 should work fine.

[uttarayan21]

Compiling libmacchina v0.2.9

It's compiling the old version of libmacchina, libmacchina 0.3.0 should work fine.

Ah It didn't show up in cargo outdated so I didn't pay any attention to the version.

So pass the buffer to String::to_utf8 before creating Spans?

Yeah pretty much.

[grtcdr]

Ah It didn't show up in cargo outdated so I didn't pay any attention to the version.

Is outdated built into cargo or an external crate? Seems like something I should be using...

EDIT: Make sure to upgrade libmacchina to 0.3.1, which adds support for querying battery health (linux only though)

[uttarayan21]

Is outdated built into cargo or an external crate? Seems like something I should be using...

It's an external crate.
There are a lot of useful cargo extensions. The ones I use are cargo-tree cargo-outdated cargo-bloat cargo-edit cargo-tree cargo-strip .

Also I'll write an implementation tomorrow, should be pretty easy.

EDIT: Make sure to upgrade libmacchina to 0.3.1, which adds support for querying battery health (linux only though)

Oh yeah sure.

[grtcdr]

Is outdated built into cargo or an external crate? Seems like something I should be using...

It's an external crate.
There are a lot of useful cargo extensions. The ones I use are cargo-tree cargo-outdated cargo-bloat cargo-edit cargo-tree cargo-strip .

Well, thanks a lot for these, gonna check each one out :)

Also I'll write an implementation tomorrow, should be pretty easy.

Awesome, we should be set after that and ready to merge, thanks! 🎊

[uttarayan21]

It was somewhat harder than I thought. I ended up implementing a new function in ansi_to_tui called ansi_to_text_override_color.
I'll push a release after some testing.

Screenshots.

[grtcdr]

It was somewhat harder than I thought.

What obstacles did you encounter?

I'll push a release after some testing.

Awesome! I can't wait to have my hands on this :)

[uttarayan21]

What obstacles did you encounter?

I totally forgot about the fact that the escape sequences are not all zero width unicode so there are characters like [ 0..9 m , etc. that also need to be removed.
However it was no where near the complexity of the actual parser.

Also the fact that I've been forcing myself to switch from vim to emacs didn't help at all.

[grtcdr]

I totally forgot about the fact that the escape sequences are not all zero width unicode so there are characters like [ 0..9 m , etc. that also need to be removed.
However it was no where near the complexity of the actual parser.

Ah, okay. When you first mentioned their width always being zero, I had my doubts. But seeing as I've never ventured too far into ANSI or made a parser, I took your word for it. Glad you figured it out :)

Also the fact that I've been forcing myself to switch from vim to emacs didn't help at all.

I tried to do this twice in the past, and I just couldn't...

[uttarayan21]

I tried to do this twice in the past, and I just couldn't...

Same here, this is my third time trying to switch.
I'm using evil mode and learning from scratch without any of doomemacs / spacemacs this time around.
Also a problem is I know next to nothing about lisp, so another thing to learn just to configure stuff.

[uttarayan21]

@grtcdr can you check ? I think its done.

[grtcdr]

@grtcdr can you check ? I think its done.

Everything looks good!

And it seems like performance wasn't affected.

Thanks for your contributions @uttarayan21 ❤️ , it was a pleasure seeing custom ASCII being implemented using a proper parser, I appreciate your efforts and I hope you stick around with us 🥳

[uttarayan21]

Haha it was definitely fun working on it. I'll definitely stick around for a while :)

…

On 02/05/21 01:23PM, Taha Aziz Ben Ali wrote: @grtcdr can you check ? I think its done. Everything looks good! And it seems like performance wasn't affected. Thanks for your contributions @uttarayan21 ❤️ , it was a pleasure seeing custom ASCII being implemented using a proper parser, I appreciate your efforts and I hope you stick around with us 🥳 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.*

-- Uttarayan Mondal