bat freezes with extreme long lines (compact JSON)

[dns2utf8]

Hi all

I noticed that opening long json files makes bat get stuck.
For example the unpacked version of this file:
data.tar.gz

What did you expect to happen instead?

Maybe a warning or no highlighting.

How did you install bat?

cargo install -f bat

---

bat version and environment

Software version

bat 0.18.1 ()

Operating system

Linux 5.12.2-arch1-1

Command-line

bat --diagnostic 

Environment variables

SHELL=/bin/bash
PAGER=<not set>
LESS=<not set>
BAT_PAGER=<not set>
BAT_CACHE_PATH=<not set>
BAT_CONFIG_PATH=<not set>
BAT_OPTS=<not set>
BAT_STYLE=<not set>
BAT_TABS=<not set>
BAT_THEME=<not set>
XDG_CONFIG_HOME=<not set>
XDG_CACHE_HOME=<not set>
COLORTERM=truecolor
NO_COLOR=<not set>
MANPAGER=<not set>

Config file

Could not read contents of '/home/vp/.config/bat/config': No such file or directory (os error 2).

Compile time information

• Profile: release
• Target triple: x86_64-unknown-linux-gnu
• Family: unix
• OS: linux
• Architecture: x86_64
• Pointer width: 64
• Endian: little
• CPU features: fxsr,sse,sse2
• Host: x86_64-unknown-linux-gnu

Less version

> less --version 
less 581.2 (PCRE2 regular expressions)
Copyright (C) 1984-2021  Mark Nudelman

less comes with NO WARRANTY, to the extent permitted by law.
For information about the terms of redistribution,
see the file named README in the less distribution.
Home page: https://greenwoodsoftware.com/less

[sharkdp]

Thank you for reporting this.

I would guess that it doesn't really freeze. It just takes a really long time. Could you please confirm that? Maybe run it with paging disabled (--paging=never)

[dns2utf8]

Hello

Yes, technically with --paging=never it can open the file in 25 minutes.
Without it it takes almost 60 minutes ... I still think we should address it somehow.

Cheers,
Stefan

[sharkdp]

Thank you.

Is it a file that you can share? (by attaching it to this issue?)

bat uses Sublime syntax files for highlighting (via the syntect library). They use regular expressions to matching certain syntax constructs. And matching time of regular expressions can sometimes be excessive (catastrophic backtracking) if the patterns are not optimized.

Related tickets: #221 (also JSON), #750, #677

[keith-hall]

You could try building bat with the regex-fancy feature instead of regex-onig and see if that makes a difference:

bat/Cargo.toml

Line 26 in 71f04dc

"regex-onig",

[dns2utf8]

The file I used is this one, it is attached in the first part of the issue: https://github.com/sharkdp/bat/files/6510052/data.tar.gz

I did build it with this command:

cargo build --release --features application,regex-fancy

How do I run it with the regex-fancy now, I don't see any options that match?

[dns2utf8]

I opened the file again:

time target/release/bat --paging=never /tmp/data.json
...
real    40m22.297s
user    13m3.482s
sys     26m52.858s

so the current configuration is worse than what I get when installing with cargo install bat

[sharkdp]

Thank you. I added this ticket to the list of tasks in #1751.

[keith-hall]

Note that Sublime Text will simply not highlight lines which are longer than 16k characters, maybe we can do the same in bat...

[Enselic]

Closed by #2165