Geany really slow when opening huge text files

[oscardssmith]

I have a 400mb text file (neural network weights, it's horrible I know) and whenever I open it with geany, geany basically just freezes. How hard would it be to only load the bits neaded to fill a screen (or 100) at a time instead of the whole file (assuming that's what geany is doing).

I am on ubuntu 18.04 with geany 1.32.

[elextr]

Things that are done on a whole file at load time:

1. if encoding isn't specified by the open dialog, possibly multiple passes through the file will take place to attempt to find the encoding.

2. if the file has a filetype then the editing widget will run the lexer over the file to generate highlighting

3. if the filetype has a symbol parser that will run on the entire file

But if the encoding is specified and "None" and the filetype is specified as "None" both in the "More Options" section of the Open dialog then those actions should not occur. Hopefully that will speed things up.

[oscardssmith]

would it be possible to have highlighting and parsing turn off above a specific size? Also if figuring out the filetype will take more than 5 seconds, I feel like it might be worth prompting the user, as if they know, then they will save time.

[elextr]

These things have been discussed before, and the conclusion is, no, there is no reasonable definition of "big", it totally depends on your computer speed, and also on whether you actually want those things I listed above to happen. Geany is used on Raspberry Pis and workstations with high end GPUs, "big" and "fast" is not comparable between those so there is no way to decide how long something will take.

Geany is a programming IDE, it is designed to edit reasonable sized (for humans) program language files, thats its primary use-case. It is capable of editing other files (large logs are a common example) but they are not the core focus, so the defaults are set for the core use-case and options to turn off encoding and filetypes are available for other cases if those features are not required.

Having Geany try to be "smart" in the way that you suggested will only result in it being more annoying to some part of the primary use-case population, and thats a negative.

[elextr]

PS did turning the filetype and encoding off fix the problem?

[oscardssmith]

I just switched to using less for the file. Would it be possible to have a big-file size in the various settings page? That would let it default off, but for users who want this somewhat frequently, they could set it to 1g to mean everything over 1g, turn off anything that requires storing the whole thing in memory.

[elextr]

We can stop multiple passes through files while they are opened to reduce loading delays.

But open files will always be loaded into memory, thats the way the editing widget Geany uses works, it is a gap buffer. And since the editing widget also allocates a style byte for each byte in the file, you actually use twice the memory, plus per line data.

[oscardssmith]

ok not sure much can be done that will make this work for my use case