README: a section on an educated estimation on performance implication for large files

[novoid]

Hi,

I've got pretty large Org files and as far as I understood, things like overlays do cause severe performance degradation in large Org files.

It would be nice to read about performance implications in the readme so that people using large Org files know that this is a show-stopper for this nice Org extension or not.

Thanks!

[nobiot]

Thank you for the comment. Actually, that's probably where I could use advice and experience from someone like you. I don't have a large org file, and unlikely I will ever do. I have never had any performance issues with Org mode in my 6 years or so of using it.

My motivation for me to write this library is to write a non-fiction book. With it, I should be able to assemble elements of multiple files (org files are one type of them) into a cohesive whole. The resultant file will for me is yet only a part of a book (at the moment, a chapter. One chapter is an Org file). I don't see these chapter Org files will become "a large enough" to cause performance degradation.

I would assume you can use a large org file as a source to pull pars from. But since you are really not editing a large file (you are only visiting it to copy a part of it), I don't know if it leads to a performance issue.

With this set up, where do you see a potential cause of performance degradation?

If I were to make any logical guess (not based on experience as I have none), I would not be technically qualified to make any comment on performance in relation to overlays and Org mode (I am only a dilettante in Elisp).

[novoid]

Hi,
It seems to be the case that we're both not qualified to judge performance issues here. The only things I know about performance degradation is when I suffer from it and somebody on GitHub or reddit helps me to narrow it down. Most of the times, the usage of overlays were causing performance issues.

As far as I understood: when you do have many overlays in deeply nested headings that are collapsed and you scroll over the lines with the collapsed headings, Emacs needs to handle those overlays and therefore is not able to perform a simple line change without issues.

You may close this issue any time when we both are not qualified to answer it.

[nobiot]

@novoid Thank you for your response.
This is useful; I will keep it in mind.

I took it that:

1. You are visiting a large org file
2. You have folded a large subtree with "deeply nested headings" -- probably a large number of headlines, each of which is also folded.
3. You are moving your cursor over the folded large subtree

|
* Deeply nested, folded large subtree...

I am moving the cursor "|" to cross this headline

At the moment, Org-transclusion is at a very early stage, and it needs to unfold the whole folded subtree in most operations -- it cannot work while keeping a subtrees collapsed. From this perspective, at the moment, I do not think Org-transclusion is really good fit when you have a large Org file (I believe you can still use such a file as a source).

Thank you again for your feedback. I will keep this performance issue in mind for future feature development.