Add support for thread folding in mu4e-headers

[zakkak]

Implementation for rfe #75
Add ability to fold/unfold threads.

[zakkak]

I use it with

(define-key 'mu4e-headers-mode-map (kbd "TAB") 'mu4e-headers-toggle-thread-folding)

TAB folds the current thread (even if point is not in the root message).
C-u TAB folds the subthread, i.e., all messages bellow the message at point.
TAB on a folded thread/subthread unfolds it.

Works with nested foldings as well, e.g., folding a subthread and then the whole thread.

[zakkak]

I use it with

(define-key 'mu4e-headers-mode-map (kbd "TAB") 'mu4e-headers-toggle-thread-folding)

TAB folds the current thread (even if point is not in the root message).
C-u TAB folds the subthread, i.e., all messages bellow the message at point.
TAB on a folded thread/subthread unfolds it.

Works with nested foldings as well, e.g., folding a subthread and then the whole thread.

[djcb]

Oh, nice work! Some minor issue - seems tabbing takes the point to the next line; would be nice if type tab twice would return to the starting situation.

Another thing -- would it be hard to allow for buffer-wide operation, ie. collapse / uncollapse all threads? Perhaps with mu4e-headers-for-each?

[djcb]

Oh, nice work! Some minor issue - seems tabbing takes the point to the next line; would be nice if type tab twice would return to the starting situation.

Another thing -- would it be hard to allow for buffer-wide operation, ie. collapse / uncollapse all threads? Perhaps with mu4e-headers-for-each?

[zakkak]

seems tabbing takes the point to the next line

Yes, that was on purpose. I was thinking that since one wants to fold a thread probably he is not interested in it and one would like to move to the next thread. I was inspired by the way we mark threads.

would be nice if type tab twice would return to the starting situation.

That makes sense as well.

Another thing -- would it be hard to allow for buffer-wide operation, ie. collapse / uncollapse all threads? Perhaps with mu4e-headers-for-each?

If it is guaranteed that all messages belonging to the same thread are traversed one after another with mu4e-headers-for-each, then it should be easy.

Actually, this PR should be the start for more advanced features. For instance:

• Start view with all threads folded
• Add the number of unread messages in the string replacing the overlay (currently set to "...").
• Mark folded threads with the face (and maybe flags) of any unread messages. This way one can see which folded threads have unread messages.
• "Arbitrary folding along the lines of gmail where previously read messages in a thread are folded automatically" as per @seanfarley.

The last two seem a long shot, but if you want I could implement the first two in this PR, or create new ones after this one gets merged upstream.

NOTE: I have found a small issue. If you press the down arrow instead of 'n' to get to a message that is not threaded (has no replies) and press tab then the cursor moves to the end of the previous line, but the correct line stays highlighted. I don't know why this happens, if you have any clue, please enlighten me :)

[zakkak]

seems tabbing takes the point to the next line

Yes, that was on purpose. I was thinking that since one wants to fold a thread probably he is not interested in it and one would like to move to the next thread. I was inspired by the way we mark threads.

would be nice if type tab twice would return to the starting situation.

That makes sense as well.

Another thing -- would it be hard to allow for buffer-wide operation, ie. collapse / uncollapse all threads? Perhaps with mu4e-headers-for-each?

If it is guaranteed that all messages belonging to the same thread are traversed one after another with mu4e-headers-for-each, then it should be easy.

Actually, this PR should be the start for more advanced features. For instance:

• Start view with all threads folded
• Add the number of unread messages in the string replacing the overlay (currently set to "...").
• Mark folded threads with the face (and maybe flags) of any unread messages. This way one can see which folded threads have unread messages.
• "Arbitrary folding along the lines of gmail where previously read messages in a thread are folded automatically" as per @seanfarley.

The last two seem a long shot, but if you want I could implement the first two in this PR, or create new ones after this one gets merged upstream.

NOTE: I have found a small issue. If you press the down arrow instead of 'n' to get to a message that is not threaded (has no replies) and press tab then the cursor moves to the end of the previous line, but the correct line stays highlighted. I don't know why this happens, if you have any clue, please enlighten me :)

[djcb]

Cool!

For the first item -- I think I'd prefer to get us to back
to starting position, just like e.g. org-mode does. We could have a
different function (perhaps bound to ) do the tab-and-next,
which can be useful as well. What do you think?

For the other items - indeed, mu4e-headers-for-each works in
headers-order, so that should work fine. Perhaps easiest for now would
be to do any collapsing into some kind of hook after all headers are
listed.

And agreed, let's look at the more complicated ones after the simpler ones are
there.

Anyway, with the tab-change, I think we can take the PR, and then work on
the rest. Nice things!

[djcb]

Cool!

For the first item -- I think I'd prefer to get us to back
to starting position, just like e.g. org-mode does. We could have a
different function (perhaps bound to ) do the tab-and-next,
which can be useful as well. What do you think?

For the other items - indeed, mu4e-headers-for-each works in
headers-order, so that should work fine. Perhaps easiest for now would
be to do any collapsing into some kind of hook after all headers are
listed.

And agreed, let's look at the more complicated ones after the simpler ones are
there.

Anyway, with the tab-change, I think we can take the PR, and then work on
the rest. Nice things!

[djcb]

Oh, and I'll take a look at that small issue mentioned.

[djcb]

Oh, and I'll take a look at that small issue mentioned.

[zakkak]

The issue seems to disappear when I use save-excursion, which I will from now on. As a result you can ignore it, it won't exist in the next push ;)

[zakkak]

The issue seems to disappear when I use save-excursion, which I will from now on. As a result you can ignore it, it won't exist in the next push ;)

[zakkak]

The function now takes two arguments, one for subthread folding and another for moving to the next thread after folding.

I also implemented the message counting. When folded, it will print +N where N is the number of messages in the folded thread/subthread. It also counts unread messages and displays them next to the total count like +N(M).

Ignore my previous comment about the issue, it looks like I can't always rely on save-excursion. The problem is with folding a thread while in it (not at its first message). In this case I have to move the cursor to the first message in the thread. So the glitch is still here, it is weird though that it only appears when moving down with the arrow key and then folding while on the first message of a thread.

I also had to play around with the position of the cursor for the overlays. Leaving it to the end-of-line results in the point appearing next to the display string, so I move it by -1. The above misbehavior might also relate to this.

Another bug/feature I observe is that when folding nested folds the display results in something like +7(3) +4(1) +2(1) where each +N(M) is the number of messages (and unread messages) in each folding level, from outermost to innermost.

[zakkak]

The function now takes two arguments, one for subthread folding and another for moving to the next thread after folding.

I also implemented the message counting. When folded, it will print +N where N is the number of messages in the folded thread/subthread. It also counts unread messages and displays them next to the total count like +N(M).

Ignore my previous comment about the issue, it looks like I can't always rely on save-excursion. The problem is with folding a thread while in it (not at its first message). In this case I have to move the cursor to the first message in the thread. So the glitch is still here, it is weird though that it only appears when moving down with the arrow key and then folding while on the first message of a thread.

I also had to play around with the position of the cursor for the overlays. Leaving it to the end-of-line results in the point appearing next to the display string, so I move it by -1. The above misbehavior might also relate to this.

Another bug/feature I observe is that when folding nested folds the display results in something like +7(3) +4(1) +2(1) where each +N(M) is the number of messages (and unread messages) in each folding level, from outermost to innermost.

[zakkak]

We should probably make the display string customizable at some point.

[zakkak]

We should probably make the display string customizable at some point.

[djcb]

Cool stuff, @zakkak! If you can fix those little things (hopefully easy to fix!), I'll merge this. I think it would be quite an improvement for mu4e!

[djcb]

Cool stuff, @zakkak! If you can fix those little things (hopefully easy to fix!), I'll merge this. I think it would be quite an improvement for mu4e!

[zakkak]

I think I will need some help to fix them. I am not very familiar with moving the point around.

Does mu4e introduce any invisible stuff at the beginning or the end of each line in the mu4e-headers view?

[zakkak]

I think I will need some help to fix them. I am not very familiar with moving the point around.

Does mu4e introduce any invisible stuff at the beginning or the end of each line in the mu4e-headers view?

[djcb]

Okay, I'll play around with it. And yes, there's a marker at the beginning of lines; see mu4e~headers-docid-pre

[djcb]

Okay, I'll play around with it. And yes, there's a marker at the beginning of lines; see mu4e~headers-docid-pre

[djcb]

I've pushed a zakkak/thread-folding branch here. Let's see if I have some hacking time this weekend...

As for #' vs ' - they are not the same, but I don't the difference matters much. But let's use the latter, it's simpler.

[djcb]

I've pushed a zakkak/thread-folding branch here. Let's see if I have some hacking time this weekend...

As for #' vs ' - they are not the same, but I don't the difference matters much. But let's use the latter, it's simpler.

[andersjohansson]

Regarding #' and ':

Use a sharp quote (#') when quoting function names. It's a good hint for the byte-compiler, which will warn you if the function is undefined. Some macros can also behave differently otherwise (like cl-labels).

https://github.com/bbatsov/emacs-lisp-style-guide

Don't ask me about what difference it really makes though...

[andersjohansson]

Regarding #' and ':

Use a sharp quote (#') when quoting function names. It's a good hint for the byte-compiler, which will warn you if the function is undefined. Some macros can also behave differently otherwise (like cl-labels).

https://github.com/bbatsov/emacs-lisp-style-guide

Don't ask me about what difference it really makes though...

[hexa00]

Hi,
I've been using this branch for a few months and really like it :)

However I really wish I could fold/unfold all threads, has anyone figured this out already ?

I tried the naive way calling like :

(defun mu4e-headers-fold-all ()
"Folds all threads"
(interactive)
(mu4e-headers-for-each 'mu4e-headers-toggle-thread-folding))

But that was too naive it seems...

Thanks,

[hexa00]

Hi,
I've been using this branch for a few months and really like it :)

However I really wish I could fold/unfold all threads, has anyone figured this out already ?

I tried the naive way calling like :

(defun mu4e-headers-fold-all ()
"Folds all threads"
(interactive)
(mu4e-headers-for-each 'mu4e-headers-toggle-thread-folding))

But that was too naive it seems...

Thanks,

[dakrone]

What's the word on this, is it something that's likely to be merged into mu4e, or something users should use a custom build to add support for?

[dakrone]

What's the word on this, is it something that's likely to be merged into mu4e, or something users should use a custom build to add support for?

[zakkak]

It is still far from perfect. Since I am pretty busy the last few months I don't know when and if I will manage to get it in a good shape to be merged upstream.

Any contributions are welcome.

[zakkak]

It is still far from perfect. Since I am pretty busy the last few months I don't know when and if I will manage to get it in a good shape to be merged upstream.

Any contributions are welcome.

[zakkak]

Bump. I finally got some time to take a look at the code and fix a few things.

So, folding seems to work better now.

Known issues

Marking folded threads:

1. applies mark only to visible message (maybe this is OK as is) and
2. breaks visualization by hiding the message and showing only the overlay display (not OK).

[zakkak]

Bump. I finally got some time to take a look at the code and fix a few things.

So, folding seems to work better now.

Known issues

Marking folded threads:

1. applies mark only to visible message (maybe this is OK as is) and
2. breaks visualization by hiding the message and showing only the overlay display (not OK).

[hexa00]

Hi,
Thanks for working on this !

I just updated and I notice this small issue when folding I get:
Subject +8(8)h

Notice the 'h" , this can be any char, most likely tied to some text in the area....

Also, not sure I understand known issue #2.. ?

[hexa00]

Hi,
Thanks for working on this !

I just updated and I notice this small issue when folding I get:
Subject +8(8)h

Notice the 'h" , this can be any char, most likely tied to some text in the area....

Also, not sure I understand known issue #2.. ?

[zakkak]

Hi,

I can't reproduce the Subject +8(8)h issue.

Regarding the second Known Issue, it refers to applying some mark at a folded thread. After applying the mark the whole line disappears for me and I am only left with the counts, e.g., Date | From | Subject +8(8) becomes +8(8).

[zakkak]

Hi,

I can't reproduce the Subject +8(8)h issue.

Regarding the second Known Issue, it refers to applying some mark at a folded thread. After applying the mark the whole line disappears for me and I am only left with the counts, e.g., Date | From | Subject +8(8) becomes +8(8).

[hexa00]

You mean like doing on a folded thread ?

That works fine here, only issue is that the cursor goes to the end of the line...

[hexa00]

You mean like doing on a folded thread ?

That works fine here, only issue is that the cursor goes to the end of the line...

[hexa00]

Oops I mean doing "insert" on a folded thread,, I used '<' and '>' and github just removed the tag...

[hexa00]

Oops I mean doing "insert" on a folded thread,, I used '<' and '>' and github just removed the tag...

[zakkak]

Hmmm, <insert> seems to not trigger the issue, try marking a folded thread as read and apply the action.

[zakkak]

Hmmm, <insert> seems to not trigger the issue, try marking a folded thread as read and apply the action.

[zakkak]

More accurately if you choose to apply a mark after <insert> the problem appears again.

[zakkak]

More accurately if you choose to apply a mark after <insert> the problem appears again.

[hexa00]

OK I see, I can reproduce now, thanks for the explaination. (Note that I get +8(8)h )

On another note the only big missing features for me would be to:

• Have some way to fold all threads.
• Keep the folded thread folded when mu refreshes...

Have you considered those ?

Thanks,

[hexa00]

OK I see, I can reproduce now, thanks for the explaination. (Note that I get +8(8)h )

On another note the only big missing features for me would be to:

• Have some way to fold all threads.
• Keep the folded thread folded when mu refreshes...

Have you considered those ?

Thanks,

[zakkak]

Well those make sense and would be great to have.
One step at a time :)

[zakkak]

Well those make sense and would be great to have.
One step at a time :)

[hexa00]

hehe OK :) Just making sure I haven't missed it.

Thanks again!

[hexa00]

hehe OK :) Just making sure I haven't missed it.

Thanks again!

[zakkak]

@djcb could you please have a look in to this and let me know of any comments/directions?

[zakkak]

@djcb could you please have a look in to this and let me know of any comments/directions?

[djcb]

He @zakkak, nice to see this! I like the functionality; the implementation is a bit complicated (probably necessarily so). Anyway, are you happy with how it works? Ie., does it all work as expected?

[djcb]

He @zakkak, nice to see this! I like the functionality; the implementation is a bit complicated (probably necessarily so). Anyway, are you happy with how it works? Ie., does it all work as expected?

[zakkak]

Other than not supporting marking folded threads (#783 (comment)), I am pretty happy about it and use it on a daily base.

[zakkak]

Other than not supporting marking folded threads (#783 (comment)), I am pretty happy about it and use it on a daily base.

[felipeochoa]

I've refactored this a bit to split out the complexity into different functions[1]. I've also added a mu4e-headers-fold-all that I like adding to mu4e-headers-found-hook. There's also now an option to override the "folded slug" formatting under mu4e-headers-folding-slug-function.

There are a couple of small bugs in the folding:

1. Pressing n on a header with a folded thread moves point to EOL instead of down to the next header. I think the fix is in mu4e~headers-move, where forward-line needs to become forward-visible-line, but I'm not sure.
2. Marking a header while point is on the overlay breaks the folding. There are two solutions here:
   • The quick one: This happens because mu4e-mark-at-point calls an unqualified remove-overlays, splitting the folding overlay. If instead these overlays defined some property like mu4e-marking-overlay, the call to remove-overlays could filter these out and avoid this problem.
   • The complete one: We should probably keep a hash of folded messages. That way we can forbid marking of folded headers with a check in mu4e-mark-at-point or mu4e~headers-mark. This hash could also be used re-instate folding after an index update

Sorry I don't have time to make this into a PR and fix these bugs, but hopefully this pushes the ball forward a bit.

[1] https://gist.github.com/felipeochoa/614308ac9d2c671a5830eb7847985202

[felipeochoa]

I've refactored this a bit to split out the complexity into different functions[1]. I've also added a mu4e-headers-fold-all that I like adding to mu4e-headers-found-hook. There's also now an option to override the "folded slug" formatting under mu4e-headers-folding-slug-function.

There are a couple of small bugs in the folding:

1. Pressing n on a header with a folded thread moves point to EOL instead of down to the next header. I think the fix is in mu4e~headers-move, where forward-line needs to become forward-visible-line, but I'm not sure.
2. Marking a header while point is on the overlay breaks the folding. There are two solutions here:
   • The quick one: This happens because mu4e-mark-at-point calls an unqualified remove-overlays, splitting the folding overlay. If instead these overlays defined some property like mu4e-marking-overlay, the call to remove-overlays could filter these out and avoid this problem.
   • The complete one: We should probably keep a hash of folded messages. That way we can forbid marking of folded headers with a check in mu4e-mark-at-point or mu4e~headers-mark. This hash could also be used re-instate folding after an index update

Sorry I don't have time to make this into a PR and fix these bugs, but hopefully this pushes the ball forward a bit.

[1] https://gist.github.com/felipeochoa/614308ac9d2c671a5830eb7847985202

[zakkak]

It sure does. Thanks for your contributions.

I will try to get your changes on my branch, but it would be better if you could create a PR.
In the meantime if anyone else is willing to do the PR, it will be much appreciated.

[zakkak]

It sure does. Thanks for your contributions.

I will try to get your changes on my branch, but it would be better if you could create a PR.
In the meantime if anyone else is willing to do the PR, it will be much appreciated.

[sebhahn]

Is this feature merged anytime soon?

[sebhahn]

Is this feature merged anytime soon?

[zakkak]

Probably not, it is not stable yet.

Unfortunately I don't have much time to work on it so if anyone is willing to take it over, it would be great.

[zakkak]

Probably not, it is not stable yet.

Unfortunately I don't have much time to work on it so if anyone is willing to take it over, it would be great.

[stefanv]

@zakkak Can you expand a bit on what needs to be done?

[stefanv]

@zakkak Can you expand a bit on what needs to be done?

[zakkak]

There are the issues mentioned here regarding marking and applying actions on folded threads.
There is also this patch that needs to be merged and it also lists some more issues.

[zakkak]

There are the issues mentioned here regarding marking and applying actions on folded threads.
There is also this patch that needs to be merged and it also lists some more issues.

[titaniumbones]

I'm not in a position to track down and fix these bugs but just want to add a word of encouragement to whomever might be able to od it -- I've just loaded @felipeochoa 's gist version and while I'm sure at some point the bugs that @zakkak mentions will become frustrating, already I notice a dramatic improvement in usage of mu4e. I would love to see this merged to master so everyone can take advantage of it.

[titaniumbones]

I'm not in a position to track down and fix these bugs but just want to add a word of encouragement to whomever might be able to od it -- I've just loaded @felipeochoa 's gist version and while I'm sure at some point the bugs that @zakkak mentions will become frustrating, already I notice a dramatic improvement in usage of mu4e. I would love to see this merged to master so everyone can take advantage of it.