Add support for IMAP labels

[mrennings]

Plesse add support for IMAP Labels/flags/tags, just like Mozilla Thunderbird does. Thank you.

[PanderMusubi]

What needs to be done is:

• read IMAP labels and show these on a message and in folders
• create, edit and remove mapping of IMAP labels to color, order and more meaningful text
• adding and removing IMAP labels on a message

The order would be used to sort multiple IMAP labels (1st sort), fallback is to order the labels on meaningful text (2nd sort) and fallback for that is to sort them on their original value (3rd sort).

These labels can be more useful than the well known stars, especially when using https://en.wikipedia.org/wiki/Getting_Things_Done

The displaying of labels would use up header space of the message view, but it is up to the user to keep meaningful labels short and to add colors. In that way, only one line in the header would be needed. Displaying in folders could be done by color only, only challenge is what to do when multiple colors are added. Thunderbird gives priority to the most important label. For this order can be used. So if a message has labels "C" and "F" which are respectively mapped to "To Do", order=0, color=#EE88EE and to "Waiting", order=1, color=#FF8800, then on message view both label will be shown, first To Do in purple and then Waiting in orange. In folder view, the message will have purple text color in the subject.

Optionally, instead of giving the subject of a message in folder view a different color, the user might want the background color of that message in folder view to have a color. That would be similar as how it is offered in Thunderbird. Perhaps the user can choose either way.

See also #558 and https://code.google.com/p/k9mail/issues/detail?id=2532 and https://code.google.com/p/k9mail/issues/detail?id=4101

[igoralmeida]

JFYI,

read IMAP labels and show these on a message and in folders

#558 does that (in the ugliest way possible).

create, edit and remove mapping of IMAP labels to color,

Not there yet, but close. I lack the Android UI skills to make this usable, but the backend code is pretty much there already.

order and more meaningful text
adding and removing IMAP labels on a message

will still take some work after #558.

[PanderMusubi]

Looking forward to the result. You can ask for help at the UI guys

[PanderMusubi]

Added bounty for this one, see https://www.bountysource.com/issues/26376747-add-support-for-imap-labels Hope this helps.

[ccoenen]

I actually collected these for another issue, but I thought maybe the links are useful in one place:

• https://code.google.com/archive/p/k9mail/issues/2532 (2010! Seven years!)
• https://code.google.com/archive/p/k9mail/issues/4101 (lots of discussion)
• IMAP Keywords support. #146 (PR)
• IMAP Keywords (rebased) #558 (PR)
• Add support for IMAP labels #769 (this ticket :D )
• Ability to see IMAP keywords in message view #2260

And in one of these I found two bountys:

• https://www.bountysource.com/issues/26376747-add-support-for-imap-labels
• https://www.bountysource.com/issues/817243-feature-suggestion-use-imap-keywords-user-defined-flags-tags-labels

[cketti]

I believe this feature needs some design work before it is ready to be implemented.

Here's my list of unordered thoughts on this:

• While this might have been motivated by the IMAP feature to store arbitrary flags I don't see why we need to make this an IMAP-only feature. Other account types could have local-only tags.
• In my opinion tags should be account-specific. Imagine I have a personal account and a shared company account. I don't want the tags used in the company account to show up when working with my personal account and vice versa.
• When tags are supported by the app people want to search/filter for them and maybe even sort by tag.
• People might consider messages with (certain) tags important enough to want to keep them on the device even when they fall outside the sync window.
• For accounts that support server-side tags there needs to be a mechanism to map the apps tags to whatever the server is using.
• We need to figure out what tags to show to the user by default. For local-only tags it's only a question of coming up with a set of default tags. For IMAP we probably want to figure out what custom keywords the user is already using and then try to map that to a list of default tags and hide the rest.
• When an IMAP server doesn't support arbitrary keywords we need to support both local and synced tags.
• How should tags be displayed in the message list? I think changing the backgrounds adds way to much noise. Changing the text color is probably not much better. Maybe add some kind of colored indicator?
• "System flags" and tags are different things to the user. I don't like to conflate them using flags like IMAP does it. But to be able to have different data models (server vs. local) we need a new layer that translates data from the remote store to the model we use inside the app. I believe PR Add support for CONDSTORE and QRESYNC #2607 is a first step towards that (at least for IMAP).
• Currently MessageListFragment more or less directly accesses the database and expects to get all the data it wants to display using a single query. That makes it hard to extract some of the data to different database tables. I believe that needs to be changed before we can implement this feature in a clean way.

[philipwhiuk]

"System flags" and tags are different things to the user. I don't like to conflate them using flags like IMAP does it.

conflicts with

While this might have been motivated by the IMAP feature to store arbitrary flags

and the fact that clients like Thunderbird already support this.

I don't see why we need to make this an IMAP-only feature.

In reality while I mentioned WebDAV briefly, WebDAV doesn't document any way of storing tags. And POP3 obviously doesn't. So synced tags really are IMAP only. Local tags obviously could be stored on any message. I don't think that's a particular problem - you just don't update custom tags if the account doesn't support fetching/setting custom tags.

How should tags be displayed in the message list? I think changing the backgrounds adds way to much noise. Changing the text color is probably not much better. Maybe add some kind of colored indicator?

We don't have much space. A small chit maybe, next to the attachment chit on the redesign. But I don't think we should spend too much time on how it looks pre-redesign, only that it's considered in the re-design. Changing the background/text also prohibits multiple tags.

MLF is indeed ugly. It's a view and a controller that enforces behaviour on the data model.

I'm more than a bit grumpy if we really think killing the PR is a good idea though.

[cketti]

"System flags" and tags are different things to the user. I don't like to conflate them using flags like IMAP does it.

conflicts with

While this might have been motivated by the IMAP feature to store arbitrary flags

and the fact that clients like Thunderbird already support this.

I don't get that. From the user's perspective e.g. marking a message as read and adding a tag are totally unrelated operations in Thunderbird, too. All I wanted to say was that I don't want tags to be saved as flags in our database just because that's how it works with IMAP. It's an implementation detail that we don't have to copy in our internal data model. But of course we will map "K-9 Mail tags" to IMAP flags/keywords.

I'm more than a bit grumpy if we really think killing the PR is a good idea though.

I don't see all the changes I consider necessary for this to be implemented cleanly happening anytime soon. What's the alternative to closing the PR? Merging the code and hoping that someone will clean it up later hasn't worked once in the past.

[Valodim]

I do agree that tags should be separate from other flags in our data model. I don't quite see why this can't work in a minimum viable way and sync with imap though?

At some point we have to work with the resources we actually have, or we set ourselves up for stagnation. It's possible that this is not one of these cases where we a reasonable intermediate feature is achievable, but we should at least make an effort to find out, especially if someone is actually doing the work.

[PanderMusubi]

I am also in favor of simple independent implementation for IMAP only. First, start to show them, editting, managing names and colors, searching, ordering can be done later.

[Trogel]

PR #2668 started as a working prototype. So far it at least helped to spot several design issues that need clarification. In addition, I already learned a lot working on it. :-)

Personally I don't mind whether the PR is kept alive as work in progress or gets dropped and (some day) succeeded by one or more new PRs. In any case, I am happy to contribute to the design work and the implementation, if there is a chance to make progress. I can spend some hours or a day every now and then.

Regarding cleaning up later, my experience is that it usually never happens. Therefore the fundamental design should be sound before some PR gets merged. At the same time I think we can put some features on the wish list for later.

What is the preferred way to go forward? Continue the discussion here and sketch a high-level overall plan? Raise other, more specific issues or PRs? I can imagine a granularity on the level of "design data model for tags", which might result in some page in the development section on k9mail.github.io, and later "extend local store for tags", then work on IMAP, UI, and so on.

Here are my thoughts on some of the specific issues raised above:

I don't want tags to be saved as flags in our database just because that's how it works with IMAP.

I agree, we should keep that separate. Since this touches the very base of PR #2668 and its predecessors (8eea618), I also think that starting from scratch makes sense.

In my opinion tags should be account-specific.

For me one global set of tags would be fine, But I agree that there are use-cases where account-specific tags make sense. And I think, there are also use-cases where one set of tags should be used for more than one account. So, there is a third option - sadly, it is more complex:

Have one or more sets of tags on the global level and link each account to one of these sets.

Changing one of the simpler implementations later into this wouldn't be fun. So, we might better live with the higher complexity from the start. Or is it over the top?

How should tags be displayed in the message list?

Changing the background color was inspired by Thunderbird. It is exactly what I need, but I did not expect all people to like it. It should be optional, and there should be other options.

In the PR I already considered to optionally add a set of colored tag icons left of the star icon in the message list and message view. The star icon could be made optional also in the message view. And the tag icons could be limited to just one - at the users choice. Just some ideas for a start...

Changing the background/text also prohibits multiple tags.

Multiple tags can still be assigned to a mail, but this way we can show only one of them in the list. The PR picks the color of the first assigned tag according to the user-defined order, again inspired by Thunderbird. The full list can be seen in the message view.

[philipwhiuk]

Currently MessageListFragment more or less directly accesses the database and expects to get all the data it wants to display using a single query. That makes it hard to extract some of the data to different database tables. I believe that needs to be changed before we can implement this feature in a clean way.

It should be possible to both refactor the tables and keep it as a single query. I'll open a ticket for reviewing/reworking how we organise data in the database.

[Trogel]

Are there further opinions regarding the question whether there should be

1. just one global set of tags,
2. one set per account, or
3. multiple sets on the global level with each account being linked to one of them?

I would like to sketch a data model just for the basic sets (or set) of tags, for example on a new page in the development section of the website. This could then serve as a basis for further discussion.

And, of course, please stop me at any time, if you think, I am on the wrong track!

[severo]

I imagine the best way would be to have a list of tags per account, possible retrieved from the IMAP server if it implements the functionnality (https://tools.ietf.org/html/rfc3501#section-2.3.2), and if it doesn't, to propose the classical list of five tags in Thunderbird.

[ccoenen]

Just tossing my use case in here for reference. If anyone else has different use cases, it's perhaps also a good idea to comment on that here. If you have the same use-case, i'd suggest just using the reaction-smileys on the specific post. (also: if you think this is a stupid idea, stop me, too ;-) )

So, here it goes:

I am using K-9 with three different accounts. I access those accounts only from K-9 and Thunderbird (with the exception of some webmailer every now and then, but I digress).
I use the standard Thunderbird labels, available via number keys. I do not use them for what they were meant to be, I also don't bother renaming them to my usage. I just want a message to show up as "red" or "green" in my inbox.

To me, having a centralised set of labels would be enough. I do not require @Trogel's option 2 or option 3 (I am not at all against those, I'd just not need them).

[severo]

I use the same way as @ccoenen

[Trogel]

Thanks for your feedback! Since there is demand for both options 1 and 2, I'd go with option 3 that provides most flexibility.

PR k9mail/k9mail.github.io#57 starts off a design document for message tags as a basis for further discussion. Looking forward to your feedback in the PR!

[wokawoka]

Any news on IMAP labels implementation?
It would be really a great feature.
Since I remember of having seen screenshots documenting the development of this function here on github (or on bountysource?) some months ago, are there by chance experimental versions with this feature?
Thanks guys

[Trogel]

I am sorry, but I simply ran out of time. Unfortunately I cannot say when the situation will change.

#2668 won't be merged (which is fine for me) and k9mail/k9mail.github.io#57 now looks somewhat over the top to me. Therefore and also since I cannot work on them in the foreseeable future, I will close both pull requests.

Thanks for your great work on k9! It is still my favourite mail client on Android. Just the IMAP keywords are missing... ;-)

@wokawoka: There are a few screenshots in #2668, but there are no experimental versions of k9 with that feature available, as far as I know.

[PanderMusubi]

Perhaps increasing the bounty will help.

[h4emp3]

I too have the exact same use-case as @ccoenen and would very much appreciate this feature being added. And personally I would be totally fine with an absolutely minimal approach, like having K9 just show already existing labels set with another client (e.g. Thunderbird).

[EnigmaSolved]

I, too, would love to have this functionality. My use is pretty similar to #769 (comment)

Also, for handling colors (ie, coloring the background of a message summary in list view) when there are multiple tags: I recommend doing what Thunderbird does and have the highest order tag's color be the one that is used for the background color. I've setup my tags with an understanding of that behavior and it's worked fine for me. And then of course within the message itself display each tag with it's corresponding color. The screenshots from the earlier attempt at all this look right to me: #2668 (comment)

[rezazarchi]

• How should tags be displayed in the message list? I think changing the backgrounds adds way to much noise. Changing the text color is probably not much better. Maybe add some kind of colored indicator?

I and My colleague (@mhf1998) designed a demo for message list which may help you to have vision about its design:

[PanderMusubi]

If possible, please support the default colors Thunderbird also uses for these five default tags.

[ccoenen]

Should be noted that these tags are localised, and therefore you might not be able to rely just on the name at hand! A more international approach might be to scan for used tags and make it really simple to assign a color to them.

[PanderMusubi]

Can the localization be supported so it works out of the box? The files needed for this are available in Thunderbird's repo.

[CoachDom]

What's the current status of this? I currently sort my Zoho mail using tags so it's a bit painful that all emails to all domains are sitting in one mailbox

[PanderMusubi]

I could only find these tests:

• releases-comm-central-master/mail/components/extensions/test/xpcshell/test_ext_messages.js
• releases-comm-central-master/mail/base/test/browser/browser_tagsMode.js

Where can I find the source files that define the default tags, and possibly where they are translated?

[PanderMusubi]

Ah, that was a bit of a search. Here are the tag definitions to (preferably) support by default as they are used too by Thunderbird: https://github.com/mozilla/releases-comm-central/blob/master/mail/locales/en-US/chrome/messenger/messenger.properties#L207

Because the Thunderbird in use can be in another language, the translations of tags can be found in these downloads:

• German https://pontoon.mozilla.org/translations/?code=de&slug=thunderbird&part=mail/chrome/messenger/messenger.properties
• French https://pontoon.mozilla.org/translations/?code=fr&slug=thunderbird&part=mail/chrome/messenger/messenger.properties
• Dutch https://pontoon.mozilla.org/translations/?code=nl&slug=thunderbird&part=mail/chrome/messenger/messenger.properties
• etc.

The locals that are available are: https://github.com/mozilla/releases-comm-central/blob/master/mail/locales/shipped-locales these need to be cross checked with the locales supported by K-9 which are in https://github.com/thundernest/k-9/tree/main/app/ui/legacy/src/main/res (values-*). Additionally only well-translated languages from https://explore.transifex.com/k-9/k9mail/

If the developer working on supporting IMAP labels finds it useful, I can make a Python script that does all of the above and generate localized names for default Thunderbird IMAP tags. Perhaps that must be a setting in K-9 to use this mapping or not.

[PanderMusubi]

@CoachDom does Zoho mail have any default tags it supports? https://www.zoho.com/mail/help/applying-tags.html?zredirect=f&zsrc=langdropdown&lb=de

[PanderMusubi]

Here is a script that collects all tag translations https://pastebin.com/vjPyiMjJ (at the moment, it will only write to the command line)

[bikb]

Hello, would it make sense to implement it as an plugin / addon?

[PanderMusubi]

For me, this should be standard functionality.

[bikb]

So is there a Roadmap available? I think Mozilla is a big institution which plans the development. But where is the plan and who can be assigned to this open issue?

[qftlzxfz]

So is there a Roadmap available? I think Mozilla is a big institution which plans the development. But where is the plan and who can be assigned to this open issue?

I propose to discard previous ideas and discussions and just copy Thunderbird desktop functionality to Thunderbird android.

For anyone using tags it is a major handicap of Thunderbird android not supporting labels.