NOTES.txt

Notes on the IMAP protocol
--------------------------

These notes are mainly based on my reading of RFC 3501. They document thoughts and issues I had while implementing
a server conforming to IMAP4rev1 (whatever that means), and I'd be more than happy to update this file with hints and
notes received from others. It probably will grow in step with my blood pressure while implementing this abomination
of a protocol.

Mailbox names
	The entire section 5.1 should IMO be subject to a major rewrite, shortening and general un-f*cking.
	It's a product of its time (I love 7bit ASCII too, I get it), but UTF-7 is idiotic enough as specified,
	no need to mess with the alphabet just because (and what if I want my hierarchy separator to be ','. huh?).
	Spending an entire printed page on how the encoding differs from the standard UTF-7 encoding might have
	just warranted creating a new encoding and being done with it.
	Noting that any character being an 'ATOM-SPECIAL' is, well, special is fine, but referring people to a
	syntax spec spanning several printed pages just sucks for implementors (and having to track down an ABNF
	reference 8 levels deep is just no fun).
	Also, noting that control and wildcard characters in a mailbox name is a bad idea but not forbidding it
	seems weird to me.
	I can live with 'INBOX' being a special name, but did it have to be case-insensitive too?
	And what use do namespaces (the word shows up twice in the document aside from the section introducing
	the concept, and one of those is a clarification) have in a protocol which NO ONE USES FOR NEWSGROUPS?

Client state
	Most other mail protocols are simple state machines, where the server just needs to keep track of
	which state a client is in and maybe what data it already sent for a command. IMAP has that as well
	as per-user (who may or may not have multiple concurrent connections to a server) state and per-command
	(of which there may be multiple in-flight for every connection) state, all of which serves to just form
	an absolute nightmare of a state machine.

Mailbox hierarchy structure
	The implications of the DELETE command forbid a purely graph-based view of mailboxes, similar to
	a file-system, as it is possible to delete a mailbox having children (inferiors in IMAP parlance)
	without deleting those inferiors (the deleted mailbox is to be marked NOSELECT thereafter), effectively
	creating a dangling branch in the graph. Whether this deleted-but-still-recognized state implies that
	using CREATE on a NOSELECT mailbox should work is a question that seemingly nobody asked but I'd guess
	so.

	This also has implications on RENAME, as 1) it also renames any hierarchical children of a mailbox
	and 2) does not allow renaming to an already existing mailbox. In combination with the braindead
	hierarchy concept this means we cannot simply check that the root node to be renamed does not exist and
	assume that we can safely rename the child nodes, too. Instead we have to scan for all inferiors and see
	if renaming them would conflict with some existing mailbox.

Subscriptions
	What do they even do? My best guess is that they were some kind of mechanism to subscribe to newsgroups,
	which of course is totally irrelevant when implementing a basic mail server. Note that the concept of
	a set of "active" or "subscribed" mailboxes is introduced in 6.3.6, which is only ever used in the three
	commands dealing with subscriptions and never explained or used in any useful contexts.
	Also note that this has nothing to do with the "selected" mailbox as set by EXAMINE/SELECT.

Flags
	Multiple semi-related concepts rolled into one functionality. We learn that flags can be either stored
	permanently or have only session lifetime (which is fine by me), and that there are several system flags,
	whose mapping into this binary is never really clarified. Some of these special flags are extra special,
	so they have their own special untagged responses including the number of mails tagged with them.
	This maybe might have been nicer to do in a generic way (which also exists, but is different), but what do
	I know. Some of the more special flags also do not only adorn messages, but also trigger system actions such
	as deletion. The later parts of 2.3.2 also seem to refer to user-defined flags as 'keywords' (which at that
	point also may refer to something else entirely, who knows with this document). There is even a clarification
	noting this, however it does nothing to clarify. Flags can also be explicitly set (by a client), implicitly set
	(inferred from their state or simply generated by a server) and explicitly removed (by a client) or just
	disappear (of which any clients may or may not be notified). Section 6.3.1 implies that flags MAY be defined
	on a per-mailbox basis, however if that means whether we should return flags that are actually APPLIED to mails
	in the mailbox or if there is some sort of scoping mechanism is left open (other than saying to return those
	which are "applicable" to a mailbox, which is not only ambiguous but technically wrong as flags apply to
	messages not mailboxes).

Client synchronization
	Oh boy does this feel bolted on afterwards. But it's the one thing that really drives people to use IMAP,
	aside from being able have their mails waste space on a server instead of their own machines (well, that
	and sorting mail into an array of glorified recycling bins with varying halflifes).
	Previously on this show, I mentioned a client may have multiple connections to a server, which for
	examples offices love to share mail accounts. This requires a synchronization mechanism for mailbox and
	message state, the former of which is of course totally ignored in IMAP (it may come up in a later side
	note for an unrelated response, so I may update this then), meaning while one client creates a forest of
	mailboxes, all others for the same user will only know about that once they decide to LIST all mailboxes.
	While it might seem useful for a client to be able to detect other connections for the same user (some form of
	peer discovery), apparently it's not as IMAP has no such thing. In a sane world, commands concerning mailbox
	and message state might have been required to be mirrored for concurrent same-client connections in some
	generic fashion (possibly even allowing protocol extensions to synchronize state), alas it was not to be.
	Instead NOOP is used to shoe-horn in updates about mail status changes (for the selected mailbox only (I guess)),
	with IMAP IDLE being an addition that does next to nothing to ameliorate this except introducing a command that
	allows the server to do the same thing but with one really long-running command.