Surprises

Efficiency Concerns

tkbiff is very efficient in its interactions with mail servers (including POP and IMAP) and will cause insignicant additional load. In addition, tkbiff can be fine tuned to further reduce resource consumption both in terms of its frequency of polling and in the amount of information transferred.

Mixing Protocols

tkbiff can behave incorrectly if it is using a different protocol than another mail client accessing the same mailbox - for instance, if you are reading a mailbox using POP while tkbiff is accessing the mailbox using IMAP. The problem isn't tkbiff's fault. Rather, the problem is that these servers use incompatible locking methods and headers. In fact, it is inadvisable to even use two different versions of the same server simultaneously.

POP is particularly problematic in that two POP clients even of the same version can give anomolous results when used to access the same mailbox. Using POP, the act of checking if a message has been read, necessarily reads the message. This means that if you run multiple POP clients that are checking for new mail, only the first finds the mail appears to be new. For this reason, when tkbiff is using POP it defaults to displaying old mail.

This race condition is not a problem with IMAP or file-based access. IMAP specifically provides for simultaneous access (although IMAP provides no guarantee that such access will be timely!) and file-based access works correctly assuming the file system is administered with mail access in mind.

POP and IMAP surprises

IMAP can provide timely simultaneous access to mail. However, it does not guarantee timely access. (In fact, it doesn't guarantee simultaneous access is possible at all.) Some IMAP servers are good, some not so good. For example, mail clients which leave the mail box selected while idle prevent some IMAP servers from informing any other clients of new mail. Some versions of Netscape did this, for instance.

POP has several optional commands (TOP and UIDL) that can improve performance dramatically. tkbiff automatically uses these commands if your POP server supports them (either one or both). Fortunately, virtually all POP servers support them. (For this, we can thank Netscape and Eudora, each of which insists on one of the commands. So if you're using either of those tools, your POP server is good.) If your POP server does not support either TOP or UIDL, everytime tkbiff checks for new mail from a POP server, the entire mailbox must be downloaded. Obviously in this situation, 100Mb+ mailboxes place a large load on network traffic, particularly if the links are slow.

By comparison, IMAP and file-based access are much better. IMAP only needs to download a handful of bytes for each message indicating whether it's been read or not. File-based access only requires checking the size and date of the file. For these reasons, tkbiff is configured by default to check POP servers much less frequently than file or IMAP servers.

Leaving aside network bandwidth, large mailboxes cause great impact on POP servers. (This is a general behavior and has nothing to do with tkbiff specifically.) As the POP RFC says, ... the limited capabilities of the POP3 do not permit efficient handling of maildrops which have hundreds or thousands of messages. In fact, this is a problem for some IMAP servers, too.

Reducing the size of your folders may improve the performance of IMAP servers - sometimes dramatically. Some IMAP servers use memory directly related to the size of a folder. For example, if you have a 100Mb folder, then the server may require that much memory (whether this is true depends upon on the server and how it is configured). This is not due to the IMAP specification but rather some implementations of it.

As an aside, IMAP is a very complex specification. Although its design enables a reduction of communications bandwidth, IMAP has a reverse cost of increasing the complexity of both servers and clients. Supporting all the features of IMAP takes a lot of work. It is not surprising that some implementations choose to make some parts efficient (hoping that is what will be most helpful to clients) and other part inefficient (hoping clients will rarely use certain features). However, clients may have different requirements than the server implementors expected and it is not surprising when this mismatch produces poor behavior. This is a classic outcome of large specifications that offer multiple ways of doing the same thing.

tkbiff reports read mail as new mail when using POP with keep-mail-on-server. This is because POP offers no way to peek at a message (i.e., look at a message without considering it read). This shouldn't cause a significant problem because (at least in the default configuration), tkbiff doesn't put such mail in the to be announced list. However, such mail will still appear in the display. This latter behavior may be turned off by toggling the appropriate elements of the display array. However, this is a similar problem faced by any POP client. As the POP RFC states POP3 is not intended to provide extensive manipulation operations of mail on the server; normally, mail is downloaded and then deleted.

tkbiff does not support IMAP servers prior to version 4rev1 (RFC2060, December 1996). tkbiff does not support POP servers prior to version 3 (RFC1939, May 1996). If you have such old servers, you should upgrade (for lots of other reasons unrelated to tkbiff).

Missing or Duplicate Message Ids

Duplicate messages without message-id headers are problematic because there is no way to know whether such messages are copies of one another. A duplicate message could have been sent out twice on purpose or could have been sent out twice unintentionally (for example if a recipient is listed twice in a mailing list). Lack of a message id makes it impossible to distinguish these two cases - defeating the very purpose of tkbiff (which is, after all to notify users of new mail).

The current implementation of tkbiff takes a practical stance - on messages lacking message ids, tkbiff derives a message id by computing a checksum on the body. This works well on file-based protocols. This works less well on IMAP and POP because retrieving message bodies solely to compute a checksum is a waste of bandwidth, particularly if it has to be done every time mail is checked. Thus, tkbiff ignores such messages by default when using IMAP or POP unless the variable fake_message_ids is set to 1 (default is 0).

When faking message ids, tkbiff tries a number of things to avoid having to transfer the message. Only in extremely rare cases will the message have to be transferred. Indeed, for most people, this will never occur. Indeed, many servers automatically rewrite message headers to force message ids. And most clients generate them to begin with. (RFC822 does not require such headers but RFC822bis says Though optional, every message SHOULD have a Message-ID: field.) An example in which fake messages ids are necessary is when using an IMAP server on top of mbox-based formats which are also being written by /bin/mail's localmail-style delivery (in which /bin/mail writes directly to the mbox). /bin/mail is one such client that does not write message-ids and letting /bin/mail do the delivery bypasses the opportunity for an SMTP server to add a message id. Such configurations are thankfully rare.

tkbiff treats multiple messages with the same message-id as a single message. That is, you will only get notified once for each different message-id. This is in keeping with the meaning of a message-id.

NFS

It is a bad idea for mailboxes to be stored on partitions with NFS-caching enabled since new mail cannot be detected in such mailboxes until the cache is flushed. tkbiff will work on such misconfigured hosts if the variable suppress_nfs_caching exists. Although tkbiff provides support for NFS caching, other mail clients are likely to be confused by it.



Go to tkbiff homepage.
Go to tkbiff documentation.

Last edited: Tue Oct 8 15:49:23 EDT 2002 by Don Libes