[fetchmail]fetchmail vs Maillenium; mail truncated to 80K

[jcfoley@comcast.net]

You're probably using a Comcast POP3 server.  Many others have
experienced this problem.  The problem is that the server truncates
the amount of data returned by the POP3 TOP command.  Comcast changed
to the Maillennium POP3 server in Summer 2003.  For several months
they refused to acknowledge any issue at their end that would account
for email truncation.  Recently the Comcast Government Affairs Manager
at Comcast of Montgomery (Maryland) sent me the information at the end
of this message.

I believe the Outlook Express flaw they reference was fixed a few
years ago.  Regardless it does seem to be a strange and non-conforming
server implementation that silently does the wrong thing specified by
the RFC and every other server I've used.

On the other hand, people have made the comment that fetchmail should
not be relying on TOP because a) that's not what it is for and/or b)
it is an optional POP3 command.

Item I8 of the fetchmail FAQ which appears to be maintained by Eric
S. Raymond says, "Don't mistake this for a fetchmail bug."

It would be nice to hear from a fetchmail expert/authority on whether
fetchmail is doing the right thing by using TOP and for a rationale of
the FAQ's response.

If fetchmail's use of TOP is legitimate then maybe Comcast would
uncripple their server if more people complained.

Jim Foley

=======================================================================
=======================================================================

Date: Wed, 3 Mar 2004 11:59:17 -0500

Mr. Foley, this email responds to the questions you posed following our
conference call.

First, Comcast does support POP 3 TOP commands, however Comcast has found
that increasing the amount of data TOP returns beyond the value of 64K has a
tendency to crash Microsoft Outlook Express when an abnormally large header
is sent.  Increasing the value beyond 64K would open the platform to
malicious use of large headers that adversely impacts system performance.
Virtually all of Comcast's high-speed Internet customers use Outlook
Express. Comcast has not received requests from other subscribers who seek
to use the TOP command in the manner you have requested.  Further, Comcast
has not received any other complaints regarding email truncation with the
TOP command.  Should you wish to continue checking your mail through manual
commands you might try using the RETR command, which will return the entire
message.

...



Date: Fri, 5 Mar 2004 16:28:11 -0500

Mr. Foley:

This is in response to your question regarding "POP 3 RFC compliance."  We
have tried to answer your question about Comcast's services by talking about
the specific application in which you are interested and how that
application relates to technical information regarding the configuration of
Comcast's Internet service.  We have provided you all the information that
we can by explaining that Comcast limits the optional POP 3 Top Command to a
value of 64k because any larger value has a tendency to crash Microsoft
Outlook and could leave Comcast's system open to the malicious use of large
headers intended to impair system performance.

The decision by Comcast to place limitations on the optional POP 3 TOP email
commands is a technical business decision made by Comcast in the best
interest of all its customers and its system. ...

...

With respect to the specific RFC at issue, RFC 1939, POP 3, it is our
understanding that it is a protocol "intended to permit a workstation to
dynamically access a maildrop on a server host in a useful fashion.
Usually, this means that the POP3 protocol is used to allow a workstation to
retrieve mail that the server is holding for it.  Pop 3 is not intended to
provide extensive manipulation operations of mail on the server."  POP 3 was
created in May 1996 and has not been revised since, despite the many changes
in computer hardware and software related to handling of email since that
time.  In any event, the TOP command is identified as an optional POP 3
command in RFC 1939.

...