aboutsummaryrefslogtreecommitdiffstats
path: root/fetchmail-features.html
blob: 91dee6bc82f530aeaea859786f3704b585763421 (plain) 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Fetchmail Feature List</title>
<link rev="made" href="mailto:esr@snark.thyrsus.com" />
<meta name="description" content="The fetchmail brag sheet." />
<meta name="keywords" content="fetchmail, POP, POP3, IMAP, IMAP2bis, IMAP4" />
<style type="text/css">
/*<![CDATA[*/
 h1.c1 {text-align: center}
/*]]>*/
</style>
</head>
<body>
<table width="100%" cellpadding="0" summary="Canned page header">
<tr>
<td width="30%">Back to <a href="index.html">Fetchmail Home Page</a></td>
<td width="30%" align="right">$Date$</td>
</tr>
</table>

<hr />
<h1 class="c1">Fetchmail Feature List</h1>

<h2>Since 5.0:</h2>

<ul>
<li>STARTTLS is supported in both POP and IMAP.</li>

<li>ESMTP AUTH (RFC 2554) is supported.</li>

<li>Has the capability of adding trace information to the Received
header to faciliate mail filtering by mailserver and remote
account.</li>

<li>Fetchmail now has options to handle SSL certificate
validation.</li>

<li>Fetchmail can be told to fall back to delivering via local
sendmail if it can't open port 25.</li>

<li>Support for AUTH=CRAM-MD5 under POP3, a la RFC2195.</li>

<li>Support for ODMR (On-Demand Mail Relay), RFC 2645.</li>

<li>It's now easy to deliver mail to a local LMTP socket.</li>

<li>The interface option now checks both local and remote interface
IPs.</li>

<li>The plugin facility has been enhanced; %h and %p options are
now available to pass in the hostname and service port number.</li>

<li>Added a dropdelivered option to discard Delivered-To headers.
This addresses a problem with using fetchmail and postfix as a
relay inside a domain; when postfix sees incoming messages with
delivered-to headers looking exactly the same as the ones it adds
himself, it bounces the message.</li>

<li>Added --smtpname to set username and domain portion of SMTP
"RCPT TO" command. &lt;fetchmail@mail.julianhaight.com&gt;.</li>

<li>Added "from" server's IP address to inserted Received line
&lt;fetchmail@mail.julianhaight.com&gt;.</li>

<li>Fetchmail now runs on BeOS, thanks to David Reid
&lt;david@jetnet.co.uk&gt;.</li>

<li>In IMAP, unseen-message counting and indexing is now done by
SEARCH UNSEEN at the beginning of each poll or re-poll (rather than
with the UNSEEN and RECENT responses and FLAGS queries on
individual messages). This significantly cuts down on traffic to
and from the server, and gives more reliable results.</li>

<li>The aka option now matches hostname suffixes, so (for example)
saying `aka netaxs.com' will match not just netaxs.com but also
(say) pop3.netaxs.com and mail.netaxs.com.</li>

<li>Fetchmail can optionally use the RFC 2177 IDLE extension on an
IMAP server that supports it. On IMAP servers that don't, it can
simulate it using periodic NOOP commands.</li>

<li>Fetchmail now recognizes the RFC 2449 extended responses
[IN-USE] and [LOGIN-DELAY].</li>

<li>Fetchmail running in daemon mode now restarts itself quietly
when the rc file is touched.</li>

<li>Following recent court decisions and changes in U.S. federal
regulatory policy, hooks for Secure Sockets Layer (SSL) are now
part of the main fetchmail distribution. The distribution still
contains no actual cryptographic code.</li>

<li>NTLM support under IMAP, so fetchmail can query Microsoft
Exchange servers.</li>

<li>Expunge option can now be used to break POP3 retrieval into
subsessions.</li>

<li>Support for AUTH=CRAM-MD5 under IMAP, a la RFC2195.</li>
</ul>

<h2>Since 4.0:</h2>

<ul>
<li>The interface and monitor options now work with freeBSD.</li>

<li>Fetchmail now sends RFC1894-conformant bouncemail on SMTP and
LMTP errors.</li>

<li>Full support for LMTP according to RFC2033.</li>

<li>True multi-language support using GNU gettext.</li>

<li>Support for use of HESIOD with Kerberos.</li>

<li>The --bsmtp option supports recording fetched mail as a BSMTP
batch.</li>

<li>The --limit option can now be used in daemon mode, with
oversized-message notifications being mailed to the calling
user.</li>

<li>Configurable support for the <a
href="http://www.demon.net/info/helpdesk/demon_products/mail/sdps-tech.shtml">
SDPS extensions</a> in <a
href="http://www.demon.net/">www.demon.net</a>'s POP3 service.</li>

<li>There is now an interactive GUI fetchmail configurator,
fetchmailconf.</li>

<li>Code is 64-bit clean and Y2K-safe.</li>

<li>Automatically decodes armored 7-bit MIME into 8 bits (this can
be suppressed).</li>

<li>You can specify which SMTP error is recognized as a spam
block.</li>

<li>Support for Kerberos V authentication.</li>

<li>Support for IMAP-OTP authentication using Craig Metz's patches
for UW IMAP.</li>

<li>Support for IPv6</li>

<li>Support for IMAP with RFC1731-conformant GSSAPI
authentication.</li>

<li>Fixed and verified support for Cyrus IMAP server, M$ Exchange,
and Post Office/NT.</li>

<li>Support for responding with a one-time password when a POP3
server issues an RFC1938-conforming OTP challenge.</li>

<li>Support for Compuserve's RPA authentication protocol for POP3
(not compiled in by default, but configurable).</li>
</ul>

<h2>Since 3.0:</h2>

<ul>
<li>Support for IMAP RFC 1731 authentication with Kerberos v4.</li>

<li>Support for multiple-folder retrieval in a single session under
IMAP.</li>

<li>Following SMTP 571 response to a From line, fetchmail no longer
downloads the bodies of spam messages.</li>

<li>Support for a `hunt list' of SMTP hosts.</li>

<li>Support for ESMTP 8BITMIME and SIZE options.</li>

<li>Support for ESMTP ETRN command.</li>

<li>The stripcr &amp; forcecr options to explicitly control
carriage-return stripping and LF-&gt;CRLF mapping before mail
forwarding.</li>
</ul>

<h2>Since 2.0:</h2>

<ul>
<li>Support for secure use with ssh.</li>

<li>Mailserver passwords can be parsed out of your .netrc
file.</li>

<li>When forwarding mail via SMTP, fetchmail respects the 571 "spam
filter" response and discards any mail that triggers it.</li>

<li>Transaction and error logging may optionally be done via
syslog.</li>

<li>(Linux only) Security option to permit fetchmail to poll a host
only when a point-to-point link to a particular IP address is
up.</li>

<li>RPOP support (restored; had been removed in 1.8).</li>
</ul>

<h2>2.0 and earlier versions:</h2>

<ul>
<li>Support POP2, APOP, RPOP, IMAP2, IMAP2bis, IMAP3, IMAP4,
IMAP4rev1. .</li>

<li>Support for Kerberos V4 user authentication (either MIT or
Cygnus).</li>

<li>Host is auto-probed for a working server if no protocol is
specified for the connection. Thus you don't need to know what
servers are running on your mail host in advance; the verbose
option will tell you which one succeeds.</li>

<li>Delivery via SMTP to the client machine's port 25. This means
the retrieved mail automatically goes to the system default MDA as
if it were normal sender-initiated SMTP mail.</li>

<li>Configurable timeout to detect if server connection is
dropped.</li>

<li>Support for retrieving and forwarding from multi-drop mailboxes
that is guaranteed not to cause mail loops.</li>

<li>Large user community -- fetchmail has a large user base (the
author's beta list includes well over two hundred people). This
means feedback is rapid, bugs get found and fixed rapidly.</li>

<li>Carefully written, comprehensive and up-to-date man page
describing not only modes of operation but also how to diagnose the
most common kinds of problems and what to do about deficient
servers.</li>

<li>Rugged, simple, and well-tested code -- the author relies on it
every day and it has never lost mail, not even in experimental
versions. (In the project's entire history there has only been one
recorded instance of lost mail, and that was due to a quirk in some
Microsoft code.)</li>

<li>Strict conformance to relevant RFCs and good debugging options.
You could use fetchmail to test and debug server
implementatations.</li>

<li>For anybody who cares, fetchmail is Y2K safe.</li>
</ul>

<h2>Features in common with other remote-mail retrieval
programs:</h2>

The other programs I have checked include fetchpop1.9,
PopTart-0.9.3, get-mail, gwpop, pimp-1.0, pop-perl5-1.2, popc,
popmail-1.6 and upop. 

<ul>
<li>Support for POP3.</li>

<li>Easy control via command line or free-format run control
file.</li>

<li>Daemon mode -- fetchmail can be run in background to poll one
or more hosts at a specified interval.</li>

<li>From:, To:, Cc:, and Reply-To: headers are rewritten so that
usernames relative to the fetchmail host become fully-qualified
Internet addresses. This enables replies to work correctly. (Would
be unique to fetchmail if I hadn't added it to fetchpop.)</li>

<li>Message and header processing are 8-bit clean.</li>
</ul>

<hr />
<table width="100%" cellpadding="0" summary="Canned page footer">
<tr>
<td width="30%">Back to <a href="index.html">Fetchmail Home Page</a></td>
<td width="30%" align="right">$Date$</td>
</tr>
</table>

<br clear="left" />
<address>Eric S. Raymond <a href="mailto:esr@thyrsus.com">&lt;esr@snark.thyrsus.com&gt;</a></address>
</body>
</html>
generated by cgit v1.2.3 (git 2.39.1) at 2025-07-06 23:38:52 +0000
le things getting passed to the DNS lookup routines.<P> Alternative ways to handle the problem: ignore DNS errors (treating them as a non-match on the mailserver domain), or forward messages with errors to fetchmail's invoking user in addition to any other recipients. These would fit an assumption that DNS lookup errors are likely to be permanent problems associated with an address.<P> <H1>IPv6 and IPSEC</H1> The IPv6 support patches are really more protocol-family independence patches. Because of this, in most places, "ports" (numbers) have been replaced with "services" (strings, that may be digits). This allows us to run with certain protocols that use strings as "service names" where we in the IP world think of port numbers. Someday we'll plumb strings all over and then, if inet6 is not enabled, do a getservbyname() down in SocketOpen. The IPv6 support patches use getaddrinfo(), which is a POSIX p1003.1g mandated function. So, in the not too distant future, we'll zap the ifdefs and just let autoconf check for getaddrinfo. IPv6 support comes pretty much automatically once you have protocol family independence.<P> Craig Metz used his inner_connect() function to handle most of the connect work. This is a nonstandard function not likely to ever exist in a system's libc, but we can just include that source file if the day comes when we want to support IPv6 without the inet6-apps library. It just makes life easier.<P> <H1>Checklist for Adding Options</H1> Adding a control option is not complicated in principle, but there are a lot of fiddly details in the process. You'll need to do the following minimum steps. <UL> <LI>Add a field to represent the control in <code>struct query</code> or <code>struct hostdata</code>. <LI>Pick a token to declare the option in the .fetchmailrc file. Add the token to <code>rcfile_l</code>. <LI>Go to <code>rcfile_y.y</code>. Add the token to the grammar. Don't forget the <code>%token</code> declaration. <LI>Pick a long-form option name, and a one-letter short option if any are left. Go to <code>options.c</code>. Pick a new <code>LA_<code> value. Hack the <code>longoptions</code> table to set up the association. Hack the big switch statement to set the option. Hack the `?' message to describe it. <LI>If the default is nonzero, set it in <code>def_opts</code> near the top of <code>load_params</code> in <code>fetchmail.c</code>. <LI>Add code to dump the option value in <code>fetchmail.c:dump_params</code>. <LI>Add proper <code>FLAG_MERGE</code> actions in fetchmail.c's optmerge() function. <LI>Document the option in fetchmail.man. This will require at least two changes; one to the collected table of options, and one full text description of the option. <LI>Add the new token and a brief description to the header comment of sample.rcfile. <LI>Add an entry to NEWS. </UL> There may be other things you have to do in the way of logic, of course.<P> <H1>Lessons learned</H1> <H3>1. Server-side state is essential</H3> The person(s) responsible for removing LAST from POP3 deserve to suffer. Without it, a client has no way to know which messages in a box have been read by other means, such as an MUA running on the server.<P> The POP3 UID feature described in RFC1725 to replace LAST is insufficient. The only problem it solves is tracking which messages have been read <EM>by this client</EM> -- and even that requires tricky, fragile implementation.<P> The underlying lesson is that maintaining accessible server-side `seen' state bits associated with Status headers is indispensible in a Unix/RFC822 mail server protocol. IMAP gets this right.<P> <H3>2. Readable text protocol transactions are a Good Thing</H3> A nice thing about the general class of text-based protocols that SMTP, POP2, POP3, and IMAP belongs to is that client/server transactions are easy to watch and transaction code correspondingly easy to debug. Given a decent layer of socket utility functions (which Carl provided) it's easy to write protocol engines and not hard to show that they're working correctly.<P> This is an advantage not to be despised! Because of it, this project has been interesting and fun -- no serious or persistent bugs, no long hours spent looking for subtle pathologies.<P> <H3>3. IMAP is a Good Thing.</H3> If there were a standard IMAP equivalent of the POP3 APOP validation, POP3 would be completely obsolete.<P> <H3>4. SMTP is the Right Thing</H3> In retrospect it seems clear that this program (and others like it) should have been designed to forward via SMTP from the beginning. This lesson may be applicable to other Unix programs that now call the local MDA/MTA as a program.<P> <H3>5. Syntactic noise can be your friend</H3> The optional `noise' keywords in the rc file syntax started out as a late-night experiment. The English-like syntax they allow is considerably more readable than the traditional terse keyword-value pairs you get when you strip them all out. I think there may be a wider lesson here.<P> <H1>Motivation and validation</H1> It is truly written: the best hacks start out as personal solutions to the author's everyday problems, and spread because the problem turns out to be typical for a large class of users. So it was with Carl Harris and the ancestral popclient, and so with me and fetchmail.<P> It's gratifying that fetchmail has become so popular. Until just before 1.9 I was designing strictly to my own taste. The multi-drop mailbox support and the new --limit option were the first features to go in that I didn't need myself.<P> By 1.9, four months after I started hacking on popclient and a month after the first fetchmail release, there were literally a hundred people on the fetchmail-friends contact list. That's pretty powerful motivation. And they were a good crowd, too, sending fixes and intelligent bug reports in volume. A user population like that is a gift from the gods, and this is my expression of gratitude.<P> The beta testers didn't know it at the time, but they were also the subjects of a sociological experiment. The results are described in my paper, <cite>The Cathedral And The Bazaar</cite>, available on the <a href="http://www.tuxedo.org/~esr/fetchmail">Fetchmail home page</a>. <H1>Credits</H1> Special thanks go to Carl Harris, who built a good solid code base and then tolerated me hacking it out of recognition. And to Harry Hochheiser, who gave me the idea of the SMTP-forwarding delivery mode.<P> Other significant contributors to the code have included Dave Bodenstab (error.c code and --syslog), George Sipe (--monitor and --interface), Gordon Matzigkeit (netrc.c), Al Longyear (UIDL support), Chris Hanson (Kerberos V4 support), anc Craig Metz (OPIE, IPv6, IPSEC).<P> <H1>Conclusion</H1> At this point, the fetchmail code appears to be pretty stable. It will probably undergo substantial change only if and when support for a new retrieval protocol or authentication method is added.<P> <H1>Relevant RFCS</H1> Not all of these describe standards explicitly used in fetchmail, but they all shaped the design in one way or another.<P> <DL> <DT>RFC821<DD> SMTP protocol <DT>RFC822<DD> Mail header format <DT>RFC937<DD> Post Office Protocol - Version 2 <DT>RFC974<DD> MX routing <DT>RFC976<DD> UUCP mail format <DT>RFC1081<DD> Post Office Protocol - Version 3 <DT>RFC1123<DD> Host requirements (modifies 821, 822, and 974) <DT>RFC1176<DD> Interactive Mail Access Protocol - Version 2 <DT>RFC1203<DD> Interactive Mail Access Protocol - Version 3 <DT>RFC1225<DD> Post Office Protocol - Version 3 <DT>RFC1344<DD> Implications of MIME for Internet Mail Gateways <DT>RFC1413<DD> Identification server <DT>RFC1428<DD> Transition of Internet Mail from Just-Send-8 to 8-bit SMTP/MIME <DT>RFC1460<DD> Post Office Protocol - Version 3 <DT>RFC1521<DD> MIME: Multipurpose Internet Mail Extensions <DT>RFC1869<DD> SMTP Service Extensions (ESMTP spec) <DT>RFC1652<DD> SMTP Service Extension for 8bit-MIMEtransport <DT>RFC1725<DD> Post Office Protocol - Version 3 <DT>RFC1730<DD> Interactive Mail Access Protocol - Version 4 <DT>RFC1731<DD> IMAP4 Authentication Mechanisms <DT>RFC1732<DD> IMAP4 Compatibility With IMAP2 And IMAP2bis <DT>RFC1734<DD> POP3 AUTHentication command <DT>RFC1870<DD> SMTP Service Extension for Message Size Declaration <DT>RFC1891<DD> SMTP Service Extension for Delivery Status Notifications <DT>RFC1892<DD> The Multipart/Report Content Type for the Reporting of Mail System Administrative Messages <DT>RFC1893<DD> Enhanced Mail System Status Codes <DT>RFC1894<DD> An Extensible Message Format for Delivery Status Notifications <DT>RFC1938<DD> A One-Time Password System <DT>RFC1939<DD> Post Office Protocol - Version 3 <DT>RFC1985<DD> SMTP Service Extension for Remote Message Queue Starting <DT>RFC2060<DD> Internet Message Access Protocol - Version 4rev1 <DT>RFC2061<DD> IMAP4 Compatibility With IMAP2bis <DT>RFC2062<DD> Internet Message Access Protocol - Obsolete Syntax </DL> <HR> <table width="100%" cellpadding=0><tr> <td width="30%">Back to <a href="index.html">Fetchmail Home Page</a> <td width="30%" align=center>To <a href="/~esr/sitemap.html">Site Map</a> <td width="30%" align=right>$Date: 1998/06/29 21:38:03 $ </table> <P><ADDRESS>Eric S. Raymond <A HREF="mailto:esr@thyrsus.com">&lt;esr@snark.thyrsus.com&gt;</A></ADDRESS> </BODY> </HTML>
generated by cgit v1.2.3 (git 2.39.1) at 2025-07-06 23:38:51 +0000