Improve SSL/TLS documentation, and rearrange manual

Move SSL subsection up under AUTHENTICATION and ENCRYPTION,
where it belongs.

3 files changed, 113 insertions, 92 deletions

diff --git a/README.SSL b/README.SSL
index 9cbb50ce..ba3de41a 100644
--- a/README.SSL
+++ b/README.SSL
@@ -6,7 +6,8 @@ Preface
 
 Note: there is a separate document "README.SSL-SERVER" describing the server- 
 side requirements for proper SSL support.  It has checklist-style and is not 
-specific to fetchmail.
+specific to fetchmail. This document is about the client-side (fetchmail side)
+setup.
 
 In case of troubles, mail the README.SSL-SERVER file to your ISP and 
 have them check their server configuration against it.
@@ -21,7 +22,11 @@ If your server does not support this, you may have to specify --sslproto
 ssl3+.  This is in order to prefer the newer TLS protocols, because SSLv2
 and v3 are broken.
 
-	-- Matthias Andree, 2019-08-18
+Finally, due to other defaults changing, and several mail services not
+supporting in-band negotiation of SSL or TLS by means of STLS or STARTTLS,
+you may need to add ssl or --ssl to your configuration.
+
+	-- Matthias Andree, 2021-03-29
 
 
 Quickstart
@@ -45,6 +50,9 @@ or these options in the rcfile (after the respective "user"... options)
       sslproto auto   sslcertck
 
 
+Note that some services do not offer STLS or STARTTLS, but most do
+offer SSL or TLS on a separate, dedicated, "secure" port:
+
 For use of SSL or TLS on a separate port, if the whole TCP connection is 
 SSL-encrypted from the very beginning (SSL- or TLS-wrapped), use these
 command line options (in the rcfile, omit all leading "--"):
diff --git a/fetchmail-FAQ.html b/fetchmail-FAQ.html
index 2c2a5cda..899bdb07 100644
--- a/fetchmail-FAQ.html
+++ b/fetchmail-FAQ.html
@@ -160,9 +160,9 @@ methods</h2>
 <a href="#K2">K2. How can I use fetchmail with IPv6 and IPsec?</a><br/>
 <a href="#K3">K3. How can I get fetchmail to work with ssh?</a><br/>
 <a href="#K4">K4. What do I have to do to use the IMAP-GSS protocol?</a><br/>
-<a href="#K5">K5. How can I use fetchmail with SSL?</a><br/>
+<a href="#K5">K5. How can I use fetchmail with SSL or TLS?</a><br/>
 <a href="#K6">K6. How can I tell fetchmail not to try TLS if the server
-    advertises it? Why does fetchmail use SSL even though not configured?</a><br/>
+    advertises it? Why does fetchmail use STARTTLS, STLS, TLS or SSL even though not configured?</a><br/>
 
 <h2 id="C_R">Runtime fatal errors</h2>
 
@@ -2076,7 +2076,7 @@ Kerberos principal.</p>
 cleartext in your .fetchmailrc, or across the network.</p>
 
 <h2><a id="K5" name="K5">K5. How can I use fetchmail with
-SSL?</a></h2>
+SSL or TLS?</a></h2>
 
 <p>You'll need to have the <a
 href="https://www.openssl.org/">OpenSSL</a> libraries installed, and they
@@ -2087,9 +2087,9 @@ suffice. If you have them installed in a non-default location,
 you'll need to specify the OpenSSL installation directory as an argument
 to --with-ssl after an equal sign.</p>
 
-<p>Fetchmail binaries built this way support <code>ssl</code>,
+<p>Fetchmail binaries built this way support <code>ssl</code> and <code>tls</code>,
 <code>sslkey</code>, and <code>sslcert</code> options that control
-SSL encryption, and will automatically use <code>tls</code> if the
+SSL encryption, and will automatically try to negotiate <code>starttls</code> or <code>stls</code>if the
 server offers it. You will need to have an SSL/TLS-enabled mailserver to
 use these options. See the manual page for details and some words
 of care on the limited security provided.</p>
@@ -2117,8 +2117,8 @@ poll MYSERVER port 993 plugin "openssl s_client -connect %h:%p"
 
 <p>You should note that SSL or TLS are only secure against a
 "man-in-the-middle" attack if the client is able to verify that the
-peer's public key is the correct one, and has not been substituted by an
-attacker. fetchmail can do this in one of two ways: by verifying the SSL
+presented peer's public key is the correct one, and has not been substituted by an
+attacker along the way. fetchmail can do this in one of two ways: by verifying the SSL
 certificate, or by checking the fingerprint of the peer's public
 key.</p>
 
@@ -2154,13 +2154,13 @@ poll pop3.example.com proto pop3 uidl no dns
 self-signed certificate), then this certificate validation will always
 fail.</p>
 
-<p>Certificate verification is always attempted. If it fails, by default a
-warning is printed but the connection carries on (which means you are not
-protected against attack). If your server's certificate has been properly
-set up and verifies correctly, then add the "sslcertck" option to enforce
-validation. If your server doesn't have a valid certificate though (e.g. it
+<p>Certificate verification is always attempted. If it fails, since v6.4.0, by default the connection aborts (6.3 and older would carry on after printing a warning). 
+
+If your server doesn't have a valid certificate though (e.g. it
 has a self-signed certificate) then it will never verify, and the only way
-you can protect yourself is by checking the fingerprint.</p>
+you can protect yourself is by checking the fingerprint. You should then contact
+the operator and as for properly issued certificates.
+</p>
 
 <p>To check the peer fingerprint: first use fetchmail -v once to connect to
 the host, at a time when you are pretty sure that there is no attack in
@@ -2180,16 +2180,26 @@ a man-in-the-middle attack is in progress - or it might just mean that the
 server changed its key. It's up to you to determine which has happened.</p>
 
 <h2><a id="K6" name="K6">K6. How can I tell fetchmail not to use TLS
-	if the server advertises it? Why does fetchmail use SSL even
+	if the server advertises it? Why does fetchmail use STARTTLS, STLS, TLS or SSL even
 	though not configured?</a></h2>
 
-<p>Some servers advertise STLS (POP3) or STARTTLS (IMAP), and fetchmail
-will automatically attempt TLS negotiation if SSL was enabled at compile
+<p>Some options in fetchmail - including sslcertck that has become the
+default in v6.4.0 - require fetchmail to negotiate SSL or TLS.
+In other situations, fetchmail tries to negotiate SSL or TLS 
+opportunistically: Some servers advertise STLS (POP3) or STARTTLS (IMAP),
+and fetchmail will automatically attempt TLS negotiation if SSL was enabled at compile
 time.  This can however cause problems if the upstream didn't configure
 his certificates properly.</p>
 
+<p>In some situations, the server does not offer STARTTLS or STLS, but
+it would offer a TLS-wrapped operation on a dedicated, separate port.
+In such a situation, adding ssl to the rcfile (or --ssl on the command line) is
+all there is to it. Fetchmail will use the "other" default port for the "secure"
+service.</p>
+
 <p>In order to prevent fetchmail 6.4.0 and newer versions from trying
-STLS or STARTTLS negotiation, add this option:</p>
+STLS or STARTTLS negotiation, and only as a last resort because it exposes
+all communication to potential eavesdroppers, you could add this option:</p>
 <pre>sslproto ''</pre>
 
 <hr/>
diff --git a/fetchmail.man b/fetchmail.man
index 7846ace1..855bb00b 100644
--- a/fetchmail.man
+++ b/fetchmail.man
@@ -71,7 +71,8 @@ language, please use English.
 
 
 
-.SS CONCEPTS
+
+.SH CONCEPTS
 If \fBfetchmail\fP is used with a POP or an IMAP server (but not with
 ETRN or ODMR), it has two fundamental modes of operation for each user
 account from which it retrieves mail: \fIsingledrop\fP- and
@@ -1195,6 +1196,78 @@ On mailservers that do not provide ordinary user accounts, your user-id and
 password are usually assigned by the server administrator when you apply for
 a mailbox on the server.  Contact your server administrator if you don't know
 the correct user-id and password for your mailbox account.
+
+.SS Secure Socket Layers (SSL) and Transport Layer Security (TLS)
+.PP
+All retrieval protocols can use SSL or TLS wrapping for the
+transport. Additionally, POP3 and IMAP retrival can also negotiate
+SSL/TLS by means of STARTTLS (or STLS).
+.PP
+Note that fetchmail currently uses the OpenSSL library, which is
+severely underdocumented, so failures may occur just because the
+programmers are not aware of OpenSSL's requirement of the day.
+For instance, since v6.3.16, fetchmail calls
+OpenSSL_add_all_algorithms(), which is necessary to support certificates
+using SHA256 on OpenSSL 0.9.8 -- this information is deeply hidden in
+the documentation and not at all obvious.  Please do not hesitate to
+report subtle SSL failures.
+.PP
+You can access SSL encrypted services by specifying the options starting
+with \-\-ssl, such as \-\-ssl, \-\-sslproto, \-\-sslcertck, and others.
+You can also do this using the corresponding user options in the .fetchmailrc
+file.  Some services, such as POP3 and IMAP, have
+different well known ports defined for the SSL encrypted services.  The
+encrypted ports will be selected automatically when SSL is enabled and
+no explicit port is specified.   Also, the \-\-sslcertck command line or
+sslcertck run control file option should be used to force strict
+certificate checking with older fetchmail versions - see below.
+.PP
+If SSL is not configured, fetchmail will usually opportunistically try to use
+STARTTLS. STARTTLS can be enforced by using \-\-sslproto\~auto and
+defeated by using \-\-sslproto\~''.
+TLS connections use the same port as the unencrypted version of the
+protocol and negotiate TLS via special command. The \-\-sslcertck
+command line or sslcertck run control file option should be used to
+force strict certificate checking - see below.
+.PP
+.B \-\-sslcertck is recommended:
+When connecting to an SSL or TLS encrypted server, the
+server presents a certificate to the client for validation.  The
+certificate is checked to verify that the common name in the certificate
+matches the name of the server being contacted and that the effective
+and expiration dates in the certificate indicate that it is currently
+valid.  If any of these checks fail, a warning message is printed, but
+the connection continues.  The server certificate does not need to be
+signed by any specific Certifying Authority and may be a "self-signed"
+certificate. If the \-\-sslcertck command line option or sslcertck run
+control file option is used, fetchmail will instead abort if any of
+these checks fail, because it must assume that there is a
+man-in-the-middle attack in this scenario, hence fetchmail must not
+expose cleartext passwords. Use of the sslcertck or \-\-sslcertck option
+is therefore advised; it has become the default in fetchmail 6.4.0.
+.PP
+Some SSL encrypted servers may request a client side certificate.  A client
+side public SSL certificate and private SSL key may be specified.  If
+requested by the server, the client certificate is sent to the server for
+validation.  Some servers may require a valid client certificate and may
+refuse connections if a certificate is not provided or if the certificate
+is not valid.  Some servers may require client side certificates be signed
+by a recognized Certifying Authority.  The format for the key files and
+the certificate files is that required by the underlying SSL libraries
+(OpenSSL in the general case).
+.PP
+A word of care about the use of SSL: While above mentioned
+setup with self-signed server certificates retrieved over the wires
+can protect you from a passive eavesdropper, it doesn't help against an
+active attacker. It's clearly an improvement over sending the
+passwords in clear, but you should be aware that a man-in-the-middle
+attack is trivially possible (in particular with tools such as
+.URL "https://monkey.org/~dugsong/dsniff/" "dsniff" ,
+).  Use of strict certificate checking with a certification authority
+recognized by server and client, or perhaps of an SSH tunnel (see below
+for some examples) is preferable if you care seriously about the
+security of your mailbox and passwords.
+
 .SH POP3 VARIANTS
 .PP
 Early versions of POP3 (RFC1081, RFC1225) supported a crude form of
@@ -1218,6 +1291,7 @@ database.
 
 \fBNote that APOP is no longer considered resistant against
 man-in-the-middle attacks.\fP
+
 .SS RETR or TOP
 \fBfetchmail\fP makes some efforts to make the server believe messages
 had not been retrieved, by using the TOP command with a large number of
@@ -1242,7 +1316,7 @@ implies that in "keep" setups, "uidl" must be set if "TOP" is desired.
 fetchmail, but the behavior may change in future versions. In
 particular, fetchmail may prefer the RETR command because the TOP
 command causes much grief on some servers and is only optional.
-.SH ALTERNATE AUTHENTICATION FORMS
+.SH ALTERNATE AUTHENTICATION FORMS/METHODS
 .PP
 If your \fBfetchmail\fP was built with Kerberos support and you specify
 Kerberos authentication (either with \-\-auth or the \fI.fetchmailrc\fP
@@ -1291,77 +1365,6 @@ capability response. Specify a user option value that looks like
 \&'user@domain': the part to the left of the @ will be passed as the
 username and the part to the right as the NTLM domain.
 
-.SS Secure Socket Layers (SSL) and Transport Layer Security (TLS)
-.PP
-All retrieval protocols can use SSL or TLS wrapping for the
-transport. Additionally, POP3 and IMAP retrival can also negotiate
-SSL/TLS by means of STARTTLS (or STLS).
-.PP
-Note that fetchmail currently uses the OpenSSL library, which is
-severely underdocumented, so failures may occur just because the
-programmers are not aware of OpenSSL's requirement of the day.
-For instance, since v6.3.16, fetchmail calls
-OpenSSL_add_all_algorithms(), which is necessary to support certificates
-using SHA256 on OpenSSL 0.9.8 -- this information is deeply hidden in
-the documentation and not at all obvious.  Please do not hesitate to
-report subtle SSL failures.
-.PP
-You can access SSL encrypted services by specifying the options starting
-with \-\-ssl, such as \-\-ssl, \-\-sslproto, \-\-sslcertck, and others.
-You can also do this using the corresponding user options in the .fetchmailrc
-file.  Some services, such as POP3 and IMAP, have
-different well known ports defined for the SSL encrypted services.  The
-encrypted ports will be selected automatically when SSL is enabled and
-no explicit port is specified.   Also, the \-\-sslcertck command line or
-sslcertck run control file option should be used to force strict
-certificate checking with older fetchmail versions - see below.
-.PP
-If SSL is not configured, fetchmail will usually opportunistically try to use
-STARTTLS. STARTTLS can be enforced by using \-\-sslproto\~auto and
-defeated by using \-\-sslproto\~''.
-TLS connections use the same port as the unencrypted version of the
-protocol and negotiate TLS via special command. The \-\-sslcertck
-command line or sslcertck run control file option should be used to
-force strict certificate checking - see below.
-.PP
-.B \-\-sslcertck is recommended:
-When connecting to an SSL or TLS encrypted server, the
-server presents a certificate to the client for validation.  The
-certificate is checked to verify that the common name in the certificate
-matches the name of the server being contacted and that the effective
-and expiration dates in the certificate indicate that it is currently
-valid.  If any of these checks fail, a warning message is printed, but
-the connection continues.  The server certificate does not need to be
-signed by any specific Certifying Authority and may be a "self-signed"
-certificate. If the \-\-sslcertck command line option or sslcertck run
-control file option is used, fetchmail will instead abort if any of
-these checks fail, because it must assume that there is a
-man-in-the-middle attack in this scenario, hence fetchmail must not
-expose cleartext passwords. Use of the sslcertck or \-\-sslcertck option
-is therefore advised; it has become the default in fetchmail 6.4.0.
-.PP
-Some SSL encrypted servers may request a client side certificate.  A client
-side public SSL certificate and private SSL key may be specified.  If
-requested by the server, the client certificate is sent to the server for
-validation.  Some servers may require a valid client certificate and may
-refuse connections if a certificate is not provided or if the certificate
-is not valid.  Some servers may require client side certificates be signed
-by a recognized Certifying Authority.  The format for the key files and
-the certificate files is that required by the underlying SSL libraries
-(OpenSSL in the general case).
-.PP
-A word of care about the use of SSL: While above mentioned
-setup with self-signed server certificates retrieved over the wires
-can protect you from a passive eavesdropper, it doesn't help against an
-active attacker. It's clearly an improvement over sending the
-passwords in clear, but you should be aware that a man-in-the-middle
-attack is trivially possible (in particular with tools such as
-.URL "https://monkey.org/~dugsong/dsniff/" "dsniff" ,
-).  Use of strict certificate checking with a certification authority
-recognized by server and client, or perhaps of an SSH tunnel (see below
-for some examples) is preferable if you care seriously about the
-security of your mailbox and passwords.
-
 .SS ESMTP AUTH
 .PP
 \fBfetchmail\fP also supports authentication to the ESMTP server on the