(Also known as project Pavlova. This is only a draft write-up; a request for comments, if you will.
I hope to present patches implementing this proposal for Apex soon in the hope that this can appear in a version in the near future, and to submit it as a patch for StrongDC and for vanilla DC++, as well as an official ADC extension. -O1)
KEYP - KeyPrint, fingerprint-based TLS authentication for adcs: (Draft)
This extension adds a simple, but secure way to protect against man-in-the-middle attacks against ADC when wrapped with TLS (1.0 or later). It does not require setting up a CA or signing keys, but you still can if you want to.
Specifically, we introduce a "KeyPrint" - the hash of either the certificate signing the server certificate (in a CA-style key-signing configuration) or, simply, the server certificate itself (in a self-signed configuration).
We add this KeyPrint (or simply, KP) to the hub's adcs: URL in a URL-parameter, so that any attempted certificate substitution attack on a TLS hub connection to the hub will be detected.
Additionally, clients should publish their own certificates' KeyPrint in a "KP" string in the ADC "INF" broadcast. Assuming one trusts the hub enough not to maliciously change the KeyPrints en route (a reasonable assumption given the hub's existing position of trust), and given that the connection to the hub has been similarly authenticated (either as above or via a directly downloaded trusted certificate), client-client connections are also protected against attempted man-in-the-middle attacks - without messing around having to get everyone's certificates signed in advance.
2. KeyPrint Specification
2.1. A KeyPrint consists of a hash name, followed by a / (forward slash), followed by a base32-encoded cryptographic hash of either the certificate directly (which is appropriate in the case of a self-signed certificate), or a certificate providing the base of a valid signature chain (which may be more appropriate a CA-signed certificate).
2.2. It is suggested that TTH, which is the Tiger Tree Hash exactly as defined in the [TIGR] extension to ADC, be used for this, as it marks a readily-available, convenient, yet secure baseline. We use "TTH" to specify this; again, in the same way it is specified in [TIGR].
2.3. In a secure ADC (adcs:) hub URL, we add the KeyPrint to the URL as a URL parameter named "kp", encoded (for example) as follows:-
2.4. Clients which have generated a TLS certificate for secure client-client communication (which in this protocol need not be signed, so this should probably be done by default!) should publish their own KeyPrints (as above) as a string field named KP in their INF broadcast to ADC hubs.
2.5. If a client receives a KP field in an FINF broadcast via a hub to it is connected using adcs: and a trusted key as above (or otherwise), it should be regarded as the valid and correct KeyPrint for that client's IP/port/hub combination, replacing any earlier KeyPrint for that IP/port/hub combination.
3. KeyPrint Verification
3.1. When initiating a TLS handshake with a remote host where we know their KeyPrint, we can verify that a man-in-the-middle attack is not occurring by checking if the hash given in the KeyPrint matches exactly:
a root certificate presented in a valid signature chain which covers the certificate presented during the handshake by the remote host; or the entire certificate presented during the handshake by the remote host. 3.2. If we know a remote host's KeyPrint, and in connecting to that remote host, we are presented with a certificate which does not match, and which also does not have form the root of a valid signature chain covering that certificate: then we have detected a certificate substitution attack and we MUST abort the connection with a user-visible, non-modal error stating, for example "Crypto error: Detected attempted man-in-the-middle attack, aborting". (This error quite possibly represents a real attempted attack that has been foiled; we may try auto-reconnecting but we should NEVER ignore it, or it will succeed. We may wish to avoid stating the KeyPrint of the certificate that was actually received)
3.3. Optionally, when receiving a TLS handshake, if we know what the remote host's KeyPrint ought to be, we could also verify this, though it is pointed out that only the initiating side needs to check this for the man-in-the-middle protection to be valid; specifically the hub doesn't need to remember, or even understand, clients' KeyPrints.
4. Security Considerations
4.1.1. The KeyPrint in the adcs: url prevents a clandestine man-in-the-middle (certificate substitution) attack on the TLS session with the hub, if and only if the kp parameter hasn't been modified by an attacker.
4.1.2. Suggested countermeasures against this include the following:-
126.96.36.199. Even if otherwise unprotected, an attacker would likely only have a maximum of one opportunity to make the substitution (by changing the kp parameter when the URL is first sent to the user); if it's changed later, it will be detected. ("Casual first-use" security, as often used in SSH.)
188.8.131.52. Checking the fingerprint or transferring the hub URL out-of-band (for example by telephone, or in person) could reveal or thwart man-in-the-middle attacks; something's wrong if you don't have the same URL as the hub operator gives out. (PGP-style "sneakernet" security.)
184.108.40.206. We would be protected if the URL were in a file signed by some secure means by a party we have already decided we trust not to change the URLs maliciously. Encryption would not be necessary, only a signature to ensure no untrusted party has modified the KP fields. (A signed hublist, possibly.)
220.127.116.11. Finally, we would be protected were the hublist/URL retrieved from an https: site signed by a CA we trust not to issue signatures for any domain to someone who doesn't own that domain, and the owner of that domain trusted not to modify the KP fields. (SSL-style security.)
4.2.1. We are, in particular, trusting the hub not to maliciously modify the KP fields sent by the clients through their TINF broadcasts. However:-
the hub would gain nothing from swapping the KeyPrints of its users, as it would not gain anything from mounting a man-in-the-middle attack on its own users that it does not already have the ability to do, as it's the hub; and if we do not trust them enough not to maliciously modify the data, why are we choosing to connect to them and transmit our IP address through them? it could be detected by cross-checking with other clients/hubs, as connecting through a malicious hub would clearly show you to have a different KeyPrint to any other hub; therefore any hub doing this would obviously be quickly removed from hublists.
4.3.1. The certificates, including the name fields, are sent in the clear during the initial handshake. Therefore it is recommended to avoid identifying marks in the certificates CommonName fields (for example) that would clearly single them out as being TLS keys used by adcs:, and the CID field most definitely should not appear. Quite possibly no name fields should appear, or they should be blank.
[TIGR] - ADC specification, §7.1 - TIGR - Tiger tree hash support: http://adc.sourceforge.net/ADC.html#_tigr_...ee_hash_support
 Regarding the choice of TTH as a baseline hash; it is both convenient - being already present in the TIGR extension and in practice available in all ADC clients - and not insecure - having no known practical second-preimage attacks. It is most certainly not weaker than MD5 or SHA-1, which are often widely used elsewhere in TLS/SSL.
We note further that if a problem arose, then as with the rest of ADC, we could switch to another hash such as SHA-256, or a future SHA-3 algorithm.