As some of you already know, I've spent much of the last few weeks working on ÂµTP support in Tramsmission -- see this posting for more information. I'm now struggling with deciding when to use ÂµTP for outgoing connections, which is what some will remember as item (2) of my original critique of ÂµTP.
I believe that what ÂµTorrent does is to alway try ÂµTP first, and only fallback to TCP on timeout (can anyone confirm that?). If done naively, this has the effect of strongly favouring ÂµTP-supporting peers over TCP-only ones, as TCP is only attempted after a timeout, at which point all of our connection slots may be used up by ÂµTP connections. A simple mitigation for that is to keep a slot reserved during the timeout -- if a peer got a connection slot, it cannot lose it to a ÂµTP-supporting peer during the timeout/fallback sequence.
I'm trying to implement some further mitigations to prevent advantaging ÂµTP peers too much. The first is to skip the ÂµTP phase when a peer was obtained over PEX without the ÂµTP PEX flag, and connect directly over TCP in this case. Pretty obvious, I guess, and it only helps with PEX peers.
The other mitigation is to associate with every peer a "utp_failed" flag which is set when we fallback to TCP. This way the ÂµTP advantage exists the first time we connect to a peer; when reconnecting to a previously connected peer, we know how to contact this peer straight away.
Trouble is -- this flag sometimes gets set spuriously. What happens is that ÂµTP only sends 4 SYNs before giving up, so in times of congestion we might be spuriously falling back to TCP, and setting the utp_failed flag. So, paradoxically, ÂµTP which is supposed to help with congestion is not used when there is congestion.
So do people have any ideas on when to reset the utp_failed flag? Right now, I'm resetting it when a peer announces the "ut_holepunch" extension, since it implies support for ÂµTP, but that leaves Transmission, libtorrent and KTorrent peers with no way to signal support for ÂµTP. Shall I define a new "tr_utp" extension, which signals that a peer speaks ÂµTP, or shall I define a more general "tr_transport" extension that can signal the transport protocols that a peer speaks (including "look, you've contacted me over ÂµTP, but in the future please know I really prefer TCP").
I'm looking at you, Arvid, since none of this is useful unless implemented by libtorrent.
I'm not sure about the importance of resetting such a flag, since it is only set after connecting to a peer at least once. So it would depend on likely it is that we connect to a peer more at once AND keep state information about that peer in memory/on disk.
Assuming reconnects happen frequently then we can just use an exponential backoff (based on the # of connection attempts) to determine the interval at which we'll probe the utp capability.
And if we assume reconnects don't happen often then it's not really all that important and we can still leave exponential backoff as placebo.
Adding flags to the LTEP handshake is relatively cheap when it comes to compatibilty, so i have no objection to that approach. I think a simple "supports utp" flag will be sufficient, expressing preferred protocols rarely works if preferences are conflicting.
I already answered your suspicions a while ago
I don't believe ÂµTorrent is doing what you describe. Opening both ÂµTP and TCP at the same time would be a waste of resources, and would leave hundreds of sockets in TIME_WAIT.
Please check the code, Ultima.
ÂµTorrent does in fact try TCP and ÂµTP at the same time. Sometimes ÂµTP happens first because of the half-open limit, but ÂµT generally tries to do both. It will keep waiting for a response to ÂµTP SYNs for I think about 60 seconds and drop an established TCP connection if it gets a response.
Please read the rules before posting.
Oh my... I'm at a loss for words.
Okay, so here's my proposal.
A peer that supports this extension includes an entry "tr_transport" in the top-level dictionary of the LTEP handshake. This entry's value is a list of the following strings:
* "tcp" if the peer accepts incoming TCP connections;
* "utp" if the peer accepts incoming ÂµTP connections.
Any other values in the list are reserved for future extensions, and should be silently ignored.
Note that the "tcp" and "utp" strings are for *incoming* connections. Hence, a peer that knows that it is behind a firewall and hence cannot accept incoming TCP connections may choose to omit the string "tcp".
Here's a quick summary of the explanations on IRC.
Greg has confirmed that ÂµTorrent tries ÂµTP and TCP in parallel; he said that he hasn't noticed any issues from TIME_WAIT sockets. (I'm seeing anywhere between 5 and 70 TIME_WAIT sockets per torrent, btw. Filling the TIME_WAIT table would be pretty bad, since it would give an huge advantage to ÂµTP peers.)
The strategy I'm using in Transmission is the following. On initial connect, we try ÂµTP first, and then fall back to TCP if we time-out (after 30 seconds). In order to avoid giving too much of an advantage to ÂµTP peers, the connection slot is reserved, and cannot be stolen by a ÂµTP peer while we wait for the time-out.
In order to avoid the 30-second time-out on a subsequent reconnect, we maintain a utp_failed boolean flag associated with the address. When we fall back to TCP, utp_failed is set, after which the given peer will only ever be contacted over TCP.
I've found that utp_failed gets set spuriously fairly often. That's no big deal, it simply means that some peers will only be contacted over TCP (we'll still accept incoming ÂµTP from them, and reset the flag if that ever happens), but it'd be good to have some way to reset a spuriously-set utp_failed flag. For ÂµTorrent peers, there's the ut_holepunch extension, which I'm assuming indicates support for ÂµTP; but what about Transmission, libtorrent and KTorrent?
Last edited by jch (2011-01-20 15:02:54)