forum.bittorrent.org

jch

Member

A few comments about µTP (BEP-29)

The µTP work done by the employees of BitTorrent, Inc., is an attempt at layering the peer-to-peer part of the BitTorrent protocol above a homebrew transport protocol rather than TCP.

µTP is meant to provide for background, or "less than best-effort" traffic, i.e. yield to competing TCP flows.  It does so by detecting impending congestion through delay measurements, similarly to TCP-Vegas and TCP-LP, and reducing its congestion window aggressively whenever delay increases.

Unfortunately, µTP was designed in secret, with no input from either the congestion control or the BitTorrent communities -- only very recently has BEP-29 been published, at a time when 3 major versions of µTP-enabled µTorrent had already shipped.  Because of that, it has not been subjected to any form of (informal) peer review, and, unsurprisingly enough, it has a number of flaws.

1. µTP doesn't allow repacketisation

Unlike TCP, µTP doesn't allow repacketisation of a resent segment.  This breaks PMTU-discovery, and has a number of unpleasant performance consequences in the presence of packet loss.  Please see http://forum.bittorrent.org/viewtopic.php?id=119 for more information.

2. µTP doesn't allow negotiation, not even in a weak form

It is not known how µTorrent decides whether contact a peer over TCP or µTP -- neither the tracker protocol nor the DHT nor the PEX protocol have been extended to provide information about the transport protocol being used.  It is has been claimed that µTorrent tries both protocols in parallel, which is wasteful both of network resources and peer memory.

Apparently, even the simple workaround of allowing a peer that does not implement µTP to reply with a packet saying "please try TCP instead" has not been considered.

3. µTP doesn't interoperate with legacy BitTorrent.

It is only possible to use µTP to speak with a peer that speaks µTP.  Since the congestion control algorithm in µTP is a sender-only modification of Van Jacobson, there is no fundamental reason for this limitation -- if the congestion control algorithm were used with TCP, one endpoint could be using VJ, and the other less than best-effort.

4. The congestion control algorithm used by µTP hasn't been studied

Because µTP was designed in secret, there are no publishes analyses of µTP's behaviour.

For example, µTP reacts to packet loss less conservatively than TCP -- it multiplies its congestion window by 0.78 rather than 0.5.  While this is fine when the bottleneck router uses a tail-drop queue, since the delay-based congestion control will kick in, it might or might not lead to congestion collapse when the bottleneck router uses an aggressive, loss-based AQM such as BLUE.

Please note that I am not claiming that µTP will actually exhibit such mis-behaviours.  I am merely claiming that in the absence of published data, there is no way to be confident that µTP will not exhibit gross mis-behaviour.

5. There are no provisions for ECN.

µTP, as currently defined, cannot make use of Explicit Congestion Notification.

6. µTP is an all-new transport protocol

All of the flaws above are due to µTP being a new transport protocol, rather than simply a modification to TCP.  Simply layering BitTorrent peer-to-peer traffic over TCP-LP solves all of the issues outlines above, while apparently meeting all of the design goals of µTP.

--Juliusz

arvid

Administrator

Re: A few comments about µTP (BEP-29)

The µTP work done by the employees of BitTorrent, Inc., is an attempt at layering the peer-to-peer part of the BitTorrent protocol above a homebrew transport protocol rather than TCP.

It doesn't replace TCP at the transport layer obviously.

µTP is meant to provide for background, or "less than best-effort" traffic, i.e. yield to competing TCP flows.  It does so by detecting impending congestion through delay measurements, similarly to TCP-Vegas and TCP-LP, and reducing its congestion window aggressively whenever delay increases.

There are some very important differences between the uTP congestion control mechanism and TCP vegas. The most important one being that uTP measures one-way delay, and not RTT. This makes it more accurate in measuring the up-stream congestion.

Unfortunately, µTP was designed in secret, with no input from either the congestion control or the BitTorrent communities

Why do you think there was no input from the congestion control community?

-- only very recently has BEP-29 been published, at a time when 3 major versions of µTP-enabled µTorrent had already shipped.  Because of that, it has not been subjected to any form of (informal) peer review, and, unsurprisingly enough, it has a number of flaws.

It hasn't been reviewed by people because it has been implemented in a released client? That seems like an unlikely causal relation.

1. µTP doesn't allow repacketisation

Unlike TCP, µTP doesn't allow repacketisation of a resent segment.  This breaks PMTU-discovery, and has a number of unpleasant performance consequences in the presence of packet loss.  Please see http://forum.bittorrent.org/viewtopic.php?id=119 for more information.

As I pointed out in the thread you refer to, this is not a problem. There are plenty of widely deployed protocols that don't do repacketisation and that rely on routers to be able to fragment packets. Now, that said, this doesn't mean that uTP necessarily relies on IP fragmentation. There's nothing preventing an implementation from doing PMTU discovery.

2. µTP doesn't allow negotiation, not even in a weak form

It is not known how µTorrent decides whether contact a peer over TCP or µTP -- neither the tracker protocol nor the DHT nor the PEX protocol have been extended to provide information about the transport protocol being used.  It is has been claimed that µTorrent tries both protocols in parallel, which is wasteful both of network resources and peer memory.

Apparently, even the simple workaround of allowing a peer that does not implement µTP to reply with a packet saying "please try TCP instead" has not been considered.

This is obviously outside of the scope for any transport protocol. You wouldn't expect TCP to support this either.

3. µTP doesn't interoperate with legacy BitTorrent.

It is only possible to use µTP to speak with a peer that speaks µTP.  Since the congestion control algorithm in µTP is a sender-only modification of Van Jacobson, there is no fundamental reason for this limitation -- if the congestion control algorithm were used with TCP, one endpoint could be using VJ, and the other less than best-effort.

Are you saying that it would be an actual realistic solution to use raw sockets, re-implement TCP, stick all our data that we need in an extension header, and go with that?

I don't think its reasonable to require end-users to run a client with root privileges.

If you're referring to some socket options to select transport protocol used with TCP, there are two problems. the LEDBAT congestion control algorithm isn't implemented in any operating system. The other less-than-ideal congestion control mechanisms are not implemented in the vast majority of operating systems.

4. The congestion control algorithm used by µTP hasn't been studied

Because µTP was designed in secret, there are no publishes analyses of µTP's behaviour.

For example, µTP reacts to packet loss less conservatively than TCP -- it multiplies its congestion window by 0.78 rather than 0.5.  While this is fine when the bottleneck router uses a tail-drop queue, since the delay-based congestion control will kick in, it might or might not lead to congestion collapse when the bottleneck router uses an aggressive, loss-based AQM such as BLUE.

Please note that I am not claiming that µTP will actually exhibit such mis-behaviours.  I am merely claiming that in the absence of published data, there is no way to be confident that µTP will not exhibit gross mis-behaviour.

See the IETF ledbad paper. And just because there are no papers ready today doesn't mean that it hasn't been studied.

5. There are no provisions for ECN.

µTP, as currently defined, cannot make use of Explicit Congestion Notification.

Did you mean "uTP, as currently defined, doesn't make use of ECN"? Obviously there's nothing stopping an implementation from doing so. At some point it might even be defined in the BEP.

6. µTP is an all-new transport protocol

All of the flaws above are due to µTP being a new transport protocol, rather than simply a modification to TCP.  Simply layering BitTorrent peer-to-peer traffic over TCP-LP solves all of the issues outlines above, while apparently meeting all of the design goals of µTP.

Please explain how it does so.

jch

Member

Re: A few comments about µTP (BEP-29)

The most important one being that uTP measures one-way delay, and not RTT.

As does TCP-LP.  And this still doesn't explain the need for a new transport encapsulation, rather than working within the framework of TCP.

There are plenty of widely deployed protocols that don't do repacketisation and that rely on routers to be able to fragment packets.

It's certainly not a good idea for bulk transport, as explained in detail in Mogul et al., which I cited in the other thread.

There's nothing preventing an implementation from doing PMTU discovery.

Please address the scenario that I outlined in the other thread.

the LEDBAT congestion control algorithm isn't implemented in any operating system.

I'm suggesting that you should implement a less than best-effort congestion control algorithm in your operating system of choice.  (Windows does have the ability to switch congestion control algorithms, since they are able to switch between Reno and CTCP.)

This approach has the following advantages:

  * it can be used by all applications, not only µTorrent;

and the following flaws:

  * it can be used by all applications, not only µTorrent.

--Juliusz

jch

Member

Re: A few comments about µTP (BEP-29)

Why do you think there was no input from the congestion control community?
[...]
the LEDBAT congestion control algorithm...

I find that I'm increasingly confused.

Could you please clarify the relation between LEDBAT and BEP-29?  For example, why does BEP-29 mandate multiplying by 0.78 on packet loss, while draft-shalunov-ledbat-congestion-00 clearly says

4.8. Halve on loss

   Further, to deal with severe congestion when most packets are lost
   and to provide a safety net against incorrect queuing delay
   estimates, we'll halve the window when a loss event is detected.
   We'll do so at most once per RTT.

arvid

Administrator

Re: A few comments about µTP (BEP-29)

The most important one being that uTP measures one-way delay, and not RTT.

As does TCP-LP.  And this still doesn't explain the need for a new transport encapsulation, rather than working within the framework of TCP.

There's nothing preventing an implementation from doing PMTU discovery.

Please address the scenario that I outlined in the other thread.

I did in that thread. All uTP connections start off with 300 byte packets, the packet size is not increased more often than every 10 seconds (in current uTorrent). That leaves plenty of time to probe the MTU before having to send payload using larger packets. Another potential way to do MTU discovery is to rely on IP fragmentation to complete the over-sized packets when the first ICMP packet too big is received.

the LEDBAT congestion control algorithm isn't implemented in any operating system.

I'm suggesting that you should implement a less than best-effort congestion control algorithm in your operating system of choice.  (Windows does have the ability to switch congestion control algorithms, since they are able to switch between Reno and CTCP.)

How would you work around the problem that not all operating systems have raw sockets? and that not all operating systems lets any user use raw sockets?

jch

Member

Re: A few comments about µTP (BEP-29)

All uTP connections start off with 300 byte packets, the packet size is not increased more often than every 10 seconds (in current uTorrent). That leaves plenty of time to probe the MTU before having to send payload using larger packets.

I'm confused.  Are you assuming that routing is static?

rely on IP fragmentation to complete the over-sized packets when the first ICMP packet too big is received.

Yep, that's my solution number 4 in the other thread.  However, according to one of your colleagues (on IRC), this is not what µTorrent does.

--Juliusz

arvid

Administrator

Re: A few comments about µTP (BEP-29)

I find that I'm increasingly confused.

Could you please clarify the relation between LEDBAT and BEP-29?  For example, why does BEP-29 mandate multiplying by 0.78 on packet loss, while draft-shalunov-ledbat-congestion-00 clearly says

The algorithm is the same, the multiplier differs. We may decide to change it if it causes any problem. So far it is believed that the multiplier doesn't play a very significant role, since packet loss experienced by uTP is not likely due to congestion. The exception is if we're already experiencing long delays, in which case the multiplier won't make a difference anyway, just like in TCP when your congestion window is 0, the multiplier doesn't matter.

As for the relationship between uTP and LEDBAT, one of the LEDBAT authors was the designer of the congestion control that we use in uTP and has been closely involved in uTP development.

jch

Member

Re: A few comments about µTP (BEP-29)

The exception is if we're already experiencing long delays, in which case the multiplier won't make a difference

Not necessarily.  If the bottleneck router is using an aggressively tuned low-delay AQM (say, very aggressive RED, or ordinary BLUE), then using a larger multiplier will starve Reno.

What I'm concerned about is that you'll be discouraging the deployment of low-latency AQMs, which is ungood for everyone.

--Juliusz

The 8472

Azureus Developer

Re: A few comments about µTP (BEP-29)

If you're referring to some socket options to select transport protocol used with TCP, there are two problems. the LEDBAT congestion control algorithm isn't implemented in any operating system. The other less-than-ideal congestion control mechanisms are not implemented in the vast majority of operating systems.

You don't need raw sockets or root/admin access for the application itself.

What is necessary is a kernel that allows you to add a congestion controller, under linux that's apparently possible. Windows has multiple congestion controllers too, but i'm not sure if microsoft provides any APIs to let developers roll their own.

So a userspace application with regular permissions could use TCP with a custom congestion controller that has to be loaded into the kernel with root/admin permissions.

I'm wondering why nobody even investigated that possibility.

The thing is that everyone would benefit from this solution. All applications interested in latency-preserving less-than-best-effort behavior could use it. Layer 4-awar Routers/other Network gear would still be able to rely on the known TCP properties and not everyone would have to re-implement the same thing over and over again.

---

Az dev

ghazel

BitTorrent Developer

Re: A few comments about µTP (BEP-29)

When asked to compare/contrast TCP-LP with uTP, Stansilav Shalunov said:

"
The uTP congestion control falls into the category of end-to-end
delay-based mechanisms.  TCP-LP is in the same category, as are more
well-known FAST TCP, TCP Vegas, TCP Westwood, and others.  Like uTP
and unlike many other delay-based congestion control mechanisms,
TCP-LP is designed to yield to standard TCP.  The similarity more or
less ends there.

The most crucial difference is that the design goal of uTP is to
soak the network without the user noticing any extra delay, while the
design goal of TCP-LP is to yield to TCP without regard for minimizing
the delay experienced by the user.  There are also many important
differences in implementation choices, but the fact the TCP-LP does
not set out to do what uTP tries to achieve is, of course, more
important.

                                   uTP         TCP-LP
Uses delay                     Yes         Yes
One-way delay               Yes         Yes
Designed to yield             Yes         Yes
Minimizes delay               Yes         No
TCP framing                   No           Yes
Kernel mods                   No           Yes
Robust estimators           Yes         No
Designed to converge      Yes         No
On-off congestion           No           Yes
Single-RTT reaction         Yes         ?*
Bounded extra delay        Yes         No**
Minimizes serialization       Yes         No
Tested over wide range    Yes***    No

* TCP-LP's reaction time is tied to the setting of gamma.  I don't
believe that any setting would produce desirable reactions, but TCP-LP
simply uses gamma that is independent of the congestion window
(gamma=1/8), thus defining reaction in terms of ACKs and not time.

** The extra delay implicitly targeted by TCP-LP, which does not have
an explicit target, is delta*(max-min).  Again, delta is a fixed
parameter (0.15 in the paper).  max-min is likely, for long flows, to
be the buffer size on the bottleneck device divided by the line rate.
For home connections, this is often a few seconds (I found 3-5 seconds
to be widespread).  With these settings, TCP-LP would implicitly
target 600ms extra delay, not counting extra serialization delay,
which might add 50ms or so for 1500-B packets.  Because of the gamma
smoothing, however, even this 650ms implicit target is not a bound on
the delay and may be exceeded.  In particular, at low rates, such as
ones typically found on home connections, it may be exceeded for
several RTTs.

*** The uTP congestion control has been tested and used on real
wide-area networks with similar parameter settings on connections with
speeds ranging over five decimal orders of magnitude.  The TCP-LP
paper describes simulations with a narrow range of bottleneck speeds
and no tests at all.
"

ghazel

BitTorrent Developer

Re: A few comments about µTP (BEP-29)

2. µTP doesn't allow negotiation, not even in a weak form

It is not known how µTorrent decides whether contact a peer over TCP or µTP -- neither the tracker protocol nor the DHT nor the PEX protocol have been extended to provide information about the transport protocol being used.  It is has been claimed that µTorrent tries both protocols in parallel, which is wasteful both of network resources and peer memory.

Apparently, even the simple workaround of allowing a peer that does not implement µTP to reply with a packet saying "please try TCP instead" has not been considered.

This sort of problem exists inside the same transport. When BitTorrent protocol encryption (http://en.wikipedia.org/wiki/BitTorrent … encryption) was added, no negotiation as you suggest was implemented either. Extended information was added to PEX, and likewise uTP has a similar extension in PEX to indicate that the peer supports uTP (bit 4). Since a large percentage of peers out there are uTorrent which does support uTP, we have found the memory and network waste of trying both simultaneously to be insignificant compared to the benefits of trying one at a time or waiting until trackers around the world add some sort of extension. Clients obviously can implement this however they like, and it's not related to uTP itself.

3. µTP doesn't interoperate with legacy BitTorrent.

It is only possible to use µTP to speak with a peer that speaks µTP.  Since the congestion control algorithm in µTP is a sender-only modification of Van Jacobson, there is no fundamental reason for this limitation -- if the congestion control algorithm were used with TCP, one endpoint could be using VJ, and the other less than best-effort.

Unfortunately implementing a modified congestion controller in Windows would require Administrator access to install, and could crash the entire OS instead of just uTorrent if there is a bug. This was not considered worth it. You're right that we're giving up some benefit by not using our congestion control with legacy TCP clients, but we are also gaining some benefit and not causing any significant problems -- peers can still connect to uTorrent over TCP. We actually go even farther to rate-limit our TCP sessions based of information from active uTP connections.

4. The congestion control algorithm used by µTP hasn't been studied

Because µTP was designed in secret, there are no publishes analyses of µTP's behaviour.

Yes. Also the congestion control of the Skype protocol (http://en.wikipedia.org/wiki/Skype_protocol) has not been studied either, since it was designed in secret. This happens. However we did, unlink Skype, decided to publish the spec and bring it to the IETF.

6. µTP is an all-new transport protocol

All of the flaws above are due to µTP being a new transport protocol, rather than simply a modification to TCP.  Simply layering BitTorrent peer-to-peer traffic over TCP-LP solves all of the issues outlines above, while apparently meeting all of the design goals of µTP.

TCP-LP did not meet our design goals (see above post).

The 8472

Azureus Developer

Re: A few comments about µTP (BEP-29)

3. µTP doesn't interoperate with legacy BitTorrent.

It is only possible to use µTP to speak with a peer that speaks µTP.  Since the congestion control algorithm in µTP is a sender-only modification of Van Jacobson, there is no fundamental reason for this limitation -- if the congestion control algorithm were used with TCP, one endpoint could be using VJ, and the other less than best-effort.

Unfortunately implementing a modified congestion controller in Windows would require Administrator access to install, and could crash the entire OS instead of just uTorrent if there is a bug. This was not considered worth it. You're right that we're giving up some benefit by not using our congestion control with legacy TCP clients, but we are also gaining some benefit and not causing any significant problems -- peers can still connect to uTorrent over TCP. We actually go even farther to rate-limit our TCP sessions based of information from active uTP connections.

Considering that installing anything into %programfiles% requires administrator permissions anyway i see the permissions issue as not really significant.

System stability of course is a concern, but given the low complexity of a congestion controller and the fact that it would only be used by applications specifically requesting it via socket options it would be trivial to not use it in case problems arise.

LEDBAT plugged into TCP vs. µTP
Disadvantages:
-potential stability issues with kernel drivers
Advantages:
-it only has to be implemented once (or thrice, if you consider win,linux,osx), making it available for all bittorrent apps and possible other interested parties, too. thus reducing the necesasry time to get it deployed by all bittorrent clients
-all network components that are TCP-aware can still work on that basis
-only congestion control behavior is changed, no new pitfalls due to a new transport protocol
-not reinventing the wheel

---

Az dev

ghazel

BitTorrent Developer

Re: A few comments about µTP (BEP-29)

Unfortunately implementing a modified congestion controller in Windows would require Administrator access to install, and could crash the entire OS instead of just uTorrent if there is a bug. This was not considered worth it.

Considering that installing anything into %programfiles% requires administrator permissions anyway i see the permissions issue as not really significant.

uTorrent can be installed to any directory, and does not require access to %programfiles%. Since some users do use it this way, especially users who are not administrators of the network they are affecting, it does not make sense to exclude them from the benefits of improved congestion control.

System stability of course is a concern, but given the low complexity of a congestion controller and the fact that it would only be used by applications specifically requesting it via socket options it would be trivial to not use it in case problems arise.

Are you volunteering to implement it? We decided creating the protocol on an application layer was a quicker and easier way to get the transport up and running for uTorrent. Plus, we got the added benefit of UDP hole punching -- a benefit Azureus sees from its UDP based transport as well, no?

jch

Member

Re: A few comments about µTP (BEP-29)

The most crucial difference is that the design goal of uTP is to soak the network without the user noticing any extra delay, while the design goal of TCP-LP is to yield to TCP without regard for minimizing the delay experienced by the user.

Thanks for all the info, ghaziel (and StaÅ›).  I am convinced that performing less than best-effort congestion control is a good idea, and the intuitions you provide confirm that Ledbat is a good idea.

However, I'd like to remind everyone that all but one of my objections are about the µTP framing; the one the one that isn't is about the difference between µTP's and Ledbat's reaction to loss.  My claim here is that properly reacting to loss remains essential even when you react to delay, which appears to be confirmed by Figure 3 in [1].

--Juliusz

[1] Dario Rossi, Claudio Testa, Silvio Valenti, Paolo Veglia, Luca Muscariello.  News from the Internet congestion control world.  http://arxiv.org/pdf/0908.0812

Last edited by jch (2009-11-09 09:41:30)

jch

Member

Re: A few comments about µTP (BEP-29)

Extended information was added to PEX, and likewise uTP has a similar extension in PEX to indicate that the peer supports uTP (bit 4).

Thanks, that's interesting info.  Where is it documented?

Also the congestion control of the Skype protocol has not been studied either, since it was designed in secret.

Am I reading you right?  Since other companies are behaving like thugs, we have no right to complain about BitTorrent, Inc., behaving badly?  Oh, my.

--Juliusz

The 8472

Azureus Developer

Re: A few comments about µTP (BEP-29)

Unfortunately implementing a modified congestion controller in Windows would require Administrator access to install, and could crash the entire OS instead of just uTorrent if there is a bug. This was not considered worth it.

Considering that installing anything into %programfiles% requires administrator permissions anyway i see the permissions issue as not really significant.

uTorrent can be installed to any directory, and does not require access to %programfiles%. Since some users do use it this way, especially users who are not administrators of the network they are affecting, it does not make sense to exclude them from the benefits of improved congestion control.

But neither does it make sense to implement a congestion controller - which is clearly a transport layer issue, not application layer - over and over again for every single bittorrent application (or potentially anything else that wants to use LEDBAT), does it?

By deploying it on the application level you're making it less likely for OS developers (or anyone who could write a kernel module) to implement it into network stack... the temporary solution takes away the incentive to deliver a permanent one in a timely manner.

My concern is that while everyone would like to see it in TCP they (you) rush to a solution due to short-term cocerns and once it's achieved nobody will put any significant effort into the long-term aspects. And we will have to live with yet another hack that creates more and more work to maintain for years to come.

System stability of course is a concern, but given the low complexity of a congestion controller and the fact that it would only be used by applications specifically requesting it via socket options it would be trivial to not use it in case problems arise.

Are you volunteering to implement it? We decided creating the protocol on an application layer was a quicker and easier way to get the transport up and running for uTorrent. Plus, we got the added benefit of UDP hole punching -- a benefit Azureus sees from its UDP based transport as well, no?

I would like to, but I have 0 knowledge about microsoft's kernel architecture or programming kernel modules.

I am assuming that there would be more qualified people to do that.

As far as hole punching goes... i see that - like µTP - as a transitional measure at best since IPv6 will obviate the need for NAT traversal..

---

Az dev

arvid

Administrator

Re: A few comments about µTP (BEP-29)

My concern is that while everyone would like to see it in TCP they (you) rush to a solution due to short-term cocerns and once it's achieved nobody will put any significant effort into the long-term aspects. And we will have to live with yet another hack that creates more and more work to maintain for years to come.

I believe the opposite is true. I believe uTP and the IETF LEDBAT initiative helps pushing for this kind of congestion control being standardized and deployed. My understanding of the IETF is quite limited, but I would assume that actual deployment of a congestion control algorithm is considered slightly better than simulation results, which I think helps creating confidence in it.

As far as hole punching goes... i see that - like µTP - as a transitional measure at best since IPv6 will obviate the need for NAT traversal..

I don't think there's anything wrong with short-term solutions. We have an interest in making our software work today. I also think that different people may make different estimates on _how_ short-term these solutions are.

I think that it will be quite some time before we have delay based TCP CC standardized and deployed in a significant portion of the world's computers. Just the deployment part will probably take many many years.

User294

Member

Re: A few comments about µTP (BEP-29)

Yet another comment about uTP. According to some ISPs measurements, uTP usage has caused awful packets number growth in their networks.

This causes unpleasant issues:
1) Overhead is increased. So there is less useful data and more protocol headers travel over wires.
2) UTP uses small packets. That's a mistake for bulk file transfer protocol. This causes increased load on routers. Many kinds of routers are actually can't handle large numbers of packets well enough, especially software routers used by some ISPs and SOHO routers used by users. So, uTP actually jams some routers.
3) What worse: when network is congested due to huge packets numbers, uTP seems to decrease packet size, flooding network by dozens of even smaller packets in even larger quantity and making things even worse. Result? Some networks are collapsing due to this stupid property of uTP.

So what? Many ISPs in my country have already started filtering uTP at the moment due to inadequate numbers of packets flooding their networks.

So I can suggest some kind of MTU discovery and using large packets. As large as possible. Please, never use small packets for bulk data xfers. There is no need to minimize delays of torrent data. There is need to minimize impact on other kinds of data and on user's day to day activity, etc.

rafi

Member

Re: A few comments about µTP (BEP-29)

Confirmed. Others on the uTorrent forum (and myself) have also reported high overhead  with uTP. You can also see some measurements I did - over here .

So I can suggest some kind of MTU discovery and using large packets. As large as possible. Please, never use small packets for bulk data xfers. There is no need to minimize delays of torrent data. There is need to minimize impact on other kinds of data and on user's day to day activity, etc.

You can comment on my suggestion for that, detailed over here:
http://forum.utorrent.com/viewtopic.php?id=72106

Pacing of large/MTU-sized  packets will do good to both congestion and overhead !

ghazel

BitTorrent Developer

Re: A few comments about µTP (BEP-29)

Pacing of large/MTU-sized  packets will do good to both congestion and overhead !

That's not true. Larger packets take longer to send, which results in a slower reaction to congestion. Smaller packets at lower speeds were chosen specifically after measuring the impact on congestion controller responsiveness.

Also, packet size and size-determining algorithms have been significantly improved since any of these measurements were performed. Please wait until 2.0.1 is released as a stable build before assuming that measurements in the wild will take these in to account. The BEP-29 document does not reflect the newest modifications, although I should note that BEP-29 does not define which packet sizes or techniques are required. An implementor may chose to use any size packet they like.

So, if you have comments about the way uTorrent behaves, please take that to the uTorrent forum.

jch

Member

Re: A few comments about µTP (BEP-29)

Rafi:

> Pacing of large/MTU-sized  packets will do good to both congestion and overhead !

Ghazel:

> That's not true. Larger packets take longer to send, which results in a slower reaction to congestion.

...and higher latency due to link access times.  Yes, we know.

The truth, of course, is that there is a tradeoff in choosing the optimal packet size, which depends on many factors.  Established wisdom is that, at current network speeds, the optimal packet size is somewhere around a few kB, which would imply that sending maximum sized packets on Ethernet is a good idea.

Your claims would seem to contradict the above "well-known" folklore, which I find very interesting.  Could we please see your measurements?

--Juliusz

rafi

Member

Re: A few comments about µTP (BEP-29)

ghazel wrote:
...before assuming that measurements in the wild will take these in to account

Measuring on a LAN with 'brand new' 2.01 instances (as described) can be hardly called "in the wild". You should deduct (some) conclusions from such measurement before release and not in the wild, after it's released...

...The BEP-29 document does not reflect the newest modifications

well than, it should. Why don't you just update it for us all to observe ?

if you have comments about the way uTorrent behaves, please take that to the uTorrent forum

http://forum.utorrent.com/viewtopic.php?id=72106 (and others) -> is still open for discussion, also for the devs ...

jch wrote:
...and higher latency due to link access times.  Yes, we know.

In my checks,  uT is showing completely different uTP "implementation"/behavior when  seeding, in cases of self controlled/limited speed, and when speed is limited externally. In the first case (recommended in the uT setup guide) - packets sizes are significantly lower (and btw, also when forced to be large).  So, reasons are not always only congestion control related, but seem to be also general-speed-control related.

Last edited by rafi (2010-04-08 22:08:01)

ghazel

BitTorrent Developer

Re: A few comments about µTP (BEP-29)

In my checks,  uT is showing completely different uTP "implementation"/behavior when  seeding, in cases of self controlled/limited speed, and when speed is limited externally. In the first case (recommended in the uT setup guide) - packets sizes are significantly lower (and btw, also when forced to be large).  So, reasons are not always only congestion control related, but seem to be also general-speed-control related.

Your experiment here is determining nothing about uTP packet size algorithms during bulk transfer. You're observing small packets because you are rate limiting to a very low limit (internet speeds) on a very fast line (LAN speeds). This produces small packets because our Nagle implementation has a very short timeout and your window is always empty.

This is, as well, unrelated to BEP29.

rafi

Member

Re: A few comments about µTP (BEP-29)

Limited-speed seeding tests (being controlled by the 2.01 seeder itself) *were* demonstrated/conducted to the "wild", using the new uTP header, and with similar results (like in test #2 here)

And the uT thread is still there to comment on ...

jch

Member

Re: A few comments about µTP (BEP-29)

> This produces small packets because our Nagle implementation has a very short timeout and your window is always empty.

I think you're confused.  There's no timeouts in Nagle's algorithm.

--Juliusz