forum.bittorrent.org

ghazel

BitTorrent Developer

Re: A few comments about µTP (BEP-29)

> This produces small packets because our Nagle implementation has a very short timeout and your window is always empty.

I think you're confused.  There's no timeouts in Nagle's algorithm.

Er, sorry. I was thinking of Delayed ACK.

jch

Member

Re: A few comments about µTP (BEP-29)

>> I think you're confused.  There's no timeouts in Nagle's algorithm.

> Er, sorry. I was thinking of Delayed ACK.

I'm not sure I understand how delacks can have an effect on packet size.

--Juliusz

ghazel

BitTorrent Developer

Re: A few comments about µTP (BEP-29)

>> I think you're confused.  There's no timeouts in Nagle's algorithm.

> Er, sorry. I was thinking of Delayed ACK.

I'm not sure I understand how delacks can have an effect on packet size.

They can't, I was misremembering.

rafi

Member

Re: A few comments about µTP (BEP-29)

ghazel wrote:
...This produces small packets because our Nagle implementation has a very short timeout and your window is always empty.
...
Er, sorry. I was thinking of Delayed ACK.

Well, I did look @ Wikipedia for timeouts,  and thought I could not find those cause of my poor English skills... 

I guess, that since Nagle is ruled out, that leaves us with a possible  reason for using very small packets (over loop-back, and even when the user specifies non-dynamic mode and use of   large ones initially) - the way your speed-limiter works (or maybe inappropriate packets' pacing logic ?) ,  false detection of congested conditions, or possibly - also a simple bug ?

jch wrote:
...the optimal packet size is somewhere around a few kB, which would imply that sending maximum sized packets on Ethernet is a good idea.

Your claims would seem to contradict the above "well-known" folklore, which I find very interesting.  Could we please see your measurements?

ghazel wrote:
...Please wait until 2.0.1 is released as a stable build ...

This is released, and  I guess a fix-release is probably on it's way next, why not try and fix/improve the size handling as well, and maybe get more efficient traffic too ?

ghazel

BitTorrent Developer

Re: A few comments about µTP (BEP-29)

I guess, that since Nagle is ruled out, that leaves us with a possible  reason for using very small packets (over loop-back, and even when the user specifies non-dynamic mode and use of   large ones initially) - the way your speed-limiter works (or maybe inappropriate packets' pacing logic ?) ,  false detection of congested conditions, or possibly - also a simple bug ?

This is released, and  I guess a fix-release is probably on it's way next, why not try and fix/improve the size handling as well, and maybe get more efficient traffic too ?

Please take these questions to the uTorrent forums.

rafi

Member

Re: A few comments about µTP (BEP-29)

As I've already mentioned,  this has  already been done long ago, and any dev is (still) welcome to respond, also to the proposed speed-limiter algorithm.
But, nevertheless, I'll be glad to post one more time: http://forum.utorrent.com/viewtopic.php?id=74094

followup thread on uTP performance @ uTorrent.com "Protocol Design Discussion":
http://forum.utorrent.com/viewtopic.php … 28#p478028

Last edited by rafi (2010-04-23 08:52:01)

ghazel

BitTorrent Developer

Re: A few comments about µTP (BEP-29)

Continuing the discussion from http://trac.transmissionbt.com/ticket/2338#comment:38

which can be avoided simply by layering LEDBAT over TCP

This is the basic issue. I think you are misjudging how "simple" this task is. I would love to see Linux, Windows (don't forget XP, the most popular version), and OS X support LEDBAT on TCP, but the truth is we probably have years before they even think about it, and years after that before they all get it right and compatible and stable. That is assuming they want to, and I believe the hinges strongly on adoption of LEDBAT by a very large application which is in strong need of it, i.e. BitTorrent. You could go down the road of trying to implement the TCP extension yourself on each platform as some sort of filter, if that is even possible on each major OS, but you risk users not installing it due to privileges, breaking existing TCP traffic unrelated to a BitTorrent client, and generally blue screening / kernel panicing the machine.

Investing time in an albeit simplified framing layer on UDP has served us quite well for the past several years already and is available to every user on every platform. Please have a look at the source for our implementation: http://github.com/bittorrent/libutp . If you can find edge cases where it performs poorly today, please point them out and/or submit patches.

jch

Member

Re: A few comments about µTP (BEP-29)

> I think you are misjudging how "simple" [including LEDBAT congestion control in TCP] task is.

It's certainly not difficult under Linux.  It's certainly possible under Windows 2000 and later (which allows switching between CTCP and NewReno congestion control).  I cannot fathom why it should be difficult under Mac OS X.

Obviously, this would make LEDBAT available to all applications running on the system, not only µTorrent, and I can understand why you might not want that.

> Investing time in an albeit simplified framing layer on UDP has served us quite well for the past several years already

That would be fine if you were advertising µTP as « a short-term hack to work around our unwillingness to do things properly ».  But that's not what you are doing -- you are actively promoting µTP as though it were a good thing.

> please [...] submit patches.

Sure.  Where shall we discuss my consulting fees?

--jch

ghazel

BitTorrent Developer

Re: A few comments about µTP (BEP-29)

> I think you are misjudging how "simple" [including LEDBAT congestion control in TCP] task is.

It's certainly not difficult under Linux.  It's certainly possible under Windows 2000 and later (which allows switching between CTCP and NewReno congestion control).  I cannot fathom why it should be difficult under Mac OS X.

"certainly possible" and "cannot fathom" are not exactly confidence builders. To my knowledge this has never been done before by a third party.

Obviously, this would make LEDBAT available to all applications running on the system, not only µTorrent, and I can understand why you might not want that.

This is fine with me. Preferred, actually. I would love applications like autoupdate, online backup, and other bulk and non-realtime applications to use LEDBAT instead.

> Investing time in an albeit simplified framing layer on UDP has served us quite well for the past several years already

That would be fine if you were advertising µTP as « a short-term hack to work around our unwillingness to do things properly ».  But that's not what you are doing -- you are actively promoting µTP as though it were a good thing.

It has proved to be a good thing. I am unable to say exactly how short term it is - although my estimaton of years + years earlier is probably a good one. Again, you're misjudging our specific choice to implement uTP in a more widely usable and more easily installable way as "unwillingness to do things properly". You're advocating at least three pieces of software you're not even sure are possible, because it seems like "the right way" -- I would gladly pay your consulting fees if you had any experience implementing them. We have yet to find such a person.

Projects like libswift ( http://libswift.org ) have used LEDBAT over UDP without TCP-like in-order delivery at all. That is entirely valid, and does not require modifying the operating system's functionality with regard to TCP. Did they do things properly? I think so.

jch

Member

Re: A few comments about µTP (BEP-29)

> [Making LEDBAT available to all apps by implementing it within TCP] is fine with me. Preferred, actually.

Then why don't you do it?  For goodness' sake, ghazel, you're BitTorrent, Inc., the one company directly responsible for more traffic on the Public Internet than all others put together.  If there's one entity that has the clout needed to negotiate including LEDBAT within the congestion controller in Microsoft Windows, it's you.

Have you at least asked Microsoft what it would take to implement LEDBAT within their TCP code?

--jch

The 8472

Azureus Developer

Re: A few comments about µTP (BEP-29)

While i strongly agree with the point that LEDBAT should be used as TCP congestion controller instead over yet-another-UDP-transport i want to remind people that we should cooperate here to improve things.

Having heated arguments over each others' policies will unlikely lead to any improvement.

To be realistic here... it is a lengthy process...

If we want to push LEDBAT to microsoft/linux/osx it first has to be standardized, i.e. the RFC needs to be finalized. Once that has happened we'd need someone to write a linux kernel module (since getting the kernel devs to accept it is probably the easiest way) including TCP socket parameters for controlling it... which BT Inc probably has little interest in since they mostly write for windows at the moment.

Once that has happend we could knock on apple's and microsoft's doors.

Of course some of these steps could be started prepared of time, if anyone is willing to do so.

---

Az dev

ghazel

BitTorrent Developer

Re: A few comments about µTP (BEP-29)

> [Making LEDBAT available to all apps by implementing it within TCP] is fine with me. Preferred, actually.

Then why don't you do it?  For goodness' sake, ghazel, you're BitTorrent, Inc., the one company directly responsible for more traffic on the Public Internet than all others put together.  If there's one entity that has the clout needed to negotiate including LEDBAT within the congestion controller in Microsoft Windows, it's you.

As I already said, Administrator privileges and potential for crashing the entire machine or disrupting other TCP traffic have discouraged us from taking this route. We chose UDP instead, and that worked great.

Have you at least asked Microsoft what it would take to implement LEDBAT within their TCP code?

I am aware of Microsoft developers monitoring the LEDBAT work at the IETF closely. Again, this whole process takes a very significant amount of time.

User294

Member

Re: A few comments about µTP (BEP-29)

ghazel> That's not true. Larger packets take longer to send, which results
ghazel> in a slower reaction to congestion. Smaller packets at lower speeds were chosen
ghazel> specifically after measuring the impact on congestion controller responsiveness.
In "link-local" scale that's somewhat correct. So, you solved one particular congestion problem: slow DSL user with poor uplink speed could have a bad ping times when upload channel saturated by torrent (so everything getting horribly slow). The "only" problem is that it looks like if you has failed to evaluate what happens if network congestion point happens to be anywhere else and/or due to another reasons. Or that there are also other link types with other properties and issues, etc. And that except local link there are more global issues could happen.

Many routers (firewalls, IDSes, etc), especially "software" ones are sensitive to packets per second amount (PPS). If router's CPU is relatively weak compared to link speed, rouer could limit link's speed because router's CPU could fail to route too many packets per second. In such scenario, router performance depends on packet size. Usually, router's CPU can handle roughly some N packets per second at most and not more than that. Attempt to exceed this would lead to CPU use = 100% and dropped packets. In this case, greater packets lead to a better overall speed. Only few high-profile (and hence expensive) routers are capable of maintaining "wire" link speeds while dealing with small packets. So what? Simple: uTP seems to detect such scenarios as "congestion" as well (router starts to drop packets if there are too many of them so CPU fails to handle them in real time). And then lools like uTP reduces packets size further. This action is not smart because it does not necessarily leads to a reduced load on router's CPU (it not looks like uTP really cares about reducing PPS as well). Result? Worse up and down speeds due to router hammering with a small packets. So certain link types are known to be in trouble with uTP. Many users on torrent forums have reported that disabling uTP leads to a much better UL/DL speeds. I guess uTP should not resort to very short packets at all and rather attempt to reduce it's overall speed (actually, limiting speed of TCP-based torrent to some 80% of channel capablities is enough to get ping times and overall network behavior much better on slow DSL link).

What worse: similar effects could happen in a global scale. It looks like if some part of network as whole happens to be congested, uTP will also reduce packet sizes. First of all, this action could further degrade performance of routers/firewalls/IDSes on the way, making things even worse. Since users would start hammering them with a small packets, increasing load and further reducing achievable performance. Then, users usually still want to xfer same amount of data, but there will be more headers sent for same amount of data. So more data have to be transferred in total over network for same result. Result? Some networks were seriously jammed and some ISPs have even filtered out uTP negotiation as "emergency" countermeasure to keep their networks from collapsing due to huge PPSes. The primary problem is that uTP can increase PPS by 2-3 times and not each and every network equipment can easily handle this "gift". And many such  troubles would lead to collapse scenario when uTP would reduce packet size even further so more load so packet size goes even smaller, etc. The result Networks could run in suboptimal conditions, filled by unreasonably small packets for no reason.

IMHO, these effects should be taken into account.

jch> sending maximum sized packets on Ethernet is a good idea.
Not exactly. Some PPTP/PPPoE or other similar links may run over Ethernet but result will have have MTU slightly less than 1500 bytes. Say, MTU of 1492 or 1488 bytes is not uncommon. Care should be taken here to avoid fragmentation on most link types and flavors.

rafi

Member

Re: A few comments about µTP (BEP-29)

> MTU slightly less than 1500 bytes ...
PMTUD should do the job and  take care of the proper MTU ...

Last edited by rafi (2011-01-13 10:40:49)

jch

Member

Re: A few comments about µTP (BEP-29)

> PMTUD should do the job

There's no PMTUD in libutp.

--jch

rafi

Member

Re: A few comments about µTP (BEP-29)

> There's no PMTUD in libutp.

So, why don't you just ask Arvid to put his code (now running successfully in v2.2.1) in there ? ...

-Rafi