forum.bittorrent.org

BitTorrent.org community

You are not logged in.

Announcement

Forums are closed. Use the new mailing list! https://groups.google.com/a/bittorrent.com/forum/#!forum/bt-developers

#1 2011-04-27 18:07:22

TheRook
Member

UDP+Compression

HTTP has one major advantage over UDP,  and that is compression.   If UDP supported a gzip 9 like HTTP,  then even more bandwidth could be saved.  Gzip have a better compression rate the larger the file.  This has the effect of reducing the max load per request on a tracker.

I took the raw bencoded announce that was 27,718 bytes in size (AND ONLY THE ANNOUNCE, LEAVING THE REST OF THE PROTOCOL BEHIND. ).   I then did a gzip 9 and it became a 27,657 byte file.   This was a savings of 61 bytes.  Now think of savings with 15.444.146 seeders + 7.676.410 leechers.  The total comes to 1.4 Gigabytes for each announce cycle!

Last edited by TheRook (2011-04-29 08:14:57)

Offline

#2 2011-04-28 10:06:07

The 8472
Azureus Developer

Re: UDP+Compression

udp announces use fewer packets than establishing a tcp connection and they are in a binary-encoded format instead of bencoded which is less compressible.

There is nothing to be gained here.


Az dev

Offline

#3 2011-04-28 17:28:33

TheRook
Member

Re: UDP+Compression

@The 8472  UDP and compression are not mutually exclusive.   The benefit of this feature was proved in my post.

Last edited by TheRook (2011-04-28 17:29:39)

Offline

#4 2011-04-29 03:59:16

DreadWingKnight
CBTT/BNBT Developer

Re: UDP+Compression

You can't compare compressing an http request to compressing a udp one and expect the math to come out the same. The communication protocols are different.


Guy with a few torrent programs under his belt.

Offline

#5 2011-04-29 08:03:06

TheRook
Member

Re: UDP+Compression

DreadWingKnight You 100% correct,  thats why i only compressed the announce response.    Also in HTTP a gzip'ed http response only compresses the body.  But that's totally unrelated to UDP+gzip9.


So as it stands I haven't gotten a single valid response to any of my posts on this form.  I'm disappointing to say the least.

Last edited by TheRook (2011-04-29 08:16:35)

Offline

#6 2011-04-29 08:39:16

DreadWingKnight
CBTT/BNBT Developer

Re: UDP+Compression

DreadWingKnight You 100% correct,  thats why i only compressed the announce response.    Also in HTTP a gzip'ed http response only compresses the body.  But that's totally unrelated to UDP+gzip9.

You completely ignore the fact that the responses and requests are BOTH formatted differently in HTTP vs UDP.

The response in HTTP may be compressible where the response in UDP is not.


Guy with a few torrent programs under his belt.

Offline

#7 2011-04-29 11:17:32

TheRook
Member

Re: UDP+Compression

UDP isn't magic and all data is compressible.  Your right UDP doesn't use bencodeing,  which makes me wonder why bencoding is still being used in HTTP...

Last edited by TheRook (2011-04-29 11:18:07)

Offline

#8 2011-04-29 11:58:36

DreadWingKnight
CBTT/BNBT Developer

Re: UDP+Compression

Because the HTTP announce protocol is designed to be highly extensible.
BEncoding allows that. UDP trackers don't.

Additionally, any tracker giving 27kbyte announce replies are giving out THOUSANDS of peers in a single reply. This is a misconfiguration.


Guy with a few torrent programs under his belt.

Offline

#9 2011-04-29 14:03:46

TheRook
Member

Re: UDP+Compression

Private trackers need to announce a full list of peers because a client in the private swarm won't connect to a peer not on the list as per specification (http://www.bittorrent.org/beps/bep_0027.html).  Regardless of encoding this large list is perfect candidate for compression.

Last edited by TheRook (2011-04-29 14:06:34)

Offline

#10 2011-04-29 20:20:05

DreadWingKnight
CBTT/BNBT Developer

Re: UDP+Compression

Uhh.

No.

Incoming connections are still accepted.

So, misconfiguration.


Guy with a few torrent programs under his belt.

Offline

#11 2011-04-30 11:39:56

TheRook
Member

Re: UDP+Compression

There is absolutely no requirement that announce requests must be small.   This does not promote the health of a swarm.  Further more compressing UDP is great idea that will reduce bandwidth usage for BitTorrent trackers and that is a FACT.

Last edited by TheRook (2011-04-30 12:19:02)

Offline

#12 2011-04-30 13:53:31

DreadWingKnight
CBTT/BNBT Developer

Re: UDP+Compression

There is absolutely no requirement that announce requests must be small.   This does not promote the health of a swarm.

http://wiki.theory.org/BitTorrentSpecif … r_Response

Implementer's Note: Even 30 peers is plenty, the official client version 3 in fact only actively forms new connections if it has less than 30 peers and will refuse connections if it has 55. This value is important to performance. When a new piece has completed download, HAVE messages (see below) will need to be sent to most active peers. As a result the cost of broadcast traffic grows in direct proportion to the number of peers. Above 25, new peers are highly unlikely to increase download speed. UI designers are strongly advised to make this obscure and hard to change as it is very rare to be useful to do so.

Sorry, but more isn't better.


Guy with a few torrent programs under his belt.

Offline

Board footer

Powered by FluxBB