Discussion:
[libtorrent] torrent priority in master
M***@massaroddel.de
2016-08-25 15:31:42 UTC
Permalink
From my observation and understanding of the source code - torrent priority is not
implemented anymore. Is that intentionally or a bug (or my misunderstanding)?

There is still set_priority()/priority()/m_priority but it seems they get used nowhere.


MassaRoddel

------------------------------------------------------------------------------
Arvid Norberg
2016-08-25 22:59:37 UTC
Permalink
From my observation and understanding of the source code - torrent
priority is not
implemented anymore. Is that intentionally or a bug (or my
misunderstanding)?
I thought torrent priority had deliberately been removed (but as you point
out, there's cruft left).

It's utility and implementation was questionable, it did not have a well
defined behavior nor any tests. All it did (iirc) was to scale the
bandwidth quota assigned to peers based on the priority of the torrent they
belonged to. Keeping it seemed to cause more harm than benefit, since it
made the code more complex.
There is still set_priority()/priority()/m_priority but it seems they get used nowhere.
I'll mark these as deprecated and remove the state.
--
Arvid Norberg
------------------------------------------------------------------------------
Ardhi Putra Pratama
2016-09-18 13:12:33 UTC
Permalink
Is there any way to define the priority based on bandwidth then?

Or, how do we scale the bandwidth quota?
--
Ardhi Putra Pratama
@ardhipoetra

Delft University of Technology
Post by Arvid Norberg
From my observation and understanding of the source code - torrent
priority is not
implemented anymore. Is that intentionally or a bug (or my
misunderstanding)?
I thought torrent priority had deliberately been removed (but as you point
out, there's cruft left).
It's utility and implementation was questionable, it did not have a well
defined behavior nor any tests. All it did (iirc) was to scale the
bandwidth quota assigned to peers based on the priority of the torrent they
belonged to. Keeping it seemed to cause more harm than benefit, since it
made the code more complex.
There is still set_priority()/priority()/m_priority but it seems they get used nowhere.
I'll mark these as deprecated and remove the state.
--
Arvid Norberg
------------------------------------------------------------------------------
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
------------------------------------------------------------------------------
Arvid Norberg
2016-09-18 14:24:59 UTC
Permalink
you can use the peer_class feature. each torrent has a class that it
assigns to its peers. looking at this, it doesn't look like you have direct
access to the peer class for a torrent, you may be able to enumerate them
for the session or get at it some other way, and update the priority. it
also appears as if it's only created for a torrent by setting a rate limit.

If you just want to set specific bandwidth limits, you can use the member
functions on torrent_handle to do so (which will create the peer class
under the hood if it doesn't already have one).

Perhaps it would make sense to add a member function to torrents to ask or
its peer_class, which could create one if it doesn't have one already.
Post by Ardhi Putra Pratama
Is there any way to define the priority based on bandwidth then?
Or, how do we scale the bandwidth quota?
--
Ardhi Putra Pratama
@ardhipoetra
Delft University of Technology
Post by Arvid Norberg
From my observation and understanding of the source code - torrent
priority is not
implemented anymore. Is that intentionally or a bug (or my
misunderstanding)?
I thought torrent priority had deliberately been removed (but as you
point
Post by Arvid Norberg
out, there's cruft left).
It's utility and implementation was questionable, it did not have a well
defined behavior nor any tests. All it did (iirc) was to scale the
bandwidth quota assigned to peers based on the priority of the torrent
they
Post by Arvid Norberg
belonged to. Keeping it seemed to cause more harm than benefit, since it
made the code more complex.
There is still set_priority()/priority()/m_priority but it seems they
get
Post by Arvid Norberg
used nowhere.
I'll mark these as deprecated and remove the state.
--
Arvid Norberg
------------------------------------------------------------
------------------
Post by Arvid Norberg
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
------------------------------------------------------------
------------------
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
--
Arvid Norberg
------------------------------------------------------------------------------
Ardhi Putra Pratama
2016-09-19 08:30:14 UTC
Permalink
I see. I know I can limit the bandwidth to a specific peer using
peer_class. The issue is, I want to split my own bandwidth usage to a
particular torrent. set_priority describe this function well as it
should give higher chance to unchoke one with higher priority (from
what I read). Assuming user don't specify his bandwidth (mark it as
'unlimited'), I'm looking for workaround to keep this feature without
using set_priority.
--
Ardhi Putra Pratama
@ardhipoetra

Delft University of Technology
Post by Arvid Norberg
you can use the peer_class feature. each torrent has a class that it
assigns to its peers. looking at this, it doesn't look like you have direct
access to the peer class for a torrent, you may be able to enumerate them
for the session or get at it some other way, and update the priority. it
also appears as if it's only created for a torrent by setting a rate limit.
If you just want to set specific bandwidth limits, you can use the member
functions on torrent_handle to do so (which will create the peer class
under the hood if it doesn't already have one).
Perhaps it would make sense to add a member function to torrents to ask or
its peer_class, which could create one if it doesn't have one already.
Post by Ardhi Putra Pratama
Is there any way to define the priority based on bandwidth then?
Or, how do we scale the bandwidth quota?
--
Ardhi Putra Pratama
@ardhipoetra
Delft University of Technology
Post by Arvid Norberg
From my observation and understanding of the source code - torrent
priority is not
implemented anymore. Is that intentionally or a bug (or my
misunderstanding)?
I thought torrent priority had deliberately been removed (but as you
point
Post by Arvid Norberg
out, there's cruft left).
It's utility and implementation was questionable, it did not have a well
defined behavior nor any tests. All it did (iirc) was to scale the
bandwidth quota assigned to peers based on the priority of the torrent
they
Post by Arvid Norberg
belonged to. Keeping it seemed to cause more harm than benefit, since it
made the code more complex.
There is still set_priority()/priority()/m_priority but it seems they
get
Post by Arvid Norberg
used nowhere.
I'll mark these as deprecated and remove the state.
--
Arvid Norberg
------------------------------------------------------------
------------------
Post by Arvid Norberg
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
------------------------------------------------------------
------------------
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
--
Arvid Norberg
------------------------------------------------------------------------------
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
------------------------------------------------------------------------------
Arvid Norberg
2016-09-19 15:24:19 UTC
Permalink
Yeah, even the previous priority implementation wouldn't quite do what you
want unfortunately.

The rate limiter is due for a rewrite, it's not very performant and a
little bit unusual. The way it works is that peers (individually) request
bytes from the bandwidth manager, and they declare membership of certain
"channels" (global rate limit, per torrent rate limit for instance, but
with peer classes it's more generic). Each channel periodically (every 100
ms or so) has quota replenished at a certain rate. This quota is
distributed evenly among the outstanding requests against it at the time
(and a peer gets the minimum quota of all its channels).

The way priority works is that it multiplies the size of the request, thus
gets a larger share of the quota being handed out. The problem still
remains that since it's still distributed per peer, being greatly
outnumbered may still limit the amount of quota you get.

The unchoking is done similarly, by peer. Unchoke slots are not distributed
evenly across torrents, all peers are considered and the download rate from
them is the primary factor. So the unchoker optimizes *total* download
rate, not download rate for any particular torrent or even attempts to make
it even.

(patches are welcome!) :)
Post by Ardhi Putra Pratama
I see. I know I can limit the bandwidth to a specific peer using
peer_class. The issue is, I want to split my own bandwidth usage to a
particular torrent. set_priority describe this function well as it
should give higher chance to unchoke one with higher priority (from
what I read). Assuming user don't specify his bandwidth (mark it as
'unlimited'), I'm looking for workaround to keep this feature without
using set_priority.
--
Ardhi Putra Pratama
@ardhipoetra
Delft University of Technology
Post by Arvid Norberg
you can use the peer_class feature. each torrent has a class that it
assigns to its peers. looking at this, it doesn't look like you have
direct
Post by Arvid Norberg
access to the peer class for a torrent, you may be able to enumerate them
for the session or get at it some other way, and update the priority. it
also appears as if it's only created for a torrent by setting a rate
limit.
Post by Arvid Norberg
If you just want to set specific bandwidth limits, you can use the member
functions on torrent_handle to do so (which will create the peer class
under the hood if it doesn't already have one).
Perhaps it would make sense to add a member function to torrents to ask
or
Post by Arvid Norberg
its peer_class, which could create one if it doesn't have one already.
On Sun, Sep 18, 2016 at 9:12 AM, Ardhi Putra Pratama <
Post by Ardhi Putra Pratama
Is there any way to define the priority based on bandwidth then?
Or, how do we scale the bandwidth quota?
--
Ardhi Putra Pratama
@ardhipoetra
Delft University of Technology
On Fri, Aug 26, 2016 at 12:59 AM, Arvid Norberg <
Post by Arvid Norberg
From my observation and understanding of the source code - torrent
priority is not
implemented anymore. Is that intentionally or a bug (or my misunderstanding)?
I thought torrent priority had deliberately been removed (but as you
point
Post by Arvid Norberg
out, there's cruft left).
It's utility and implementation was questionable, it did not have a
well
Post by Arvid Norberg
Post by Ardhi Putra Pratama
Post by Arvid Norberg
defined behavior nor any tests. All it did (iirc) was to scale the
bandwidth quota assigned to peers based on the priority of the torrent
they
Post by Arvid Norberg
belonged to. Keeping it seemed to cause more harm than benefit, since
it
Post by Arvid Norberg
Post by Ardhi Putra Pratama
Post by Arvid Norberg
made the code more complex.
There is still set_priority()/priority()/m_priority but it seems
they
Post by Arvid Norberg
Post by Ardhi Putra Pratama
get
Post by Arvid Norberg
used nowhere.
I'll mark these as deprecated and remove the state.
--
Arvid Norberg
------------------------------------------------------------
------------------
Post by Arvid Norberg
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
------------------------------------------------------------
------------------
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
--
Arvid Norberg
------------------------------------------------------------
------------------
Post by Arvid Norberg
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
------------------------------------------------------------
------------------
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
--
Arvid Norberg
------------------------------------------------------------------------------
Ardhi Putra Pratama
2016-09-29 16:33:09 UTC
Permalink
Arvid,

I tried to find for the old code of set_priority, however, I can't
find it. Can you tell me the commit you removed it? Or perhaps the
last version that still have the priority function.

Kind regards,
ardhi
--
Ardhi Putra Pratama
@ardhipoetra

Delft University of Technology
Post by Arvid Norberg
Yeah, even the previous priority implementation wouldn't quite do what you
want unfortunately.
The rate limiter is due for a rewrite, it's not very performant and a
little bit unusual. The way it works is that peers (individually) request
bytes from the bandwidth manager, and they declare membership of certain
"channels" (global rate limit, per torrent rate limit for instance, but
with peer classes it's more generic). Each channel periodically (every 100
ms or so) has quota replenished at a certain rate. This quota is
distributed evenly among the outstanding requests against it at the time
(and a peer gets the minimum quota of all its channels).
The way priority works is that it multiplies the size of the request, thus
gets a larger share of the quota being handed out. The problem still
remains that since it's still distributed per peer, being greatly
outnumbered may still limit the amount of quota you get.
The unchoking is done similarly, by peer. Unchoke slots are not distributed
evenly across torrents, all peers are considered and the download rate from
them is the primary factor. So the unchoker optimizes *total* download
rate, not download rate for any particular torrent or even attempts to make
it even.
(patches are welcome!) :)
Post by Ardhi Putra Pratama
I see. I know I can limit the bandwidth to a specific peer using
peer_class. The issue is, I want to split my own bandwidth usage to a
particular torrent. set_priority describe this function well as it
should give higher chance to unchoke one with higher priority (from
what I read). Assuming user don't specify his bandwidth (mark it as
'unlimited'), I'm looking for workaround to keep this feature without
using set_priority.
--
Ardhi Putra Pratama
@ardhipoetra
Delft University of Technology
Post by Arvid Norberg
you can use the peer_class feature. each torrent has a class that it
assigns to its peers. looking at this, it doesn't look like you have
direct
Post by Arvid Norberg
access to the peer class for a torrent, you may be able to enumerate them
for the session or get at it some other way, and update the priority. it
also appears as if it's only created for a torrent by setting a rate
limit.
Post by Arvid Norberg
If you just want to set specific bandwidth limits, you can use the member
functions on torrent_handle to do so (which will create the peer class
under the hood if it doesn't already have one).
Perhaps it would make sense to add a member function to torrents to ask
or
Post by Arvid Norberg
its peer_class, which could create one if it doesn't have one already.
On Sun, Sep 18, 2016 at 9:12 AM, Ardhi Putra Pratama <
Post by Ardhi Putra Pratama
Is there any way to define the priority based on bandwidth then?
Or, how do we scale the bandwidth quota?
--
Ardhi Putra Pratama
@ardhipoetra
Delft University of Technology
On Fri, Aug 26, 2016 at 12:59 AM, Arvid Norberg <
Post by Arvid Norberg
From my observation and understanding of the source code - torrent
priority is not
implemented anymore. Is that intentionally or a bug (or my misunderstanding)?
I thought torrent priority had deliberately been removed (but as you
point
Post by Arvid Norberg
out, there's cruft left).
It's utility and implementation was questionable, it did not have a
well
Post by Arvid Norberg
Post by Ardhi Putra Pratama
Post by Arvid Norberg
defined behavior nor any tests. All it did (iirc) was to scale the
bandwidth quota assigned to peers based on the priority of the torrent
they
Post by Arvid Norberg
belonged to. Keeping it seemed to cause more harm than benefit, since
it
Post by Arvid Norberg
Post by Ardhi Putra Pratama
Post by Arvid Norberg
made the code more complex.
There is still set_priority()/priority()/m_priority but it seems
they
Post by Arvid Norberg
Post by Ardhi Putra Pratama
get
Post by Arvid Norberg
used nowhere.
I'll mark these as deprecated and remove the state.
--
Arvid Norberg
------------------------------------------------------------
------------------
Post by Arvid Norberg
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
------------------------------------------------------------
------------------
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
--
Arvid Norberg
------------------------------------------------------------
------------------
Post by Arvid Norberg
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
------------------------------------------------------------
------------------
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
--
Arvid Norberg
------------------------------------------------------------------------------
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
------------------------------------------------------------------------------
Arvid Norberg
2016-09-30 01:58:03 UTC
Permalink
The torrent priority is still a thing and is still used (internally). It's
used by the various chokers as well as the upload and download quota
assignment. Since you're interested in the rate limiter, take a look at:

peer_connection::request_bandwidth()
https://github.com/arvidn/libtorrent/blob/master/src/peer_connection.cpp#L5388

which calls peer_connection::get_priority()
https://github.com/arvidn/libtorrent/blob/master/src/peer_connection.cpp#L228

get_priority() will look through the classes the peer belongs to, which
includes the classes the torrent belongs to, and find the highest priority.
It then passes that onto the bandwidth manager.
Post by Ardhi Putra Pratama
Arvid,
I tried to find for the old code of set_priority, however, I can't
find it. Can you tell me the commit you removed it? Or perhaps the
last version that still have the priority function.
Kind regards,
ardhi
--
Ardhi Putra Pratama
@ardhipoetra
Delft University of Technology
Post by Arvid Norberg
Yeah, even the previous priority implementation wouldn't quite do what
you
Post by Arvid Norberg
want unfortunately.
The rate limiter is due for a rewrite, it's not very performant and a
little bit unusual. The way it works is that peers (individually) request
bytes from the bandwidth manager, and they declare membership of certain
"channels" (global rate limit, per torrent rate limit for instance, but
with peer classes it's more generic). Each channel periodically (every
100
Post by Arvid Norberg
ms or so) has quota replenished at a certain rate. This quota is
distributed evenly among the outstanding requests against it at the time
(and a peer gets the minimum quota of all its channels).
The way priority works is that it multiplies the size of the request,
thus
Post by Arvid Norberg
gets a larger share of the quota being handed out. The problem still
remains that since it's still distributed per peer, being greatly
outnumbered may still limit the amount of quota you get.
The unchoking is done similarly, by peer. Unchoke slots are not
distributed
Post by Arvid Norberg
evenly across torrents, all peers are considered and the download rate
from
Post by Arvid Norberg
them is the primary factor. So the unchoker optimizes *total* download
rate, not download rate for any particular torrent or even attempts to
make
Post by Arvid Norberg
it even.
(patches are welcome!) :)
On Mon, Sep 19, 2016 at 4:30 AM, Ardhi Putra Pratama <
Post by Ardhi Putra Pratama
I see. I know I can limit the bandwidth to a specific peer using
peer_class. The issue is, I want to split my own bandwidth usage to a
particular torrent. set_priority describe this function well as it
should give higher chance to unchoke one with higher priority (from
what I read). Assuming user don't specify his bandwidth (mark it as
'unlimited'), I'm looking for workaround to keep this feature without
using set_priority.
--
Ardhi Putra Pratama
@ardhipoetra
Delft University of Technology
Post by Arvid Norberg
you can use the peer_class feature. each torrent has a class that it
assigns to its peers. looking at this, it doesn't look like you have
direct
Post by Arvid Norberg
access to the peer class for a torrent, you may be able to enumerate
them
Post by Arvid Norberg
Post by Ardhi Putra Pratama
Post by Arvid Norberg
for the session or get at it some other way, and update the priority.
it
Post by Arvid Norberg
Post by Ardhi Putra Pratama
Post by Arvid Norberg
also appears as if it's only created for a torrent by setting a rate
limit.
Post by Arvid Norberg
If you just want to set specific bandwidth limits, you can use the
member
Post by Arvid Norberg
Post by Ardhi Putra Pratama
Post by Arvid Norberg
functions on torrent_handle to do so (which will create the peer class
under the hood if it doesn't already have one).
Perhaps it would make sense to add a member function to torrents to
ask
Post by Arvid Norberg
Post by Ardhi Putra Pratama
or
Post by Arvid Norberg
its peer_class, which could create one if it doesn't have one already.
On Sun, Sep 18, 2016 at 9:12 AM, Ardhi Putra Pratama <
Post by Ardhi Putra Pratama
Is there any way to define the priority based on bandwidth then?
Or, how do we scale the bandwidth quota?
--
Ardhi Putra Pratama
@ardhipoetra
Delft University of Technology
On Fri, Aug 26, 2016 at 12:59 AM, Arvid Norberg <
Post by Arvid Norberg
From my observation and understanding of the source code -
torrent
Post by Arvid Norberg
Post by Ardhi Putra Pratama
Post by Arvid Norberg
Post by Ardhi Putra Pratama
Post by Arvid Norberg
priority is not
implemented anymore. Is that intentionally or a bug (or my
misunderstanding)?
I thought torrent priority had deliberately been removed (but as
you
Post by Arvid Norberg
Post by Ardhi Putra Pratama
Post by Arvid Norberg
Post by Ardhi Putra Pratama
point
Post by Arvid Norberg
out, there's cruft left).
It's utility and implementation was questionable, it did not have a
well
Post by Arvid Norberg
Post by Ardhi Putra Pratama
Post by Arvid Norberg
defined behavior nor any tests. All it did (iirc) was to scale the
bandwidth quota assigned to peers based on the priority of the
torrent
Post by Arvid Norberg
Post by Ardhi Putra Pratama
Post by Arvid Norberg
Post by Ardhi Putra Pratama
they
Post by Arvid Norberg
belonged to. Keeping it seemed to cause more harm than benefit,
since
Post by Arvid Norberg
Post by Ardhi Putra Pratama
it
Post by Arvid Norberg
Post by Ardhi Putra Pratama
Post by Arvid Norberg
made the code more complex.
There is still set_priority()/priority()/m_priority but it seems
they
Post by Arvid Norberg
Post by Ardhi Putra Pratama
get
Post by Arvid Norberg
used nowhere.
I'll mark these as deprecated and remove the state.
--
Arvid Norberg
------------------------------------------------------------
------------------
Post by Arvid Norberg
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
------------------------------------------------------------
------------------
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
--
Arvid Norberg
------------------------------------------------------------
------------------
Post by Arvid Norberg
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
------------------------------------------------------------
------------------
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
--
Arvid Norberg
------------------------------------------------------------
------------------
Post by Arvid Norberg
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
------------------------------------------------------------
------------------
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
--
Arvid Norberg
------------------------------------------------------------------------------
Continue reading on narkive:
Loading...