[libtorrent] Associating alerts with requests

[Bedeho Mender]

We have a desktop application built on libtorrent, and we are looking to
turn much of it´s core features into a reusable library, so that it can
power a more general set of applications (e.g. RPC daemon, mobile app,
etc.).

We want to allow the library to be used in a context where there are
multiple independent simultaneous users, e.g. different RPC clients calls
against an RPC daemon. In that situation, no user can be certain about how
public state changes in the library - or lack there of, correspond to calls
made to the library. This is because libtorrent does not have any way of
associating callbacks or identifiers with calls, so that
alerts can later be mapped to these calls.

The solution we are currently considering is to run regular libtorrent
session/torrent operations from our own plugin, and therefore on the main
libtorrent thread, and then send back richer custom alerts which have this
information. We are already using this approach to allow a user to control
the plugin itself, and it works very well.

Is there some simpler approach we are overlooking, if not, has there ever
been any serious consideration given to adding callbacks/identifiers to
libtorrent operations/alerts.

Bedeho
ᐧ

[Arvid Norberg]

sorry for the delay. I have been procrastinating responding because this is
something I've tried to come up with a good solution to before without much
success.

As for callbacks, the only callback there is is the alert notification
(apart from plugins I suppose). This callback isn't really intended to do
any work, just pass along the fact that there are new alerts to pop.

As for associating context with alerts, specifically alerts that are
responses to user calls, like post_session_stats(), save_resume_data(),
pause() etc. Yes, it would be great to be able to do that.

There are two somewhat separate challenges here afaics:

1. How is context captured and passed back? The simple answer is a C-style
void* user data that get passed around. This feels a little bit too 1970's
perhaps. The typical way of solving this these days is by capturing state
in a function object that later gets invoked along with its state and
everything stays type-safe. In this case though, it's kind of important to
not call functions on any arbitrary thread. I suppose some alerts could be
turned into callbacks instead, and executed from the client's thread in a
call to session::check_callbacks() or something like that, similar to
boost.asio's io_service::poll(). That would mean two separate mechanisms of
passing data out of libtorrent though, and it's not great to add that
complexity.

2. the other challenge is how this context is passed around inside
libtorrent. Some call-alert-pairs can easily pass along data, but others
may have a harder time. At least there may need to be specialized code to
do so for many of the call-alert-pairs. Perhaps the most reasonable
approach is to start by adding a mechanism for this to one or a few calls
at a time, rather then overhauling them all.

I'm obviously open to ideas or suggestions for solving this.

Post by Bedeho Mender
We have a desktop application built on libtorrent, and we are looking to
turn much of it´s core features into a reusable library, so that it can
power a more general set of applications (e.g. RPC daemon, mobile app,
etc.).
We want to allow the library to be used in a context where there are
multiple independent simultaneous users, e.g. different RPC clients calls
against an RPC daemon. In that situation, no user can be certain about how
public state changes in the library - or lack there of, correspond to calls
made to the library. This is because libtorrent does not have any way of
associating callbacks or identifiers with calls, so that
alerts can later be mapped to these calls.
The solution we are currently considering is to run regular libtorrent
session/torrent operations from our own plugin, and therefore on the main
libtorrent thread, and then send back richer custom alerts which have this
information. We are already using this approach to allow a user to control
the plugin itself, and it works very well.
Is there some simpler approach we are overlooking, if not, has there ever
been any serious consideration given to adding callbacks/identifiers to
libtorrent operations/alerts.
Bedeho
ᐧ
------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss

[Bedeho Mender]

1)
I do think it makes sense to have two separate mechanisms actually, because
they reflect two different things.

Alerts largely reflect successful state changes or events which libtorrent,
or plugins, have announced to the client. These may, or may not, be due to
a prior invocation from the client.
Callbacks however, are the results of specific invocations from the client,
which may or may not have even succeeded (e.g. trying to pause a torrent
which disappears).
The distinction becomes even more clear when you consider a scenario where
multiple independent clients talk to the same libtorrent session, e.g. over
some RPC interface when you have a libtorrent daemon. All clients will want
to know about all alerts, but the success or failure of an invocation is
only relevant to the invoker.

With this in mind, I think a libtorrent::service_callbacks() call, like you
describe, which runs on the client thread making the call, makes a lot of
sense.

2) This sounds like a reasonable approach. Keep in mind that not only
session calls would benefit from callbacks, but really any asynchronous
call, e.g. on a
torrent/peer_handle.

Looking forward to hearing what you, and possibly others, think of this.



ᐧ

Post by Arvid Norberg
sorry for the delay. I have been procrastinating responding because this is
something I've tried to come up with a good solution to before without much
success.
As for callbacks, the only callback there is is the alert notification
(apart from plugins I suppose). This callback isn't really intended to do
any work, just pass along the fact that there are new alerts to pop.
As for associating context with alerts, specifically alerts that are
responses to user calls, like post_session_stats(), save_resume_data(),
pause() etc. Yes, it would be great to be able to do that.
1. How is context captured and passed back? The simple answer is a C-style
void* user data that get passed around. This feels a little bit too 1970's
perhaps. The typical way of solving this these days is by capturing state
in a function object that later gets invoked along with its state and
everything stays type-safe. In this case though, it's kind of important to
not call functions on any arbitrary thread. I suppose some alerts could be
turned into callbacks instead, and executed from the client's thread in a
call to session::check_callbacks() or something like that, similar to
boost.asio's io_service::poll(). That would mean two separate mechanisms of
passing data out of libtorrent though, and it's not great to add that
complexity.
2. the other challenge is how this context is passed around inside
libtorrent. Some call-alert-pairs can easily pass along data, but others
may have a harder time. At least there may need to be specialized code to
do so for many of the call-alert-pairs. Perhaps the most reasonable
approach is to start by adding a mechanism for this to one or a few calls
at a time, rather then overhauling them all.
I'm obviously open to ideas or suggestions for solving this.

Post by Bedeho Mender
We have a desktop application built on libtorrent, and we are looking to
turn much of it´s core features into a reusable library, so that it can
power a more general set of applications (e.g. RPC daemon, mobile app,
etc.).
We want to allow the library to be used in a context where there are
multiple independent simultaneous users, e.g. different RPC clients calls
against an RPC daemon. In that situation, no user can be certain about

how

Post by Bedeho Mender
public state changes in the library - or lack there of, correspond to

calls

Post by Bedeho Mender
made to the library. This is because libtorrent does not have any way of
associating callbacks or identifiers with calls, so that
alerts can later be mapped to these calls.
The solution we are currently considering is to run regular libtorrent
session/torrent operations from our own plugin, and therefore on the main
libtorrent thread, and then send back richer custom alerts which have

this

Post by Bedeho Mender
information. We are already using this approach to allow a user to

control

Post by Bedeho Mender
the plugin itself, and it works very well.
Is there some simpler approach we are overlooking, if not, has there ever
been any serious consideration given to adding callbacks/identifiers to
libtorrent operations/alerts.
Bedeho
ᐧ
------------------------------------------------------------

------------------

Post by Bedeho Mender
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss

--
Arvid Norberg
------------------------------------------------------------
------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and
traffic
patterns at an interface-level. Reveals which users, apps, and protocols
are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning
reports.http://sdm.link/zohodev2dev
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss

------------------------------------------------------------------------------