Discussion:
[libtorrent] "fastresume_rejected_alert" of big file, is it bug?
linxs
2017-05-15 03:42:28 UTC
Permalink
I use client_test (of libtorrent) to download a big file (39G), named "A".
client_test generates a .resume file for "A".
But every time I restart client_test to download "A", client_test always re-checking the whole file, which consumes a lot time.
I put some code in handle_alter() to deal with "fastresume_rejected_alert" and print the error message, and get:
---------------------------------------------------------------------------------------------
****fastresume error:
***** message: fast resume rejected. check resume(): missing or invalid 'file sizes' entry, err code:130, file:H@;
---------------------------------------------------------------------------------------------
what is wrong with the resume file? It is a bug for 39G large file ?
linxs
2017-05-15 03:58:06 UTC
Permalink
This trouble happens on raspberryPI, but no on windows.
so, is it a problem of the type of "file sizes"?
Post by linxs
I use client_test (of libtorrent) to download a big file (39G), named "A".
client_test generates a .resume file for "A".
But every time I restart client_test to download "A", client_test always re-checking the whole file, which consumes a lot time.
---------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------
what is wrong with the resume file? It is a bug for 39G large file ?
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
linxs
2017-05-15 05:06:26 UTC
Permalink
I use gdb to trace client_test and set breakpoints to where all "fastresume_rejected_alert" may be posted: torrent.cpp:start(), torrent.cpp:init(), torrent.cpp:on_resume_data_checked(). ( libtorrent version: libtorrent-libtorrent-1_1_1)
only the "on_resume_data_checked()" post this alert.
gdb debug info as follow. this function is called by boost thread. How can I trace to the place where error happen?



(gdb) c
Continuing.

Breakpoint 8, libtorrent::torrent::on_resume_data_checked (this=0x76319db0, j=0x7630cf50) at torrent.cpp:2452
2452 m_ses.alerts().emplace_alert<fastresume_rejected_alert>(get_handle(), j->error.ec
(gdb) p j
$1 = (const libtorrent::disk_io_job *) 0x7630cf50
(gdb) p *j
$2 = {<libtorrent::tailqueue_node<libtorrent::disk_io_job>> = {next = 0x0}, <boost::noncopyable_::noncopyable> = {<No data fields>}, requester = 0x0, buffer = {
disk_block = 0x76310894 "\330\334\060v\210\337\060v\330\340\060v\330\334\060v0\240\061v\277\003", string = 0x76310894 "\330\334\060v\210\337\060v\330\340\060v\330\334\060v0\240\061v\277\003", resume_data = 0x76310894,
check_resume_data = 0x76310894, priorities = 0x76310894, torrent_file = 0x76310894, delete_options = 1982924948}, storage = {px = 0x7630e22c, pn = {pi_ = 0x7630e218}},
callback = {<boost::function1<void, libtorrent::disk_io_job const*>> = {<boost::function_base> = {
vtable = 0x3bf7ac <void boost::function1<void, libtorrent::disk_io_job const*>::assign_to<boost::_bi::bind_t<void, boost::_mfi::mf1<void, libtorrent::torrent, libtorrent::disk_io_job const*>, boost::_bi::list2<boost::_bi::value<boost::shared_ptr<libtorrent::torrent> >, boost::arg<1> > > >(boost::_bi::bind_t<void, boost::_mfi::mf1<void, libtorrent::torrent, libtorrent::disk_io_job const*>, boost::_bi::list2<boost::_bi::value<boost::shared_ptr<libtorrent::torrent> >, boost::arg<1> > >)::stored_vtable>, functor = {members = {obj_ptr = 0x763181f8, type = {type = 0x763181f8, const_qualified = true, volatile_qualified = false}, func_ptr = 0x763181f8, bound_memfunc_ptr = {
memfunc_ptr = &virtual table offset 1982956024, obj_ptr = 0x0}, obj_ref = {obj_ptr = 0x763181f8, is_const_qualified = true, is_volatile_qualified = false}}, data = "\370\201\061v\001\000\000\000\000\000\000"}},
static args = <optimized out>, static arity = <optimized out>}, <No data fields>}, error = {ec = {m_val = 130, m_cat = 0x459bb4 <libtorrent::get_libtorrent_category()::libtorrent_category>}, file = -1, operation = 14}, d = {
piece_hash = "\300\363\060v", '\000' <repeats 14 times>, "\377\377", links = 0x7630f3c0, io = {ref = {storage = 0x7630f3c0, piece = 0, block = 0}, offset = 0, buffer_size = 0}}, piece = 0, action = 6, ret = -2, flags = 8 '\b'}
(gdb) n
2453 , resolve_filename(j->error.file), j->error.operation_str());
(gdb) p j->error
$3 = {ec = {m_val = 130, m_cat = 0x459bb4 <libtorrent::get_libtorrent_category()::libtorrent_category>}, file = -1, operation = 14}
(gdb) bt
#0 libtorrent::torrent::on_resume_data_checked (this=0x76319db0, j=0x7630cf50) at torrent.cpp:2453
#1 0x000830cc in operator() (a0=<optimized out>, this=<optimized out>) at /home/linxs/project/rasp/boost/include/boost/function/function_template.hpp:760
#2 libtorrent::disk_io_thread::call_job_handlers (this=0x472278, userdata=***@entry=0x471484) at disk_io_thread.cpp:3577
#3 0x000841cc in operator() (a1=0x471484, p=0x472278, this=<synthetic pointer>) at /home/linxs/project/rasp/boost/include/boost/bind/mem_fn_template.hpp:165
#4 operator()<boost::_mfi::mf1<void, libtorrent::disk_io_thread, void*>, boost::_bi::list0> (a=<synthetic pointer>, f=<synthetic pointer>, this=<synthetic pointer>) at /home/linxs/project/rasp/boost/include/boost/bind/bind.hpp:319
#5 operator() (this=<synthetic pointer>) at /home/linxs/project/rasp/boost/include/boost/bind/bind_template.hpp:20
#6 asio_handler_invoke<boost::_bi::bind_t<void, boost::_mfi::mf1<void, libtorrent::disk_io_thread, void*>, boost::_bi::list2<boost::_bi::value<libtorrent::disk_io_thread*>, boost::_bi::value<void*> > > > (function=<synthetic pointer>)
at /home/linxs/project/rasp/boost/include/boost/asio/handler_invoke_hook.hpp:69
#7 invoke<boost::_bi::bind_t<void, boost::_mfi::mf1<void, libtorrent::disk_io_thread, void*>, boost::_bi::list2<boost::_bi::value<libtorrent::disk_io_thread*>, boost::_bi::value<void*> > >, boost::_bi::bind_t<void, boost::_mfi::mf1<void, libtorrent::disk_io_thread, void*>, boost::_bi::list2<boost::_bi::value<libtorrent::disk_io_thread*>, boost::_bi::value<void*> > > > (context=<synthetic pointer>, function=<synthetic pointer>)
at /home/linxs/project/rasp/boost/include/boost/asio/detail/handler_invoke_helpers.hpp:37
#8 boost::asio::detail::completion_handler<boost::_bi::bind_t<void, boost::_mfi::mf1<void, libtorrent::disk_io_thread, void*>, boost::_bi::list2<boost::_bi::value<libtorrent::disk_io_thread*>, boost::_bi::value<void*> > > >::do_complete (owner=***@entry=0x4713c8, base=0x737004f0) at /home/linxs/project/rasp/boost/include/boost/asio/detail/completion_handler.hpp:68
#9 0x0007a1fc in complete (bytes_transferred=<optimized out>, ec=..., owner=..., this=<optimized out>) at /home/linxs/project/rasp/boost/include/boost/asio/detail/task_io_service_operation.hpp:38
#10 do_run_one (ec=..., this_thread=..., lock=..., this=<optimized out>) at /home/linxs/project/rasp/boost/include/boost/asio/detail/impl/task_io_service.ipp:372
#11 boost::asio::detail::task_io_service::run (this=0x4713c8, ec=...) at /home/linxs/project/rasp/boost/include/boost/asio/detail/impl/task_io_service.ipp:149
#12 0x000aaae0 in boost::asio::io_service::run (this=0x471394) at /home/linxs/project/rasp/boost/include/boost/asio/impl/io_service.ipp:59
#13 0x00083e18 in boost::asio::detail::boost_asio_detail_posix_thread_function (arg=0x476980) at /home/linxs/project/rasp/boost/include/boost/asio/detail/impl/posix_thread.ipp:64
#14 0x76f73e90 in start_thread (arg=0x76c7a450) at pthread_create.c:311
#15 0x76d4f598 in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) fr
#0 libtorrent::torrent::on_resume_data_checked (this=0x76319db0, j=0x7630cf50) at torrent.cpp:2453
2453 , resolve_filename(j->error.file), j->error.operation_str());
(gdb)
Post by linxs
This trouble happens on raspberryPI, but no on windows.
so, is it a problem of the type of "file sizes"?
Post by linxs
I use client_test (of libtorrent) to download a big file (39G), named "A".
client_test generates a .resume file for "A".
But every time I restart client_test to download "A", client_test always re-checking the whole file, which consumes a lot time.
---------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------
what is wrong with the resume file? It is a bug for 39G large file ?
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
Arvid Norberg
2017-05-15 21:35:46 UTC
Permalink
Please dump the resume file and look for the "file_sizes" entry, to see
what it looks like. You can do that with any bdecoder, or "dump_torrent"
that comes with libtorrent.

It's possible that "stat()" fails when writing the resume data, and hence
fails to write it right.
Post by linxs
I use gdb to trace client_test and set breakpoints to where all
"fastresume_rejected_alert" may be posted: torrent.cpp:start(),
torrent.cpp:init(), torrent.cpp:on_resume_data_checked(). ( libtorrent
version: libtorrent-libtorrent-1_1_1)
only the "on_resume_data_checked()" post this alert.
gdb debug info as follow. this function is called by boost thread. How can
I trace to the place where error happen?
(gdb) c
Continuing.
Breakpoint 8, libtorrent::torrent::on_resume_data_checked
(this=0x76319db0, j=0x7630cf50) at torrent.cpp:2452
2452 m_ses.alerts().emplace_alert<
fastresume_rejected_alert>(get_handle(), j->error.ec
(gdb) p j
$1 = (const libtorrent::disk_io_job *) 0x7630cf50
(gdb) p *j
$2 = {<libtorrent::tailqueue_node<libtorrent::disk_io_job>> = {next =
0x0}, <boost::noncopyable_::noncopyable> = {<No data fields>}, requester
= 0x0, buffer = {
disk_block = 0x76310894 "\330\334\060v\210\337\060v\
330\340\060v\330\334\060v0\240\061v\277\003", string = 0x76310894
"\330\334\060v\210\337\060v\330\340\060v\330\334\060v0\240\061v\277\003",
resume_data = 0x76310894,
check_resume_data = 0x76310894, priorities = 0x76310894, torrent_file
= 0x76310894, delete_options = 1982924948}, storage = {px = 0x7630e22c, pn
= {pi_ = 0x7630e218}},
callback = {<boost::function1<void, libtorrent::disk_io_job const*>> =
{<boost::function_base> = {
vtable = 0x3bf7ac <void boost::function1<void,
libtorrent::disk_io_job const*>::assign_to<boost::_bi::bind_t<void,
boost::_mfi::mf1<void, libtorrent::torrent, libtorrent::disk_io_job
const*>, boost::_bi::list2<boost::_bi::value<boost::shared_ptr<libtorrent::torrent>
Post by linxs
, boost::arg<1> > > >(boost::_bi::bind_t<void, boost::_mfi::mf1<void,
libtorrent::torrent, libtorrent::disk_io_job const*>,
boost::_bi::list2<boost::_bi::value<boost::shared_ptr<libtorrent::torrent>
Post by linxs
, boost::arg<1> > >)::stored_vtable>, functor = {members = {obj_ptr =
0x763181f8, type = {type = 0x763181f8, const_qualified = true,
volatile_qualified = false}, func_ptr = 0x763181f8, bound_memfunc_ptr = {
memfunc_ptr = &virtual table offset 1982956024, obj_ptr =
0x0}, obj_ref = {obj_ptr = 0x763181f8, is_const_qualified = true,
is_volatile_qualified = false}}, data = "\370\201\061v\001\000\000\
000\000\000\000"}},
static args = <optimized out>, static arity = <optimized out>}, <No
data fields>}, error = {ec = {m_val = 130, m_cat = 0x459bb4
<libtorrent::get_libtorrent_category()::libtorrent_category>}, file = -1,
operation = 14}, d = {
piece_hash = "\300\363\060v", '\000' <repeats 14 times>, "\377\377",
links = 0x7630f3c0, io = {ref = {storage = 0x7630f3c0, piece = 0, block =
0}, offset = 0, buffer_size = 0}}, piece = 0, action = 6, ret = -2, flags =
8 '\b'}
(gdb) n
2453 , resolve_filename(j->error.file),
j->error.operation_str());
(gdb) p j->error
$3 = {ec = {m_val = 130, m_cat = 0x459bb4 <libtorrent::get_libtorrent_
category()::libtorrent_category>}, file = -1, operation = 14}
(gdb) bt
#0 libtorrent::torrent::on_resume_data_checked (this=0x76319db0,
j=0x7630cf50) at torrent.cpp:2453
#1 0x000830cc in operator() (a0=<optimized out>, this=<optimized out>) at
/home/linxs/project/rasp/boost/include/boost/function/
function_template.hpp:760
#2 libtorrent::disk_io_thread::call_job_handlers (this=0x472278,
#3 0x000841cc in operator() (a1=0x471484, p=0x472278, this=<synthetic
pointer>) at /home/linxs/project/rasp/boost/include/boost/bind/mem_
fn_template.hpp:165
#4 operator()<boost::_mfi::mf1<void, libtorrent::disk_io_thread, void*>,
boost::_bi::list0> (a=<synthetic pointer>, f=<synthetic pointer>,
this=<synthetic pointer>) at /home/linxs/project/rasp/
boost/include/boost/bind/bind.hpp:319
#5 operator() (this=<synthetic pointer>) at /home/linxs/project/rasp/
boost/include/boost/bind/bind_template.hpp:20
#6 asio_handler_invoke<boost::_bi::bind_t<void, boost::_mfi::mf1<void,
value<libtorrent::disk_io_thread*>, boost::_bi::value<void*> > > >
(function=<synthetic pointer>)
at /home/linxs/project/rasp/boost/include/boost/asio/
handler_invoke_hook.hpp:69
#7 invoke<boost::_bi::bind_t<void, boost::_mfi::mf1<void,
value<libtorrent::disk_io_thread*>, boost::_bi::value<void*> > >,
boost::_bi::bind_t<void, boost::_mfi::mf1<void, libtorrent::disk_io_thread,
void*>, boost::_bi::list2<boost::_bi::value<libtorrent::disk_io_thread*>,
boost::_bi::value<void*> > > > (context=<synthetic pointer>,
function=<synthetic pointer>)
at /home/linxs/project/rasp/boost/include/boost/asio/
detail/handler_invoke_helpers.hpp:37
#8 boost::asio::detail::completion_handler<boost::_bi::bind_t<void,
boost::_mfi::mf1<void, libtorrent::disk_io_thread, void*>,
boost::_bi::list2<boost::_bi::value<libtorrent::disk_io_thread*>,
base=0x737004f0) at /home/linxs/project/rasp/boost/include/boost/asio/
detail/completion_handler.hpp:68
#9 0x0007a1fc in complete (bytes_transferred=<optimized out>, ec=...,
owner=..., this=<optimized out>) at /home/linxs/project/rasp/
boost/include/boost/asio/detail/task_io_service_operation.hpp:38
#10 do_run_one (ec=..., this_thread=..., lock=..., this=<optimized out>)
at /home/linxs/project/rasp/boost/include/boost/asio/
detail/impl/task_io_service.ipp:372
#11 boost::asio::detail::task_io_service::run (this=0x4713c8, ec=...) at
/home/linxs/project/rasp/boost/include/boost/asio/
detail/impl/task_io_service.ipp:149
#12 0x000aaae0 in boost::asio::io_service::run (this=0x471394) at
/home/linxs/project/rasp/boost/include/boost/asio/impl/io_service.ipp:59
#13 0x00083e18 in boost::asio::detail::boost_asio_detail_posix_thread_function
(arg=0x476980) at /home/linxs/project/rasp/boost/include/boost/asio/
detail/impl/posix_thread.ipp:64
#14 0x76f73e90 in start_thread (arg=0x76c7a450) at pthread_create.c:311
#15 0x76d4f598 in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92
from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) fr
#0 libtorrent::torrent::on_resume_data_checked (this=0x76319db0,
j=0x7630cf50) at torrent.cpp:2453
2453 , resolve_filename(j->error.file),
j->error.operation_str());
(gdb)
Post by linxs
This trouble happens on raspberryPI, but no on windows.
so, is it a problem of the type of "file sizes"?
Post by linxs
I use client_test (of libtorrent) to download a big file (39G), named
"A".
Post by linxs
Post by linxs
client_test generates a .resume file for "A".
But every time I restart client_test to download "A", client_test always
re-checking the whole file, which consumes a lot time.
Post by linxs
Post by linxs
I put some code in handle_alter() to deal with
----------------------------------------------------------
-----------------------------------
Post by linxs
Post by linxs
***** message: fast resume rejected. check resume(): missing or invalid
----------------------------------------------------------
-----------------------------------
Post by linxs
Post by linxs
what is wrong with the resume file? It is a bug for 39G large file ?
----------------------------------------------------------
--------------------
Post by linxs
Post by linxs
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
-----------------------------------------------------------
-------------------
Post by linxs
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
--
Arvid Norberg
linxs
2017-05-16 02:34:05 UTC
Permalink
I have dumped the .resume file and found the wrong place:
(the torrent has several files and some of them are big files. )some files length values of the "file sizes" in .resume are the same, which cause the "mismatching length" fastresume error alert.
it seems that in the process the length values can't be updated to the true length and use previous file length.
i have not find the error source, maybe it happen during initializing or write_resume_data() every time.
Post by Arvid Norberg
Please dump the resume file and look for the "file_sizes" entry, to see
what it looks like. You can do that with any bdecoder, or "dump_torrent"
that comes with libtorrent.
It's possible that "stat()" fails when writing the resume data, and hence
fails to write it right.
Post by linxs
I use gdb to trace client_test and set breakpoints to where all
"fastresume_rejected_alert" may be posted: torrent.cpp:start(),
torrent.cpp:init(), torrent.cpp:on_resume_data_checked(). ( libtorrent
version: libtorrent-libtorrent-1_1_1)
only the "on_resume_data_checked()" post this alert.
gdb debug info as follow. this function is called by boost thread. How can
I trace to the place where error happen?
(gdb) c
Continuing.
Breakpoint 8, libtorrent::torrent::on_resume_data_checked
(this=0x76319db0, j=0x7630cf50) at torrent.cpp:2452
2452 m_ses.alerts().emplace_alert<
fastresume_rejected_alert>(get_handle(), j->error.ec
(gdb) p j
$1 = (const libtorrent::disk_io_job *) 0x7630cf50
(gdb) p *j
$2 = {<libtorrent::tailqueue_node<libtorrent::disk_io_job>> = {next =
0x0}, <boost::noncopyable_::noncopyable> = {<No data fields>}, requester
= 0x0, buffer = {
disk_block = 0x76310894 "\330\334\060v\210\337\060v\
330\340\060v\330\334\060v0\240\061v\277\003", string = 0x76310894
"\330\334\060v\210\337\060v\330\340\060v\330\334\060v0\240\061v\277\003",
resume_data = 0x76310894,
check_resume_data = 0x76310894, priorities = 0x76310894, torrent_file
= 0x76310894, delete_options = 1982924948}, storage = {px = 0x7630e22c, pn
= {pi_ = 0x7630e218}},
callback = {<boost::function1<void, libtorrent::disk_io_job const*>> =
{<boost::function_base> = {
vtable = 0x3bf7ac <void boost::function1<void,
libtorrent::disk_io_job const*>::assign_to<boost::_bi::bind_t<void,
boost::_mfi::mf1<void, libtorrent::torrent, libtorrent::disk_io_job
const*>, boost::_bi::list2<boost::_bi::value<boost::shared_ptr<libtorrent::torrent>
Post by linxs
, boost::arg<1> > > >(boost::_bi::bind_t<void, boost::_mfi::mf1<void,
libtorrent::torrent, libtorrent::disk_io_job const*>,
boost::_bi::list2<boost::_bi::value<boost::shared_ptr<libtorrent::torrent>
Post by linxs
, boost::arg<1> > >)::stored_vtable>, functor = {members = {obj_ptr =
0x763181f8, type = {type = 0x763181f8, const_qualified = true,
volatile_qualified = false}, func_ptr = 0x763181f8, bound_memfunc_ptr = {
memfunc_ptr = &virtual table offset 1982956024, obj_ptr =
0x0}, obj_ref = {obj_ptr = 0x763181f8, is_const_qualified = true,
is_volatile_qualified = false}}, data = "\370\201\061v\001\000\000\
000\000\000\000"}},
static args = <optimized out>, static arity = <optimized out>}, <No
data fields>}, error = {ec = {m_val = 130, m_cat = 0x459bb4
<libtorrent::get_libtorrent_category()::libtorrent_category>}, file = -1,
operation = 14}, d = {
piece_hash = "\300\363\060v", '\000' <repeats 14 times>, "\377\377",
links = 0x7630f3c0, io = {ref = {storage = 0x7630f3c0, piece = 0, block =
0}, offset = 0, buffer_size = 0}}, piece = 0, action = 6, ret = -2, flags =
8 '\b'}
(gdb) n
2453 , resolve_filename(j->error.file),
j->error.operation_str());
(gdb) p j->error
$3 = {ec = {m_val = 130, m_cat = 0x459bb4 <libtorrent::get_libtorrent_
category()::libtorrent_category>}, file = -1, operation = 14}
(gdb) bt
#0 libtorrent::torrent::on_resume_data_checked (this=0x76319db0,
j=0x7630cf50) at torrent.cpp:2453
#1 0x000830cc in operator() (a0=<optimized out>, this=<optimized out>) at
/home/linxs/project/rasp/boost/include/boost/function/
function_template.hpp:760
#2 libtorrent::disk_io_thread::call_job_handlers (this=0x472278,
#3 0x000841cc in operator() (a1=0x471484, p=0x472278, this=<synthetic
pointer>) at /home/linxs/project/rasp/boost/include/boost/bind/mem_
fn_template.hpp:165
#4 operator()<boost::_mfi::mf1<void, libtorrent::disk_io_thread, void*>,
boost::_bi::list0> (a=<synthetic pointer>, f=<synthetic pointer>,
this=<synthetic pointer>) at /home/linxs/project/rasp/
boost/include/boost/bind/bind.hpp:319
#5 operator() (this=<synthetic pointer>) at /home/linxs/project/rasp/
boost/include/boost/bind/bind_template.hpp:20
#6 asio_handler_invoke<boost::_bi::bind_t<void, boost::_mfi::mf1<void,
value<libtorrent::disk_io_thread*>, boost::_bi::value<void*> > > >
(function=<synthetic pointer>)
at /home/linxs/project/rasp/boost/include/boost/asio/
handler_invoke_hook.hpp:69
#7 invoke<boost::_bi::bind_t<void, boost::_mfi::mf1<void,
value<libtorrent::disk_io_thread*>, boost::_bi::value<void*> > >,
boost::_bi::bind_t<void, boost::_mfi::mf1<void, libtorrent::disk_io_thread,
void*>, boost::_bi::list2<boost::_bi::value<libtorrent::disk_io_thread*>,
boost::_bi::value<void*> > > > (context=<synthetic pointer>,
function=<synthetic pointer>)
at /home/linxs/project/rasp/boost/include/boost/asio/
detail/handler_invoke_helpers.hpp:37
#8 boost::asio::detail::completion_handler<boost::_bi::bind_t<void,
boost::_mfi::mf1<void, libtorrent::disk_io_thread, void*>,
boost::_bi::list2<boost::_bi::value<libtorrent::disk_io_thread*>,
base=0x737004f0) at /home/linxs/project/rasp/boost/include/boost/asio/
detail/completion_handler.hpp:68
#9 0x0007a1fc in complete (bytes_transferred=<optimized out>, ec=...,
owner=..., this=<optimized out>) at /home/linxs/project/rasp/
boost/include/boost/asio/detail/task_io_service_operation.hpp:38
#10 do_run_one (ec=..., this_thread=..., lock=..., this=<optimized out>)
at /home/linxs/project/rasp/boost/include/boost/asio/
detail/impl/task_io_service.ipp:372
#11 boost::asio::detail::task_io_service::run (this=0x4713c8, ec=...) at
/home/linxs/project/rasp/boost/include/boost/asio/
detail/impl/task_io_service.ipp:149
#12 0x000aaae0 in boost::asio::io_service::run (this=0x471394) at
/home/linxs/project/rasp/boost/include/boost/asio/impl/io_service.ipp:59
#13 0x00083e18 in boost::asio::detail::boost_asio_detail_posix_thread_function
(arg=0x476980) at /home/linxs/project/rasp/boost/include/boost/asio/
detail/impl/posix_thread.ipp:64
#14 0x76f73e90 in start_thread (arg=0x76c7a450) at pthread_create.c:311
#15 0x76d4f598 in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92
from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) fr
#0 libtorrent::torrent::on_resume_data_checked (this=0x76319db0,
j=0x7630cf50) at torrent.cpp:2453
2453 , resolve_filename(j->error.file),
j->error.operation_str());
(gdb)
Post by linxs
This trouble happens on raspberryPI, but no on windows.
so, is it a problem of the type of "file sizes"?
Post by linxs
I use client_test (of libtorrent) to download a big file (39G), named
"A".
Post by linxs
Post by linxs
client_test generates a .resume file for "A".
But every time I restart client_test to download "A", client_test always
re-checking the whole file, which consumes a lot time.
Post by linxs
Post by linxs
I put some code in handle_alter() to deal with
----------------------------------------------------------
-----------------------------------
Post by linxs
Post by linxs
***** message: fast resume rejected. check resume(): missing or invalid
----------------------------------------------------------
-----------------------------------
Post by linxs
Post by linxs
what is wrong with the resume file? It is a bug for 39G large file ?
----------------------------------------------------------
--------------------
Post by linxs
Post by linxs
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
-----------------------------------------------------------
-------------------
Post by linxs
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
--
Arvid Norberg
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
linxs
2017-05-16 03:43:24 UTC
Permalink
i got it.
source code version: 1_1_1
in storage.cpp:547, default_storage::initialize()

if (m_stat_cache.get_filesize(file_index) == stat_cache::not_in_cache)
{
file_status s;
std::string file_path = files().file_path(file_index, m_save_path);
stat_file(file_path, &s, ec.ec); // ERROR HERE
if (ec && ec.ec != boost::system::errc::no_such_file_or_directory) //ERROR HERE
{
m_stat_cache.set_error(file_index);
ec.file = file_index;
ec.operation = storage_error::stat;
break;
}
m_stat_cache.set_cache(file_index, s.file_size, s.mtime);
}
if the file of "file_path" does not exist, stat_file return with error in ec, but the following code does not check this type of error
and use the value left over by last call as right value.
after that it write this value to .resume, causing fastresume error everytime .
Post by linxs
(the torrent has several files and some of them are big files. )some files length values of the "file sizes" in .resume are the same, which cause the "mismatching length" fastresume error alert.
it seems that in the process the length values can't be updated to the true length and use previous file length.
i have not find the error source, maybe it happen during initializing or write_resume_data() every time.
Post by Arvid Norberg
Please dump the resume file and look for the "file_sizes" entry, to see
what it looks like. You can do that with any bdecoder, or "dump_torrent"
that comes with libtorrent.
It's possible that "stat()" fails when writing the resume data, and hence
fails to write it right.
Post by linxs
I use gdb to trace client_test and set breakpoints to where all
"fastresume_rejected_alert" may be posted: torrent.cpp:start(),
torrent.cpp:init(), torrent.cpp:on_resume_data_checked(). ( libtorrent
version: libtorrent-libtorrent-1_1_1)
only the "on_resume_data_checked()" post this alert.
gdb debug info as follow. this function is called by boost thread. How can
I trace to the place where error happen?
(gdb) c
Continuing.
Breakpoint 8, libtorrent::torrent::on_resume_data_checked
(this=0x76319db0, j=0x7630cf50) at torrent.cpp:2452
2452 m_ses.alerts().emplace_alert<
fastresume_rejected_alert>(get_handle(), j->error.ec
(gdb) p j
$1 = (const libtorrent::disk_io_job *) 0x7630cf50
(gdb) p *j
$2 = {<libtorrent::tailqueue_node<libtorrent::disk_io_job>> = {next =
0x0}, <boost::noncopyable_::noncopyable> = {<No data fields>}, requester
= 0x0, buffer = {
disk_block = 0x76310894 "\330\334\060v\210\337\060v\
330\340\060v\330\334\060v0\240\061v\277\003", string = 0x76310894
"\330\334\060v\210\337\060v\330\340\060v\330\334\060v0\240\061v\277\003",
resume_data = 0x76310894,
check_resume_data = 0x76310894, priorities = 0x76310894, torrent_file
= 0x76310894, delete_options = 1982924948}, storage = {px = 0x7630e22c, pn
= {pi_ = 0x7630e218}},
callback = {<boost::function1<void, libtorrent::disk_io_job const*>> =
{<boost::function_base> = {
vtable = 0x3bf7ac <void boost::function1<void,
libtorrent::disk_io_job const*>::assign_to<boost::_bi::bind_t<void,
boost::_mfi::mf1<void, libtorrent::torrent, libtorrent::disk_io_job
const*>, boost::_bi::list2<boost::_bi::value<boost::shared_ptr<libtorrent::torrent>
Post by linxs
, boost::arg<1> > > >(boost::_bi::bind_t<void, boost::_mfi::mf1<void,
libtorrent::torrent, libtorrent::disk_io_job const*>,
boost::_bi::list2<boost::_bi::value<boost::shared_ptr<libtorrent::torrent>
Post by linxs
, boost::arg<1> > >)::stored_vtable>, functor = {members = {obj_ptr =
0x763181f8, type = {type = 0x763181f8, const_qualified = true,
volatile_qualified = false}, func_ptr = 0x763181f8, bound_memfunc_ptr = {
memfunc_ptr = &virtual table offset 1982956024, obj_ptr =
0x0}, obj_ref = {obj_ptr = 0x763181f8, is_const_qualified = true,
is_volatile_qualified = false}}, data = "\370\201\061v\001\000\000\
000\000\000\000"}},
static args = <optimized out>, static arity = <optimized out>}, <No
data fields>}, error = {ec = {m_val = 130, m_cat = 0x459bb4
<libtorrent::get_libtorrent_category()::libtorrent_category>}, file = -1,
operation = 14}, d = {
piece_hash = "\300\363\060v", '\000' <repeats 14 times>, "\377\377",
links = 0x7630f3c0, io = {ref = {storage = 0x7630f3c0, piece = 0, block =
0}, offset = 0, buffer_size = 0}}, piece = 0, action = 6, ret = -2, flags =
8 '\b'}
(gdb) n
2453 , resolve_filename(j->error.file),
j->error.operation_str());
(gdb) p j->error
$3 = {ec = {m_val = 130, m_cat = 0x459bb4 <libtorrent::get_libtorrent_
category()::libtorrent_category>}, file = -1, operation = 14}
(gdb) bt
#0 libtorrent::torrent::on_resume_data_checked (this=0x76319db0,
j=0x7630cf50) at torrent.cpp:2453
#1 0x000830cc in operator() (a0=<optimized out>, this=<optimized out>) at
/home/linxs/project/rasp/boost/include/boost/function/
function_template.hpp:760
#2 libtorrent::disk_io_thread::call_job_handlers (this=0x472278,
#3 0x000841cc in operator() (a1=0x471484, p=0x472278, this=<synthetic
pointer>) at /home/linxs/project/rasp/boost/include/boost/bind/mem_
fn_template.hpp:165
#4 operator()<boost::_mfi::mf1<void, libtorrent::disk_io_thread, void*>,
boost::_bi::list0> (a=<synthetic pointer>, f=<synthetic pointer>,
this=<synthetic pointer>) at /home/linxs/project/rasp/
boost/include/boost/bind/bind.hpp:319
#5 operator() (this=<synthetic pointer>) at /home/linxs/project/rasp/
boost/include/boost/bind/bind_template.hpp:20
#6 asio_handler_invoke<boost::_bi::bind_t<void, boost::_mfi::mf1<void,
value<libtorrent::disk_io_thread*>, boost::_bi::value<void*> > > >
(function=<synthetic pointer>)
at /home/linxs/project/rasp/boost/include/boost/asio/
handler_invoke_hook.hpp:69
#7 invoke<boost::_bi::bind_t<void, boost::_mfi::mf1<void,
value<libtorrent::disk_io_thread*>, boost::_bi::value<void*> > >,
boost::_bi::bind_t<void, boost::_mfi::mf1<void, libtorrent::disk_io_thread,
void*>, boost::_bi::list2<boost::_bi::value<libtorrent::disk_io_thread*>,
boost::_bi::value<void*> > > > (context=<synthetic pointer>,
function=<synthetic pointer>)
at /home/linxs/project/rasp/boost/include/boost/asio/
detail/handler_invoke_helpers.hpp:37
#8 boost::asio::detail::completion_handler<boost::_bi::bind_t<void,
boost::_mfi::mf1<void, libtorrent::disk_io_thread, void*>,
boost::_bi::list2<boost::_bi::value<libtorrent::disk_io_thread*>,
base=0x737004f0) at /home/linxs/project/rasp/boost/include/boost/asio/
detail/completion_handler.hpp:68
#9 0x0007a1fc in complete (bytes_transferred=<optimized out>, ec=...,
owner=..., this=<optimized out>) at /home/linxs/project/rasp/
boost/include/boost/asio/detail/task_io_service_operation.hpp:38
#10 do_run_one (ec=..., this_thread=..., lock=..., this=<optimized out>)
at /home/linxs/project/rasp/boost/include/boost/asio/
detail/impl/task_io_service.ipp:372
#11 boost::asio::detail::task_io_service::run (this=0x4713c8, ec=...) at
/home/linxs/project/rasp/boost/include/boost/asio/
detail/impl/task_io_service.ipp:149
#12 0x000aaae0 in boost::asio::io_service::run (this=0x471394) at
/home/linxs/project/rasp/boost/include/boost/asio/impl/io_service.ipp:59
#13 0x00083e18 in boost::asio::detail::boost_asio_detail_posix_thread_function
(arg=0x476980) at /home/linxs/project/rasp/boost/include/boost/asio/
detail/impl/posix_thread.ipp:64
#14 0x76f73e90 in start_thread (arg=0x76c7a450) at pthread_create.c:311
#15 0x76d4f598 in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92
from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) fr
#0 libtorrent::torrent::on_resume_data_checked (this=0x76319db0,
j=0x7630cf50) at torrent.cpp:2453
2453 , resolve_filename(j->error.file),
j->error.operation_str());
(gdb)
Post by linxs
This trouble happens on raspberryPI, but no on windows.
so, is it a problem of the type of "file sizes"?
Post by linxs
I use client_test (of libtorrent) to download a big file (39G), named
"A".
Post by linxs
Post by linxs
client_test generates a .resume file for "A".
But every time I restart client_test to download "A", client_test always
re-checking the whole file, which consumes a lot time.
Post by linxs
Post by linxs
I put some code in handle_alter() to deal with
----------------------------------------------------------
-----------------------------------
Post by linxs
Post by linxs
***** message: fast resume rejected. check resume(): missing or invalid
----------------------------------------------------------
-----------------------------------
Post by linxs
Post by linxs
what is wrong with the resume file? It is a bug for 39G large file ?
----------------------------------------------------------
--------------------
Post by linxs
Post by linxs
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
-----------------------------------------------------------
-------------------
Post by linxs
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
--
Arvid Norberg
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
Arvid Norberg
2017-05-17 00:16:59 UTC
Permalink
Post by linxs
i got it.
source code version: 1_1_1
in storage.cpp:547, default_storage::initialize()
if (m_stat_cache.get_filesize(file_index) ==
stat_cache::not_in_cache)
{
file_status s;
std::string file_path = files().file_path(file_index, m_save_path);
stat_file(file_path, &s, ec.ec);
// ERROR HERE
if (ec && ec.ec != boost::system::errc::no_such_file_or_directory)
//ERROR HERE
{
m_stat_cache.set_error(file_index);
ec.file = file_index;
ec.operation = storage_error::stat;
break;
}
m_stat_cache.set_cache(file_index, s.file_size, s.mtime);
}
if the file of "file_path" does not exist, stat_file return with error in
ec, but the following code does not check this type of error
which kind of error? if it's not failing with ENOENT, presumably it's
failing with something serious, like EACCESS, no?

what is the error you get?

you can print ec.message(), ec.code(), ec.category().name() in the debugger.

In fact, this error should make it into the alert indicating saving resume
data failed too, and may be in your log (assuming you log alerts)
--
Arvid Norberg
linxs
2017-05-17 00:45:38 UTC
Permalink
in file.cpp stat_file() function, because the target file is not exist, CreateFile or ::stat will fail and "ec.assign(errno, system_category());".
this kind of error is not checked in version 1_1_1. but version 1_1_3 fixed this bug here.
i have not confirm yet in version 1_1_3 anywhere stat_file is called the return value is deal with.



void stat_file(std::string const& inf, file_status* s
, error_code& ec, int flags)
{
.....
....
// in order to open a directory, we need the FILE_FLAG_BACKUP_SEMANTICS
HANDLE h = CreateFile_(f.c_str(), 0, FILE_SHARE_DELETE | FILE_SHARE_READ
| FILE_SHARE_WRITE, NULL, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, NULL);

if (h == INVALID_HANDLE_VALUE)
{
ec.assign(GetLastError(), system_category());
TORRENT_ASSERT(ec);
return;
}
....
....

struct stat ret;
int retval;
if (flags & dont_follow_links)
retval = ::lstat(f.c_str(), &ret);
else
retval = ::stat(f.c_str(), &ret);
if (retval < 0)
{
ec.assign(errno, system_category());
return;
}
....
....
}
Post by Arvid Norberg
which kind of error? if it's not failing with ENOENT, presumably it's
failing with something serious, like EACCESS, no?
what is the error you get?
you can print ec.message(), ec.code(), ec.category().name() in the debugger.
In fact, this error should make it into the alert indicating saving resume
data failed too, and may be in your log (assuming you log alerts)
--
Arvid Norberg
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
Arvid Norberg
2017-05-17 00:53:37 UTC
Permalink
Post by linxs
in file.cpp stat_file() function, because the target file is not exist,
CreateFile or ::stat will fail and "ec.assign(errno, system_category());".
this kind of error is not checked in version 1_1_1. but version 1_1_3 fixed this bug here.
i have not confirm yet in version 1_1_3 anywhere stat_file is called the
return value is deal with.
The code you posted, where the error is checked like this:

if (ec && ec != boost::system::errc::no_such_file_or_directory)

That's essentially checking for errno == ENOENT (and system_category()).
So, my question was, what error code do you get? what was errno set to?
The code checks for ENOENT as far as I can tell, so I'm assuming it's some
other error.
--
Arvid Norberg
linxs
2017-05-17 01:21:05 UTC
Permalink
vs2013, source code version 1_1_1, boost version 1.62.0.
in this case,"ec.assign(GetLastError(), system_category());", GetLastError() return 2.

i trace into "if (ec && ec != boost::system::errc::no_such_file_or_directory)",
"boost::system::errc::no_such_file_or_directory" return an error_code object with its "m_val"=2, too.
so the term fail and is passed.
Post by Arvid Norberg
Post by linxs
in file.cpp stat_file() function, because the target file is not exist,
CreateFile or ::stat will fail and "ec.assign(errno, system_category());".
this kind of error is not checked in version 1_1_1. but version 1_1_3
fixed this bug here.
i have not confirm yet in version 1_1_3 anywhere stat_file is called the
return value is deal with.
if (ec && ec != boost::system::errc::no_such_file_or_directory)
That's essentially checking for errno == ENOENT (and system_category()).
So, my question was, what error code do you get? what was errno set to?
The code checks for ENOENT as far as I can tell, so I'm assuming it's some
other error.
--
Arvid Norberg
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
Arvid Norberg
2017-05-17 03:56:08 UTC
Permalink
Post by linxs
vs2013, source code version 1_1_1, boost version 1.62.0.
in this case,"ec.assign(GetLastError(), system_category());",
GetLastError() return 2.
I still think it would be useful to know what ec.message() and
ec.category().name() return.

2 is the value of ERROR_FILE_NOT_FOUND, see
https://msdn.microsoft.com/en-us/library/windows/desktop/ms681382(v=vs.85).aspx
which looks right to me.

i trace into "if (ec && ec != boost::system::errc::no_such_
Post by linxs
file_or_directory)",
"boost::system::errc::no_such_file_or_directory" return an error_code
object with its "m_val"=2, too.
so the term fail and is passed.
Oh, so the test works as intended then. I see, the problem in the first
code snippet you sent was that when the file didn't exist, the
uninitialized size would still be assigned to the cache.

Right, that has been fixed.
--
Arvid Norberg
linxs
2017-05-17 04:31:04 UTC
Permalink
ec.message(): the system cannot find the file specified.

ec.category().name(): system
Post by Arvid Norberg
Post by linxs
vs2013, source code version 1_1_1, boost version 1.62.0.
in this case,"ec.assign(GetLastError(), system_category());", GetLastError() return 2.
I still think it would be useful to know what ec.message() and
ec.category().name() return.
2 is the value of ERROR_FILE_NOT_FOUND, see
https://msdn.microsoft.com/en-us/library/windows/desktop/ms681382(v=vs.85).aspx
which looks right to me.
i trace into "if (ec && ec != boost::system::errc::no_such_
Post by linxs
file_or_directory)",
"boost::system::errc::no_such_file_or_directory" return an error_code
object with its "m_val"=2, too.
so the term fail and is passed.
Oh, so the test works as intended then. I see, the problem in the first
code snippet you sent was that when the file didn't exist, the
uninitialized size would still be assigned to the cache.
Right, that has been fixed.
--
Arvid Norberg
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Libtorrent-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss
Loading...