Re: Hangs when downloading large files

[Greg Ferrar <ferrar@xxxxxxxxxxxxxx>]

Hello,

I sent this on February 26, to libssh@xxxxxxxxxx, and haven't gotten a 
response. Is this the right place to be asking this sort of question? 
Has anyone seen this sort of hang? Is there something I can do to 
troubleshoot it further?


-
Greg


Greg Ferrar wrote:
> Hi,
>
> I've been using libssh successfully for some time as part of my 
> program (Sawmill), to fetch files by SFTP. But recently I've gotten 
> two separate reports of users who say Sawmill hangs while fetching 
> files. I've confirmed this. When SFTP'ing large files (over about 
> 40MB), libssh eventually hangs, consistently, at the same point.
>
> It logs in fine, make the connection, downloads part of the data, and 
> then hangs.
>
> The traceback at the point of hang is:
>
> #0 0x000000339020cff1 in recv () from /lib64/libpthread.so.0
> #1 0x00000000004f2695 in packet_read (session=0x29de150) at 
> ../src/libssh/packet.c:62
> #2 0x00000000004eb1a9 in channel_read (channel=0x29def50, 
> buffer=0x2b28d30, bytes=4, is_stderr=<value optimized out>)
> at ../src/libssh/channels.c:646
> #3 0x00000000004f3727 in sftp_packet_read (sftp=0x29df040) at 
> ../src/libssh/sftp.c:99
> #4 0x00000000004f3d8b in sftp_read_and_dispatch (session=0x9) at 
> ../src/libssh/sftp.c:158
> #5 0x00000000004f461a in sftp_read (handle=0x29ba130, data=0x29c3703, 
> len=102316) at ../src/libssh/sftp.c:854
> #6 0x000000000070bbda in SFTPSession::ReadFromOpenFile 
> (this=0x29dd6a0, buffer=0x7fffc35343d0 "\034\005?,?", bufferLen=16)
> at ../src/sftp_session.cpp:305
>
> Here are a couple of interesting points:
>
>   1. If the file is smaller, like 10MB, it completes without any 
> problems. Only files more than about 40MB hang.
>
>   2. It happens with /some/ SSH servers, but not others. Specifically, 
> if hangs with CentOS 5, x64 (OpenSSH_4.3p2, OpenSSL 0.9.8b 04 May 
> 2006). But it completes, downloading the whole 40MB file, on MacOS 
> 10.5, PowerPC G5 (OpenSSH_5.2p1, OpenSSL 0.9.7l 28 Sep 2006).
>
>   3. Using the command line "sftp" program from CentOS 5, I am able to 
> download the file from either server.
>
> I dug into the code myself a bit, but I don't really understand what's 
> happening. It seems as though libssh is expecting more data at the 
> point of hang (which seems reasonable, since the file is only partly 
> downloaded), but the server has stopped sending.
>
> I'm on libssh 0.4.1 now, but it's still hanging. Is this a known 
> problem? Any guesses about what might be going wrong? Need more 
> information?
>
> -
> Greg Ferrar