Re: SFTP read-ahead problem

[Aris Adamantiadis <aris@xxxxxxxxxxxx>]

Hi,

I agree with your analysis. Which libssh version did you use ? Could you 
try again with 0.6.0 and tell us if the same happens ?
Please attach verbose logs.

Aris

Le 08/06/15 11:46, Tilo Eckert a écrit :
> Hi,
>
> I am trying to implement a simple read-ahead mechanism for reading large
> files via SFTP using sftp_async_read_begin() and sftp_async_read(). On
> the first attempt to read a 4096 bytes block, I request N times 4K, and
> read the first 4K block. On successive reads, for each block read
> another one is requested to keep N requests on the wire.
>
> The problem is: sftp_async_read() suddenly fails with the error message
> "Short sftp packet!" after successfully reading a few hundred packets
> when N is large enough and the server is running OpenSSH. For a remote
> host over a 5 Mbit internet connection with 60ms latency, the issue
> starts when requesting more than about 70 blocks. In my tests on GBit
> LAN, the issue started for somewhat larger values of N.
>
> When I attempt to call sftp_async_read() or sftp_close() after the error
> occurs, I can see from the libssh log that all requests are actually
> fulfilled by the server (i.e. N * 4K are received by the socket). For
> certain combinations of N and network connection, calls to these
> functions also get stuck and never return. In other cases, closing fails
> with "Unknown packet type 0".
>
> Here is a sample program that demonstrates the issue:
> http://pastebin.com/gfiFKfDT
>
> I tested this with different OpenSSH versions between 6.0 and 6.8 on
> Windows and Linux, and with freeSSHd on Windows. With all OpenSSH
> versions, I get the described behavior. With freeSSHd, there are no
> issues at all. I successfully tested it with up to N=4096 (i.e. 16 MiB
> read-ahead).
>
> It looks to me like libssh starts to misinterpret packets under certain
> conditions. Maybe someone with deeper insight can investigate this.
>
> Regards,
> Tilo