Re: GSOC Project discussion : Async sftp implementation

[Jakub Jelen <jjelen@xxxxxxxxxx>]

On 3/20/23 15:41, Eshan Kelkar wrote:
> Hi Jakub,
>
>     The description of the project in [1] is written in the more high-level
>     way, which would basically mean creating some "SFTP IO structure" with
>     information about chunksize, max number of on-the-fly transfers, local
>     FD and remove SFTP handle and this would handle calling the low-level
>     functions we discuss above, transparently. Just having the file
>     descriptor inside of libssh might already avoid one copy from the
>     application buffer to libssh.
>
>
> Thanks for this description, it's clear now that a high level api
> (which internally uses a low level api) for async upload and download is
> to be created  as a part of this project.
>
> Apart from this, should creation of async variants of
> sftp_open, sftp_opendir, sftp_readdir, sftp_stat etc also be
> a part of this project?

No, these are really simple commands that return just couple of bytes of 
data in very short time. Having these functions with async semantics 
will not give us any advantage from the synchronous version as there is 
really not much we could split (unlike in case of reading or writing, 
which can be megabytes or gigabytes of data that needs to be transferred 
and that can be simply split to chunks and handled async).

> (I believe once async read and write are provided the user may expect
> async variants of the these functionalities too, so creating async
> variants of them should also be a part of this project)

As you can see in the benchmarks_async_sftp_down(), the async part is 
only the download/read.

The async variant could be useful if you would like to download/upload 
several files in parallel and would not want to be blocked on opening a 
new file, but still I believe the performance improvement would be 
negligible.

Regards,
--
Jakub Jelen
Crypto Team, Security Engineering
Red Hat, Inc.