Re: Multithreading with libssh

[Simon Moselewski <s.moselewski@xxxxxxxx>]

Hi Till,

thanks for the quick answer.
I want to collect the data as fast as possible.
All channels are to the same destination.

Problem is: You can only run one ssh_channel_request_exec per Channel. So I need 1500 channels. I would like to Open them as simultaneously as possible to Save time.

Since the Session cannot be Shared between threads I need multiple sessions.

I could also start 20 threads with one session each and 75 channels each that are used one after the other. But right now threading doesn't seem to work at all.

Is my code correct for the threading approach or am I missing something?


Regards
Simon


-------- Ursprüngliche Nachricht --------
Von: g4-lisz@xxxxxxxxxxxx
Datum: 26.09.19 14:02 (GMT+01:00)
An: libssh@xxxxxxxxxx
Betreff: Re: Multithreading with libssh

Hi Simon.

You want to open the 1500 channels simultaneously? If not, then why you
need more than one thread? And each channel is to a different
destination? If not, then why opening a new session each time? You can't
share the session between threads, but you can re-use it...

IMHO it's simpler to approach what you want than you think ;-)

Till

On 26.09.19 13:09, Simon Moselewski wrote:
> Hi everyone,
>
> I am using libssh to connect to a remote host, run commands on it and
> read the results. Since I am doing this for 1500 commands I want to
> use threads to parallelize that.
>
> I am using gcc compiler with flags -libssh -pthread and -lssh_threads.
>
> My threadless approach was:
> open session
> repeat 1500 times {
>     open channel
>     run remote command
>     close channel
> }
> close session
>
> I read in the docu, that sessions cannot be shared over multiple
> threads, so with the threadful approach I do it differently:
> repeat 1500 times {
>     open session
>     open channel
>     run remote command
>     close channel
>     close session
> }
>
> The threadful programm takes longer than my threadless approach so I
> guess something with the threading is not working correctly.
>
> I am very inexperienced with threads and would need some help here.
>
> Thanks in advance for any hints!!!
> Regards
> Simon
>
>
>
> My threadful code (shortened):
>
> ssh_threads_set_callbacks(ssh_threads_get_pthread());
> ssh_init();
> std::mutex mutex;
> std::vector<std::future<void>> tasks;
> tasks.reserve(16);
> for (list<Metric>::iterator it = metrics.begin(); it != metrics.end();
> it++) {
>     Metric metric = *it;
>     tasks.emplace_back(std::async(std::launch::async, [&ip, &user,
> &pw, metric, i, &mutex]() {
>         std::lock_guard<std::mutex> lock(mutex);
>         try {
>             ssh_session session = ssh_new();
>             ssh_options_set(session, SSH_OPTIONS_HOST, IP);
>             rc = ssh_userauth_password(session, USERNAME, PASSWORD);
>             ssh_channel channel = ssh_channel_new(session);
>             rc = ssh_channel_open_session(channel);
>             rc = ssh_channel_request_exec(channel, COMMAND);
>             // get output
>             char buffer[32] = {0};
>             int nbytes;
>             nbytes = ssh_channel_read(channel, buffer,
> sizeof(buffer)-1, 0);
>             ssh_channel_send_eof(channel);
>             //get exit code
>             int exitCode = -1;
>             exitCode = ssh_channel_get_exit_status(channel);
>             ssh_channel_close(channel);
>             ssh_channel_free(channel);
>             ssh_disconnect(session);
>             ssh_free(session);
>          }
>          catch(...) {
>               //something
>          }
>      }));
> }
> std::for_each(std::begin(tasks), std::end(tasks), [](auto& task) {
>      task.get();
> });
> std::cout << "Done.\n";
> ssh_finalize();