Re: Building for Android

[Matteo <matpen@xxxxxxx>]

More good news: it seems that, apart from the problem we are discussing,
the rest of the library works correctly.
At least according to the short test below, the results are as expected
(both on device and emulator) and I get a list of files from the server.

void callback(int priority, const char *function, const char *buffer,
void *userdata)
{
    //this can be empty
}

void MainWindow::test()
{
    ssh::Session session;
    session.setOption(SSH_OPTIONS_TIMEOUT, 20);
    session.setOption(SSH_OPTIONS_HOST, "192.168.0.13");
    session.setOption(SSH_OPTIONS_USER, "root");
    session.setOption(SSH_OPTIONS_PORT, 2222);

    session.setOption(SSH_OPTIONS_LOG_VERBOSITY, 2);
    ssh_set_log_level(SSH_LOG_FUNCTIONS);
    ssh_set_log_callback(callback);

    if (session.connect() != SSH_OK) {
        message("cannot connect");
        return;
    } else {
        message("connected");
    }

    QString pass = "Secret123";
    QByteArray passArray = pass.toUtf8();
    if (session.userauthPassword(passArray.data()) != SSH_AUTH_SUCCESS) {
        message("auth failed");
        return;
    } else {
        message("auth succeeded");
    }

    ssh::Channel* channel = new ssh::Channel(session);
    if (!channel) {
        message("cannot obtain channel");
        return;
    } else {
        message("channel obtained");
    }

    if (channel->openSession() != SSH_OK)
        message("cannot open channel session");
    else
        message("channel opened.");

    QByteArray commandArray = QString("ls").toUtf8();
    if (channel->requestExec(commandArray.data()) != SSH_OK) {
        message("command failed");
    } else {
        message("command succeeded");
    }

    message("Reading stdout");
    QByteArray result;
    char buffer[256];
    int nbytes = channel->read(buffer, sizeof(buffer), false, -1);
    message("read " + QString::number(nbytes) + " bytes");
    while (nbytes > 0)
    {
      result.append(buffer);
      nbytes = channel->read(buffer, sizeof(buffer), false, -1);
      message("read " + QString::number(nbytes) + " bytes");
    }
    message("Command output (" + QString::number(result.length()) + "):");
    message(result);

    message("Reading stderr");
    QByteArray err;
    nbytes = channel->read(buffer, sizeof(buffer), true, -1);
    message("read " + QString::number(nbytes) + " bytes");
    while (nbytes > 0)
    {
        err.append(buffer);
        nbytes = channel->read(buffer, sizeof(buffer), true, -1);
        message("read " + QString::number(nbytes) + " bytes");
    }
    message("Stderr (" + QString::number(err.length()) + "):");
    message(err);

    channel->close();
    delete channel;
    session.disconnect();
}



On 11/21/2015 02:32 PM, Matteo wrote:
> Hi Aris,
>
> Yes, I get the very same behavior in the Android emulator (ARMv7, didn't
> try x86 yet).
>
> App crashes if ssh_set_log_callback() is commented out, but works otherwise.
> This does not change if verbosity is set to -1 (or to 0).
>
> Please find the apk under the link below, let me know if you want the
> source code, too.
> https://app.box.com/s/q9aivh4etopcbhjbabqs6d53t2qyazwm
> Matteo
>
>
> On 11/21/2015 10:24 AM, Aris Adamantiadis wrote:
>> Hi Matteo,
>>
>> This is really strange. All _ssh_log is doing is preparing a buffer,
>> it's not even the function that's writing on stderr. Does it still
>> happen if you set the verbosity to -1 (absolutely silent) ?
>>
>> Can you reproduce the issue in the android emulator? if so, could you
>> share the APK ?
>>
>> Thanks,
>>
>> Aris
>>
>> On 21/11/15 03:24, Matteo wrote:
>>> Hi Aris,
>>>
>>> Thank you for your interest, and for a great library.
>>>
>>> I have good news: it seems to work now.
>>> And the funny thing is, it seems to have something to do with logging.
>>>
>>> Quick recap here, but feel free to ask for more informations:
>>> The problem is that, unfortunately, my device is not rooted, and thus
>>> access to logs and stacktraces is limited.
>>> So, while trying to find an efficient way to log library events, I
>>> stumbled upon the ssh_set_log_callback() function.
>>> That seemed perfect in this case, as I can redirect the log messages to
>>> screen and to a "private" log file that the app has access to.
>>>
>>> So I quickly implemented it into the tester program and... to my
>>> surprise, no crash anymore!
>>> I could successfully establish a connection, as confirmed also by the
>>> server logs.
>>>
>>> Here the working code:
>>> void callback(int priority, const char *function, const char *buffer,
>>> void *userdata)
>>> {
>>>     //this function can be empty, does not make any difference
>>> }
>>>
>>> void MainWindow::test()
>>> {
>>>     ssh::Session session;
>>>     session.setOption(SSH_OPTIONS_TIMEOUT, 20);
>>>     session.setOption(SSH_OPTIONS_HOST, "192.168.0.13");
>>>     session.setOption(SSH_OPTIONS_USER, "root");
>>>     session.setOption(SSH_OPTIONS_PORT, 2222);
>>>
>>>     //added these lines for logging
>>>     session.setOption(SSH_OPTIONS_LOG_VERBOSITY, 2);
>>>     ssh_set_log_level(SSH_LOG_FUNCTIONS);
>>>     ssh_set_log_callback(callback); //commenting out this line causes
>>> the crash
>>>
>>>     session.connect();
>>> }
>>>
>>> My guess is that libSSH is logging to some facility that, on Android,
>>> the app does not have access to.
>>> Please find attached an excerpt from the Android logcat with a stacktrace.
>>> The log starts with a "touch event" corresponding to pushing the button
>>> connected to MainWindow::test(), and ends with the app crashing.
>>> From the stacktrace, one can see that the last called function seems to
>>> be _ssh_log().
>>>
>>> Tomorrow I will try to slowly expand the test to a more complete use case.
>>>
>>> Regards
>>> Matteo
>>>
>>> On 11/20/2015 07:34 PM, Aris Adamantiadis wrote:
>>>> Hi Matteo,
>>>>
>>>> You did a great work compiling libssh for android. I have no idea what's
>>>> causing the crash, it would greatly help if you could provide us with a
>>>> stacktrace. Running libssh in verbose mode and capturing the output
>>>> would also help figuring out what works .
>>>>
>>>> Aris
>>>>
>>>> On 20/11/15 18:46, Matteo wrote:
>>>>> After several attempts, I could get a successful build.
>>>>> However, when I test it, it segfaults.
>>>>>
>>>>> In the following, I will try to explain what I did.
>>>>> I will try to be brief but, if there is any interest, I can provide
>>>>> more detailed instructions (or even a script that automates the whole
>>>>> process).
>>>>>
>>>>> 0) The build environment is as follows:
>>>>> - Ubuntu 15.04 64 bit with 3.19.0-33 kernel
>>>>> - cmake 3.0.2 and gcc 4.9.2 (both from repos)
>>>>> - Android NDK r9d
>>>>> - OpenSSL 1.0.2d
>>>>> - libSSH 0.7.2
>>>>> - Qt 5.4 and QtCreator 3.3.0
>>>>> - Target platform arm-linux-androideabi-4.8 API 19
>>>>> - Target device Samsung Galaxy S5 klte
>>>>>
>>>>> 1) First, we need to build the dependencies.
>>>>> zlib is already part of the NDK, so we only need to cross-compile OpenSSL.
>>>>> This is easily accomplished by following the instructions found here:
>>>>> https://wiki.openssl.org/index.php/Android
>>>>> After installation I symlinked libcrypto.so, libssl.so and the include
>>>>> files inside the NDK "usr" folder (so that they can easily be linked
>>>>> against).
>>>>>
>>>>> 2) Following Daniel Kroker's advice, I added the taka-no-me toolchain
>>>>> into the libSSH folder.
>>>>> Before I could complete the build, however, I had to solve following
>>>>> problems.
>>>>>
>>>>> 3) My version of cmake has a bug that was fixed in 3.1 an later versions.
>>>>> This prevents it to correctly detect OpenSSL 1.0.2 (it is reported to
>>>>> work with earlier versions, though).
>>>>> To fix it, I manually applied this patch:
>>>>> https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=c5d9a8283cfac15b4a5a07f18d5eb10c1f388505
>>>>>
>>>>> 4) The cmake configuration of libSSH contains a TRY_RUN() step.
>>>>> As far as I could understand, this is supposed to run a dummy test to
>>>>> check if a certain feature works correctly.
>>>>> This is obviously not possible in cross-compilation mode: fortunately,
>>>>> the error issued by cmake contains an explanation on how to fix it.
>>>>> In the libSSH folder one needs to create a file (named e.g.
>>>>> "TryRunResults.cmake") with following content:
>>>>>         set( THREADS_PTHREAD_ARG 0
>>>>>              CACHE STRING "Result from TRY_RUN" FORCE)
>>>>>
>>>>> 5) The build still fails with error "implicit declaration of function
>>>>> 'getpwuid_r'"
>>>>> This function is supposed to return informations about the current
>>>>> user, like username and home directory.
>>>>> This has no equivalent on android, so I needed to change these two
>>>>> functions in the file "misc.c":
>>>>>    char *ssh_get_user_home_dir(void)
>>>>>    char *ssh_get_local_username(void)
>>>>> For my tests, I resigned to return hard-coded values, but more work is
>>>>> required for a general solution.
>>>>>
>>>>> 6) Now the build completes successfully. These are the command issued:
>>>>> cd /path/to/downloaded/libSSH/
>>>>> mkdir build
>>>>> cd build
>>>>>         cmake \
>>>>>             -C ../TryRunResults.cmake \
>>>>>             -DCMAKE_INSTALL_PREFIX=/opt/libssh-android \
>>>>>             -DWITH_INTERNAL_DOC=OFF \
>>>>>             -DWITH_STATIC_LIB=ON \
>>>>>             -DWITH_TESTING=OFF \
>>>>>             -DWITH_SERVER=OFF \
>>>>>             -DWITH_EXAMPLES=OFF \
>>>>>             -DCMAKE_BUILD_TYPE=Release \
>>>>>             -DCMAKE_TOOLCHAIN_FILE=../android.toolchain.cmake \
>>>>>             -DANDROID_NDK="$ANDROID_NDK_ROOT" \
>>>>>             -DANDROID_NATIVE_API_LEVEL=android-19 \
>>>>>             -DANDROID_TOOLCHAIN_NAME=arm-linux-androideabi-4.8 \
>>>>>             -DANDROID_ABI="armeabi-v7a with NEON" \
>>>>>         ..
>>>>>         cmake --build .
>>>>>
>>>>> 7) Before deployment, there are two more obstacles.
>>>>> First, the Android NDK doesnt provide OpenSSL, but the library is
>>>>> installed in Android and loaded by default (taking precedence over
>>>>> libraries packaged in the APK).
>>>>> This means that we have to rename the libcrypto.so and libssl.so to
>>>>> something else, and repeat the linking step. However, see below.
>>>>> (this problem is documented here:
>>>>> https://mta.openssl.org/pipermail/openssl-users/2015-April/001183.html)
>>>>> Second, Android does not support loading of versioned libraries
>>>>> (meaning that e.g. libssh.so is a symlink to libssh.so.4.4.0).
>>>>> This means that we have to configure OpenSSL and libSSH to produce
>>>>> non-versioned files, and relink. However, see below.
>>>>> (this problem is documented here:
>>>>> https://bugreports.qt.io/browse/QTCREATORBUG-11062)
>>>>> The link above also provided a solution (hack?) for both problems,
>>>>> which I implemented with following BASH code:
>>>>>     local subs=()
>>>>>     subs+=(-e
>>>>> 's/libcrypto.so.1.0.0/libcryptm.so\x00\x00\x00\x00\x00\x00/g')
>>>>>     subs+=(-e 's/libcrypto.so/libcryptm.so/g')
>>>>>     subs+=(-e 's/libssl.so.1.0.0/libssm.so\x00\x00\x00\x00\x00\x00/g')
>>>>>     subs+=(-e 's/libssl.so/libssm.so/g')
>>>>>     subs+=(-e 's/libssh.so.4.4.0/libssh.so\x00\x00\x00\x00\x00\x00/g')
>>>>>     subs+=(-e 's/libssh.so.4/libssh.so\x00\x00/g')
>>>>>     sed "${subs[@]}" "/opt/openssl/android-19/lib/libcrypto.so" >
>>>>> "/home/matteo/deploy/libcryptm.so"
>>>>>     sed "${subs[@]}" "/opt/openssl/android-19/lib/libssl.so" >
>>>>> "/home/matteo/deploy/libssm.so"
>>>>>     sed "${subs[@]}" "/opt/libssh-android/lib/libssh.so" >
>>>>> "/home/matteo/deploy/libssh.so"
>>>>>     chmod a+x /home/matteo/deploy/*
>>>>>
>>>>> 8) I prepared a very simple test project in Qt.
>>>>> Starting from the widget-based template for android, I added a
>>>>> function that, when clicking on a QPushButton, attempts to connect to
>>>>> a ssh server in my LAN.
>>>>> void MainWindow::test()
>>>>> {
>>>>>     ssh::Session session;
>>>>>     session.setOption(SSH_OPTIONS_TIMEOUT, 20);
>>>>>     session.setOption(SSH_OPTIONS_HOST, "192.168.0.13");
>>>>>     session.setOption(SSH_OPTIONS_USER, "root");
>>>>>     session.setOption(SSH_OPTIONS_PORT, 2222);
>>>>>
>>>>>     session.connect();
>>>>> }
>>>>>
>>>>> This works if "session.connect();" is commented out, but the program
>>>>> crashes otherwise, with error:
>>>>> F/libc    (22871): Fatal signal 11 (SIGSEGV), code 1, fault addr
>>>>> 0x10100 in tid 23039 (QtThread)
>>>>> Note that the very same code works in "desktop mode".
>>>>>
>>>>> Right now I am out of ideas on how to debug this, so if someone can
>>>>> give an hint, I would much appreciate it.
>>>>> As I said above, I can provide more details and code upon request.
>>>>>
>>>>> Thanks in advance
>>>>> Matteo
>>>>>
>>>>> On 11/19/2015 09:28 AM, Daniel Kroker wrote:
>>>>>> Hi,
>>>>>>
>>>>>> its not so easy to build libssh for android. you need the android
>>>>>> toolchain look at
>>>>>> https://github.com/taka-no-me/android-cmake/blob/master/android.toolchain.cmake
>>>>>>
>>>>>> Am 19.11.2015 um 02:35 schrieb Matteo:
>>>>>>> Hi all,
>>>>>>>
>>>>>>> My project depends on libssh and I am trying to port it to Android.
>>>>>>>
>>>>>>> After some research in the Internet, it appears that I have to
>>>>>>> cross-compile libssh by using the Android NDK.
>>>>>>> However, I could not find any good sources of information about the
>>>>>>> topic.
>>>>>>>
>>>>>>> Could somebody please provide a starting point for me?
>>>>>>>
>>>>>>> Thank you in advance.
>>>>>>> Mat
>>>>>>>
>>>>>>>
>>