Re: client testsuite with sshd privilege separation

[Jakub Jelen <jjelen@xxxxxxxxxx>]

On Fri, 2017-11-24 at 07:20 +0100, Andreas Schneider wrote:
> On Wednesday, 22 November 2017 15:51:24 CET Jakub Jelen wrote:
> > On Tue, 2017-11-21 at 15:10 +0100, Jakub Jelen wrote:
> > > On Mon, 2017-11-20 at 23:11 +0100, Andreas Schneider wrote:
> > > > On Monday, 20 November 2017 18:03:59 CET Jakub Jelen wrote:
> > > > > Hello all,
> > > > > I am trying to run the libssh client testsuite with latest
> > > > > OpenSSH,
> > > > > which does not support running without privilege separation.
> > > > > According
> > > > > to OpenSSH upstream, it should not be a problem to run it as
> > > > > an
> > > > > unprivileged user, but whatever I do, I am still getting the
> > > > > following
> > > > > 
> > > > > error:
> > > > >   Bind to port 22 on 127.0.0.10 failed: Permission denied.
> > > > > 
> > > > > I ruled out SELinux already, I tried to add socket_wrapper
> > > > > debug
> > > > > environment variable, but still it does not generate any
> > > > > output.
> > > > > strace
> > > > > is not showing anything suspicious. I am out of ideas what
> > > > > else
> > > > > could
> > > > > prevent server starting. On what else is cwrap/socket_wrapper
> > > > > depending
> > > > > that could be stripped by the OpenSSH server? Note that this
> > > > > is
> > > > > happening basically before the privilege separation is being
> > > > > effective.
> > > > 
> > > > I think it clears the env, so LD_PRELOAD is not set and
> > > > socket_wrapper not
> > > > loaded.
> > > 
> > > Nope. OpenSSH does not touch environment. Once I was debugging
> > > the
> > > issue, I noticed, that the LD_PRELOAD is set up from Makefiles,
> > > but
> > > some other environment variables from the code and therefore when
> > > I
> > > was
> > > running the single test manually, it did not get used at all.
> > > 
> > > The real problem here is the uid_wrapper: As it is set up now, it
> > > is
> > > faking the root UID, therefore SSHD is believing it has a
> > > permissions
> > > to do the chroot, but fails to do that, which is in this late
> > > stage a
> > > fatal error.
> > > 
> > > Not sure what all the UID wrapper is needed here for, but my
> > > proposal
> > > would be to remove it or implement some kind of chroot wrapper to
> > > make
> > > sshd happy.
> > > 
> > > I will have a look into the options, since I believe testing
> > > against
> > > current OpenSSH is something useful. Any ideas opinion on this?
> > 
> > Finally I was able to make the client testsuite pass against
> > current
> > OpenSSH 7.6. There are several things needs to be changed to do so:
> > 
> >  * the chroot_wrapper needs to be loaded with the other wrappers to
> > make sshd happy. I was wondering that this was not yet a problem
> > with
> > other tools tested with cwrap. I will open a bug for cwrap with
> > this
> > feature request.
> > 
> >  * The sandbox privilege separation is setting rlimits
> > (RLIMIT_FSIZE),
> > which is killing privsep child, once socket_wrapper wants to write
> > the
> > PCAP file. I believe it can be useful for debugging issues, but I
> > am
> > not sure if it should come enabled by default. Can this be gated by
> > some environment variable?
> 
> I write a PCAP file by default. SOCKET_WRAPPER_PCAP_FILE is set
> somewhere in 
> tests/torture.c I guess.

With the following patch, I was able to make it working:
https://github.com/Jakuje/libssh/commit/74bef17

> 
> >  * The server sometimes took little bit more time to start and I
> > was
> > hitting occasional "No route to host" errors. Adjusting the sleep
> > to
> > usleep(1000) helped me to pass the whole testsuite.
> > 
> >  * The blowfish-cbc cipher is no longer available in OpenSSH 7.6.
> > There
> > should be possibility to exclude the tests, either automatically by
> > identifying from sshd or manually by configure step? What would be
> > a
> > preferred way?
> 
> What did OpenSSH 7.6 do? Did they completely remove it or just
> disable it by 
> default. There is a function to check the openssh version. Based on
> that we 
> could skip the test and then remove the whole feature after 0.8 has
> been 
> released.

They removed it completely so the configuration you provide to OpenSSH
says unknown cipher:

http://www.openssh.com/txt/release-7.6

Checking OpenSSH version and disabling these test cases sounds like a
good solution. I will send a separate patch.

> We could do a hacking session if you have some time ...

Certainly. I should be able to justify some time. Let me know.

-- 
Jakub Jelen
Software Engineer
Security Technologies
Red Hat, Inc.