Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strange behavior with ssh and man #258

Closed
youurayy opened this issue Aug 5, 2019 · 2 comments

Comments

@youurayy
Copy link

commented Aug 5, 2019

Hi, I'm trying to put together a simple script for quickly spinning up a few barebone Hyperkit instances (ubuntu, cloud-init, vmnet).

However I'm noticing some anomalies that look like VM corruption, so I'm putting this as an issue as maybe someone can point me in the proper direction.

Tested on 2 systems:

  1. MacBook Pro 15-inch 2018, macOS Mojave 10.14.5, APFS filesystem
  2. MacBook Pro 15-inch mid-2015, macOS Mojave 10.14.5, APFS filesystem

Replicate using my script: (usage)

brew install hyperkit
brew install qemu  # needed for creation of cloud-init ISO
curl https://raw.githubusercontent.com/youurayy/k8s-hyperkit/master/hyperkit.sh -O -
chmod +x hyperkit.sh
./hyperkit.sh image  # will download ubuntu into ./tmp
./hyperkit.sh master  # launches hyperkit
# login <yourusername>, password 'test'
# from inside the VM:
# issue 1.:
ssh localhost  # will say: ssh_dispatch_run_fatal: Connection to 127.0.0.1 port 22: incorrect signature
# (^ that's just the minimal case, I've found it by trying to ssh from the outside,
# it happens no matter what kex/ciphers are chosen, seems like something is disrupting the crypto)

# issue 2.:
man bash  # outputs gibberish

Hyperkit invocation (cat ./tmp/master/cmdline):

hyperkit \
-A \
-H \
-U 24AF0C19-3B96-487C-92F7-584C9932DD96 \
-m 4G \
-c 2 \
-s 0:0,hostbridge \
-s 2:0,virtio-net \
-s 31,lpc  \
-l com1,stdio \
-s 1:0,ahci-hd,/Users/juraj/github/k8s-hyperkit/tmp/master/ubuntu-18.04-server-cloudimg-amd64.raw \
-s 5,ahci-cd,/Users/juraj/github/k8s-hyperkit/tmp/master/cloud-init.iso \
-f "kexec,../ubuntu-18.04-server-cloudimg-amd64-vmlinuz-generic,../ubuntu-18.04-server-cloudimg-amd64-initrd-generic,earlyprintk=serial console=ttyS0 root=/dev/sda1" 

I've also tried to compile Hyperkit from source, even a few commits back, but no change.

If your key is at ~/.ssh/id_rsa.pub, it will get exported in the VM and you can try to ssh $USER@192.168.64.X (see /var/db/dhcpd_leases), in which case you'll get the ssh - incorrect signature error above.

Another thing of note is that while this is an APFS filesystem with support for sparse files (and the .raw created by my script is indeed a sparse file), this message appears in the Hyperkit startup output:
fcntl(F_PUNCHHOLE) failed: host filesystem does not support sparse files: Operation not permitted

Thanks, any help is appreciated.

ps. Docker for Mac is not installed on either machine.
ps2. The same Ubuntu image and config works fine on Windows/Hyper-V
ps3. type 'reset'<Enter> if your terminal gets garbled

@youurayy

This comment has been minimized.

Copy link
Author

commented Aug 7, 2019

So far I've narrowed it down to a failing test in Ubuntu's openssh package,

test_sshkey: ...............................
regress/unittests/sshkey/test_sshkey.c:124 test #32 "sign and verify ECDSA" - ECDSA key, banana length 1
ASSERT_INT_EQ(sshkey_verify(k, sig, len, d, l, 0), 0) failed:
sshkey_verify(k, sig, len, d, l, 0) = -21
           0 = 0
Aborted (core dumped)
Makefile:221: recipe for target 'unit' failed
make[1]: *** [unit] Error 134

which is in turn caused by openssl's ECDSA_do_sign / ossl_ecdsa_sign_sig returning a bad signature (notice the numbers are as if weirdly swapped):

# hyperkit (incorrect):
(gdb) print *sig->r->d
$13 = 8936674288764001502
(gdb) print *sig->s->d
$14 = 17802504973525174444
-----------------------
# hyper-v (correct):
(gdb) print *sig->r->d
$9 = 17557672360043182116
(gdb) print *sig->s->d
$10 = 8812791427644150443

I need to find a way how to get to the bottom of this, because when I rebuild the openssl package (maintainer config, debuild -us -uc -b) (and get a new /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0), the issue will not exhibit itself anymore (which isn't an acceptable solution here by any measure).

@youurayy

This comment has been minimized.

Copy link
Author

commented Aug 8, 2019

As mentioned in #183, there's a problem with the default Ubuntu cloud images (extension .img) when run on Hyperkit.
Running from .vmdk converted to either .raw or .qcow2 is okay.

Closing this, but ideally people should be advised.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.