Can't Install Minishift: Pre-flight check for persistent storage volume shows "Insufficient disk space"

[karlnicholas]

• Minishift version: 1.11.0
• Windows 10 Home - Version 1709 - Build 16299.125 (up to date)
• VirtualBox - Version 5.2.6 r120293 (Qt5.6.2)

1. Download Minishift
2. Run minishift start --vm-driver=virtualbox
   It ran OK once I think last week, and worked all day, but then the problem happened again the next day.
   This happened again just now:

C:\Users\karln>minishift start --vm-driver=virtualbox
-- Starting profile 'minishift'
-- Checking if requested hypervisor 'virtualbox' is supported on this platform ... OK
-- Checking if VirtualBox is installed ... OK
-- Checking the ISO URL ... OK
-- Starting local OpenShift cluster using 'virtualbox' hypervisor ...
-- Minishift VM will be configured with ...
   Memory:    2 GB
   vCPUs :    2
   Disk size: 20 GB

   Downloading ISO 'https://github.com/minishift/minishift-b2d-iso/releases/download/v1.2.0/minishift-b2d.iso'
 40.00 MiB / 40.00 MiB [===================================================================================] 100.00% 0s
-- Starting Minishift VM ............................ OK
-- Checking for IP address ... OK
-- Checking if external host is reachable from the Minishift VM ...
   Pinging 8.8.8.8 ... OK
-- Checking HTTP connectivity from the VM ...
   Retrieving http://minishift.io/index.html ... OK
-- Checking if persistent storage volume is mounted ... OK
-- Checking available disk space ... warning:
 used FAIL
   Insufficient disk space on the persistent storage volume

[gbraad]

Please check the documentation on how to handle this situation:

https://docs.openshift.org/latest/minishift/troubleshooting/troubleshooting-getting-started.html#persistent-storage-check

Depending on the hypervisor you need to either enlarge the disk, or choose to copy data out and recreate with a larger size. We are not able to perform the changing of the disk size.

[gbraad]

Please check the documentation on how to handle this situation:

https://docs.openshift.org/latest/minishift/troubleshooting/troubleshooting-getting-started.html#persistent-storage-check

Depending on the hypervisor you need to either enlarge the disk, or choose to copy data out and recreate with a larger size. We are not able to perform the changing of the disk size.

[gbraad]

Marking as invalid, as this works 'as expected'. When the disk is full, unpredictible behaviour will occur, and with this check we want to inform the user beforehand.

We might want to include a link or reference to the troubleshooting segment in the error.

[gbraad]

Marking as invalid, as this works 'as expected'. When the disk is full, unpredictible behaviour will occur, and with this check we want to inform the user beforehand.

We might want to include a link or reference to the troubleshooting segment in the error.

[budhram]

We might want to include a link or reference to the troubleshooting segment in the error.

Sure. Hence, changed the labels.

[budhram]

We might want to include a link or reference to the troubleshooting segment in the error.

Sure. Hence, changed the labels.

[karlnicholas]

Detailed log of disk check space using

minishift start --vm-driver virtualbox --show-libmachine-logs -v5

-- Checking available disk space ... (minishift) Calling .GetSSHHostname
(minishift) Calling .GetSSHPort
(minishift) Calling .GetSSHKeyPath
(minishift) Calling .GetSSHKeyPath
(minishift) Calling .GetSSHUsername
Using SSH client type: external
&{[-F /dev/null -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none docker@127.0.0.1 -o IdentitiesOnly=yes -i C:\Users\karln\.minishift\machines\minishift\id_rsa -p 65245] C:\Program Files\OpenShift\bin\ssh.exe <nil>}
About to run SSH command:
df -h /mnt/sda1 | awk 'FNR > 1 {print $2,$5}'
SSH cmd err, output: <nil>: cygwin warning:
  MS-DOS style path detected: C:\Users\karln\.minishift\machines\minishift\id_rsa
  Preferred POSIX equivalent is: /cygdrive/c/Users/karln/.minishift/machines/minishift/id_rsa
  CYGWIN environment variable option "nodosfilewarning" turns off this warning.
  Consult the user's guide for more details about POSIX paths:
    http://cygwin.com/cygwin-ug-net/using.html#using-pathnames
17.9G 0%

warning:
 used FAIL
   Insufficient disk space on the persistent storage volume

[karlnicholas]

Detailed log of disk check space using

minishift start --vm-driver virtualbox --show-libmachine-logs -v5

-- Checking available disk space ... (minishift) Calling .GetSSHHostname
(minishift) Calling .GetSSHPort
(minishift) Calling .GetSSHKeyPath
(minishift) Calling .GetSSHKeyPath
(minishift) Calling .GetSSHUsername
Using SSH client type: external
&{[-F /dev/null -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none docker@127.0.0.1 -o IdentitiesOnly=yes -i C:\Users\karln\.minishift\machines\minishift\id_rsa -p 65245] C:\Program Files\OpenShift\bin\ssh.exe <nil>}
About to run SSH command:
df -h /mnt/sda1 | awk 'FNR > 1 {print $2,$5}'
SSH cmd err, output: <nil>: cygwin warning:
  MS-DOS style path detected: C:\Users\karln\.minishift\machines\minishift\id_rsa
  Preferred POSIX equivalent is: /cygdrive/c/Users/karln/.minishift/machines/minishift/id_rsa
  CYGWIN environment variable option "nodosfilewarning" turns off this warning.
  Consult the user's guide for more details about POSIX paths:
    http://cygwin.com/cygwin-ug-net/using.html#using-pathnames
17.9G 0%

warning:
 used FAIL
   Insufficient disk space on the persistent storage volume

[karlnicholas]

Okay, I'm pretty sure I found the problem. I Set CYGWIN=nodosfilewarning in my environment because without it the SSH I have install was generating some kind of warning. Minishift seems to be installed and running OK now.

-- Checking available disk space ... (minishift) Calling .GetSSHHostname
(minishift) Calling .GetSSHPort
(minishift) Calling .GetSSHKeyPath
(minishift) Calling .GetSSHKeyPath
(minishift) Calling .GetSSHUsername
Using SSH client type: external
&{[-F /dev/null -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none docker@127.0.0.1 -o IdentitiesOnly=yes -i C:\Users\karln\.minishift\machines\minishift\id_rsa -p 49174] C:\users\karln\OpenShift\bin\ssh.exe <nil>}
About to run SSH command:
df -h /mnt/sda1 | awk 'FNR > 1 {print $2,$5}'
SSH cmd err, output: <nil>: 17.9G 0%

0% used OK

[karlnicholas]

Okay, I'm pretty sure I found the problem. I Set CYGWIN=nodosfilewarning in my environment because without it the SSH I have install was generating some kind of warning. Minishift seems to be installed and running OK now.

-- Checking available disk space ... (minishift) Calling .GetSSHHostname
(minishift) Calling .GetSSHPort
(minishift) Calling .GetSSHKeyPath
(minishift) Calling .GetSSHKeyPath
(minishift) Calling .GetSSHUsername
Using SSH client type: external
&{[-F /dev/null -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none docker@127.0.0.1 -o IdentitiesOnly=yes -i C:\Users\karln\.minishift\machines\minishift\id_rsa -p 49174] C:\users\karln\OpenShift\bin\ssh.exe <nil>}
About to run SSH command:
df -h /mnt/sda1 | awk 'FNR > 1 {print $2,$5}'
SSH cmd err, output: <nil>: 17.9G 0%

0% used OK

[gbraad]

This is a new instance? Set the test to skip for now. Or can you try starting with '--iso-url centos' ? Something ain't right, since out of the box, with 20G it should surely work. I use even less space for test instances.

…

-- Gerard Braad | http://gbraad.nl [ Doing Open Source Matters ]

[gbraad]

This is a new instance? Set the test to skip for now. Or can you try starting with '--iso-url centos' ? Something ain't right, since out of the box, with 20G it should surely work. I use even less space for test instances.

…

-- Gerard Braad | http://gbraad.nl [ Doing Open Source Matters ]

[gbraad]

Which ssh client did you install? We have tested cygwin and this should be OK. Things like Gow is known to fail. Also, openssh-win64 is OK

…

-- Gerard Braad | http://gbraad.nl [ Doing Open Source Matters ]

[gbraad]

Which ssh client did you install? We have tested cygwin and this should be OK. Things like Gow is known to fail. Also, openssh-win64 is OK

…

-- Gerard Braad | http://gbraad.nl [ Doing Open Source Matters ]

[gbraad]

Ask as I not completely following point 2:

openshift/bin/ssh.exe to c:\users\karln (from c:\program files)

openshift/bin/ssh.exe ?

…

-- Gerard Braad | http://gbraad.nl [ Doing Open Source Matters ]

[gbraad]

Ask as I not completely following point 2:

openshift/bin/ssh.exe to c:\users\karln (from c:\program files)

openshift/bin/ssh.exe ?

…

-- Gerard Braad | http://gbraad.nl [ Doing Open Source Matters ]

[karlnicholas]

the ssh.exe "installed" was from cwrsync which I added to the OC bin directory so that I could use rsync with OC to manage my permanent stores on openshift. (https://www.itefix.net/content/cwrsync-free-edition). I just DL's those binaries and copied them into the openshift/bin directory. I don't have ssh installed in Cygwin but maybe the cwrsync version uses a copy of that binary or something.

Actually, I have just DL'd the redhat Container Development Kit Version (CDK) of minishift (https://developers.redhat.com/download-manager/file/cdk-3.2.0-1-minishift-windows-amd64.exe) so that I can use all this in DevStudio, and it's all working OK now. Personally, I'm good to go and I probably saw it all working once when I initially installed the DevStudio CDK Bundle which comes with cygwin and then changed things back to my own installation of Cygwin and got hung up on the SSH issue. Sorry for the confusion -- possibly another fix might be to have the Cygwin binaries in your path and be sure SSH is installed there.

There is a another version of SSH I have which is Microsoft's beta SSH you can get by adding features in Windows/Setup but that's later in the PATH and so doesn't get executed. I didn't try it to see if it ran the minishift installation correctly.

[karlnicholas]

the ssh.exe "installed" was from cwrsync which I added to the OC bin directory so that I could use rsync with OC to manage my permanent stores on openshift. (https://www.itefix.net/content/cwrsync-free-edition). I just DL's those binaries and copied them into the openshift/bin directory. I don't have ssh installed in Cygwin but maybe the cwrsync version uses a copy of that binary or something.

Actually, I have just DL'd the redhat Container Development Kit Version (CDK) of minishift (https://developers.redhat.com/download-manager/file/cdk-3.2.0-1-minishift-windows-amd64.exe) so that I can use all this in DevStudio, and it's all working OK now. Personally, I'm good to go and I probably saw it all working once when I initially installed the DevStudio CDK Bundle which comes with cygwin and then changed things back to my own installation of Cygwin and got hung up on the SSH issue. Sorry for the confusion -- possibly another fix might be to have the Cygwin binaries in your path and be sure SSH is installed there.

There is a another version of SSH I have which is Microsoft's beta SSH you can get by adding features in Windows/Setup but that's later in the PATH and so doesn't get executed. I didn't try it to see if it ran the minishift installation correctly.

[karlnicholas]

In terms of setting the test to skip I had tried that but still had problems. I presume again that wherever the installation was checking the results from the SSH I was using was failing and so still giving problems.

Probably I'll "bite the bullet" and install ssh.exe in Cygwin and delete the weird one I ended up DL'ing.

[karlnicholas]

In terms of setting the test to skip I had tried that but still had problems. I presume again that wherever the installation was checking the results from the SSH I was using was failing and so still giving problems.

Probably I'll "bite the bullet" and install ssh.exe in Cygwin and delete the weird one I ended up DL'ing.

[gbraad]

Windows 10's SSH beta should work. Has been verified and tested by me for quite a while as part of the insider program. Cygwin is still preferred at the moment, as many of the ssh clients on Windows give weird results or unexpected output.

…

-- Gerard Braad | http://gbraad.nl [ Doing Open Source Matters ]

[gbraad]

Windows 10's SSH beta should work. Has been verified and tested by me for quite a while as part of the insider program. Cygwin is still preferred at the moment, as many of the ssh clients on Windows give weird results or unexpected output.

…

-- Gerard Braad | http://gbraad.nl [ Doing Open Source Matters ]

[karlnicholas]

Right - It's not obvious in the documentation anywhere that minishift depends on an installed SSH application and since it's an executable you tend to think it has everything it needs built in. This would go well on the front page.

Also, it might be a good idea to set the CYGWIN=nodosfilewarning env variable when running, if possible. I think the issue is that the proper Cygwin SSH has it defaulted to (no)dosfilewarning and the version I picked up has it defaulted to on for some reason. I don't think it can hurt anything. Perhaps I can submit a pull request but I'm not sure where to start looking for setting initial variables. Thanks for the follow-up.

Good to know about the SSH beta but I deleted it anyway because it doesn't support rsa.

[karlnicholas]

Right - It's not obvious in the documentation anywhere that minishift depends on an installed SSH application and since it's an executable you tend to think it has everything it needs built in. This would go well on the front page.

Also, it might be a good idea to set the CYGWIN=nodosfilewarning env variable when running, if possible. I think the issue is that the proper Cygwin SSH has it defaulted to (no)dosfilewarning and the version I picked up has it defaulted to on for some reason. I don't think it can hurt anything. Perhaps I can submit a pull request but I'm not sure where to start looking for setting initial variables. Thanks for the follow-up.

Good to know about the SSH beta but I deleted it anyway because it doesn't support rsa.

[gbraad]

We do not need to have SSH installed, as we can fall back to a SSH-implementation inside the minishift executable. However, as soon as an ssh.exe is detected on PATH, we use that instead. On Linux or macOS this 'never' causes an issue, but having a plethora of options of ssh clients on Windows creates a problem for us while testing. So, yes... we have some issues to add supported SSH clients to the documentation. it would be great to see a PR ;-)

[gbraad]

We do not need to have SSH installed, as we can fall back to a SSH-implementation inside the minishift executable. However, as soon as an ssh.exe is detected on PATH, we use that instead. On Linux or macOS this 'never' causes an issue, but having a plethora of options of ssh clients on Windows creates a problem for us while testing. So, yes... we have some issues to add supported SSH clients to the documentation. it would be great to see a PR ;-)

[LalatenduMohanty]

Resolved by #1952

[LalatenduMohanty]

Resolved by #1952