New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ESXi 6.7 does not support VNC #6482
Comments
Are you sure? Do you have any official references? There have previously been some confusion around this, for example. |
In fact we can build with ESXi 6.0 without any problem, but not with ESXi 6.7, and ALL settings (firewall, GuestIPHack, etc. is 100% identical)! On 6.7 packer puts the VMDK file on the ESXi datastore, but not the VMX file (it does exist locally in %TEMP%). Then packer tries to connect VNC, which certainly fails. So it looks like the VMX upload is the problem. Strange, because we run as 'root'. |
Hi, I have the same feeling than mkarg and a quick research has spotten me to thread on reddit providing the same information : https://www.reddit.com/r/esxi/comments/6klauz/esxi_65_vnc/. No luck in finding official reference, but the option was available only on the old clients that are now superseeded by an HTML5 version without this option but providing an official SDK for console access: https://www.vmware.com/support/developer/html-console/html-console-21-releasenotes.html It seems that the JetBrains packer plugin using direct API calls is also using this SDK, you should give a try. This SDK is not avaible in 6.0 and therefore this packer plugin is not working with version under 6.5 and the maintainer doesn't seems to be willing to do anything for this. Here the issue corresponding to this limitation: jetbrains-infra/packer-builder-vsphere#63. I hope someday this plugin will be part of core packer since it is more avanced that my try to provide a such feature. |
I just spun up a nested ESXi 6.5 (yes I know, I'll try 6.7 next) and installed a vib file I made to configure the necessary things for Packer. It looks like authentication is failing for VNC:
I'll try 6.7 next. |
Same error with 6.7
|
In our case (the one this issue originated from), the last line is "Starting HTTP server on port 8619" but there never is a line "Registering remote VM" nor "Starting virtual machine". In fact, packer directly tries to start VNC, which is pretty weird. Again, using 6.0 it works, the problem is onyl with 6.7. |
Please supply the information requested in the issue template: |
@mkarg I was able to build on 6.7 in my environment once I set @rickard-von-essen It appears that ESXi 6.5+ ignores any VNC password configured. I ran with |
@adarobin great! |
This is an issue that's been know about for quite some time - see #5580. As noted, the docs for the VMware ISO builder haven't been updated to let people know about this - I struggled for a while with this myself! If I remember rightly the issue affects ESXi 6.0 and up... There are a number of other issues with the VMware ISO docs when building on vSphere/ESXi that I was going to include in a PR but unfortunately haven't had the time to do - sorry @mkarg! Mac users should note that the VNC viewer shipped with OSX can't be used when no password has been set! TigerVNC works fine though. |
@mkarg Just in case, if you are using a vDS and have problems, the solution is to set the port binding type for the port group to 'ephemeral' - See issue #2715 and THIS VMware doc for background. Thought I'd better mention this as this is the next issue I ran into after solving the VNC one! @adarobin This issue and the solution is also missing from the docs if you fancy rolling that into your PR... For completeness I needed to set the following in
|
|
That debug log you originally had in your comment was telling you that you need to configure the firewall rules on your ESX server to allow Packer to connect in over VNC. You need to ensure the rules persist across reboots - by default ESXi will remove any customisations to firewall rules when you reboot. I've used those notes to configure my own system. Clearly, any changes you make to your system are at your own risk... 😉 |
@DanHam We meanwhile found out that all works well unless we reboot ESXi. So the actual sole problem simply is that the services.xml gets overwritten (or "reset") on ESXi reboot. We are now investigating ways to work around that. |
Sorry about this being sorta late, but is this because go-vnc doesn't support TLS* for the authentication scheme? What authentication schemes does ESXi 6.7 support? |
@arizvisa Since VNC isn't officially supported on ESXi, I don't think the authentication schemes are published anywhere. For that matter, I am able to authenticate to ESXi 6.5 and 6.7 without a entering a password even though a password is specified in the virtual machine config. |
@adarobin, one way to test would be to try specifying the different auth schemes individually and seeing which ones work. I don't have an esx instance to test against right now (or I'd do it myself), but manpage for tigervnc offers:
Another way (which I think depends on your vnc client) is that with Looking at go-vnc's source, it looks like there aren't many authentication schemes supported but fortunately mitchellh@ wrote it so that it's easily pluggable. So, adding support for more schemes shouldn't require any more work than just the authentication. |
If somebody wants to let me know if the issue is due to go-vnc not supporting a particular auth protocol for esxi6.7 (TLS*) and what auth protocols esxi6.7 supports for vnc (via my above message), lmk. I'm willing to implement it for go-vnc because I'm a protocol junkie and it'll actually coincide with my real job. |
@arizvisa ESXi 6.5 is only offering None for the authentication type. I don't have 6.7 handy to test, but I suspect it is the same. For ESXi 6.5
For ESXi 6.0
|
Awesome. Thanks @adarobin for doing that. But ah man, that sucks. i guess there's another protocol that we'll need to use. I saw some references to PCoIP and port 903. I'll do some digging. If there's another protocol to dick with I can probably intersect Thanks again guys! |
Actually, it apparently is simpler than I had originally thought. Apparently it's using mks which is just a different authentication scheme (maybe it can be integrated into the auth mechanism for go-vnc somehow despite it being a differing protocol). Someone actually implemented mks in python too: |
Another idea I just had would be tunneling the VNC connection over the SSH connection we already have open to the host. This would help keep things secure and negate the need to even open VNC in the ESXi firewall. |
yea. probably a better solution to not expose it to just anybody. i personally wouldn't trust VMware's VNC implementation against vulnerabilities. I mean, most of the potentially vulnerable surface is post-auth, but still.. you never know. i'll be lookin' into implementing mks for go-vnc (or packer) to support remote console against esxi properly, so expect that sometime in the future. |
driver_esx and related were a hack anyways. |
Back when the vsphere-iso builder was maintained by jetbrains they used a USB HID feature to enter input for the build via the console. I don't know if this feature carried over when HashiCorp took over. |
As far as I can see it did. |
We're working on tweaking the vsphere-iso builder to work with individual esxi hosts' web clients rather than only working with vCenter; since the vsphere-iso builder uses the usb keyboard instead of VNC, that will probably be our recommended path forward here. |
Indeed you are, see #9791. Today I successfully tested a build of the vsphere-iso builder that works with a single ESXi host. The USB HID scan codes work great. Demo here: https://www.youtube.com/watch?v=vIerwfvCXGg&list=PLHg__Q91ZGND0xLY27JV__GeMO9vBIQTx&index=2 |
Can some one please clarify why is it better to use individual ESXi hosts rather than vCenter, it is not clear to me. |
My guess would be you can run ESXi with a free license and not have to run a separate vCenter. |
However vCenter adds some features that don't come with ESXi itself one would mostly use vCenter to manage a cluster of ESXi servers. If you only need a single server you don't need the licence and you don't need the extra resources required to run vCenter. @chewrocca Yes, you could run ESXi server without a license. However you wouldn't be able to run Packer against that server because the API calls on an ESXi server without a license are read-only. |
As a follow up to what @SwampDragons and @swerveshot replied here about using vsphere-iso with a single ESXi host, here are the binaries that you can use to test if the changes for #9791 works for you too. |
I have been working with packer and vmware-iso builders on remote ESXi for a while now. 6.0, 6.5 worked fine with GuestIPHack and manually adding VNC port range to /etc/vmware/firewall/service.xml file. Then I upgraded ESXi to 6.7, 6.7U2 and finally to 7.0. Now I'm trying to run packer and create CentOS 7 vm and ESXi firewall blocks it. After some researching found one interesting post How to create persistent firewall rules on ESXi. The main idea there is to create and import custom VIB file. Then custom VIBName.xml will be created under /etc/vmware/firewall and here you go: custom firewall rule set is there. Wrapping-up: Is it possible to create PackerVNC.vib to be imported into vmware as persistent firewall rule set? |
@nimDevOps I made a vib which is already linked to somewhere near the top of this issue. That said, VNC support is removed from 7.0 completely so it won't help you. |
Thanks @adarobin ! I guest I will need to try vsphere-iso then. |
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further. |
Apparently ESXi 6.7 does not support VNC anymore! How to execute boot_command on ESXi 6.7?
The text was updated successfully, but these errors were encountered: