EFLOW - Cannot select another internal vmswitch other then 'Default switch'

[ericvruder]

Question Summary
I would like to be able to select another internal vmswitch other then 'Default switch'. Whenever I do so, it tells me that the only supported switch is 'Default switch'.

Running

Deploy-Eflow -acceptEula 'yes' -acceptOptionalTelemetry 'no' -enableVtpm -vnetType 'ICS' -vnetName 'IoT EFLOW Nat Switch'

gives me

- C:\iotedge\nodectl.exe network vnet create --config "C:\iotedge\\yaml\cloudvnet.yaml" failed to execute [Error: rpc error: code = Unknown desc = VMMS only supports the 'Default Switch' ICS network

Unfortunately, for 'reasons', the 'Default switch' doesn't work for me. It is not capable of providing internet access to any of the other vm's in my setup. I've tried a lot of different things, but nothing seems to work. As soon though as I create my own NAT network, (see https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/user-guide/setup-nat-network) and get my windows vm's to use it, it works.

Issue here though is that the 'Default switch' provides DHCP like functionality, and my own NAT network doesn't. That means that I have to open up the vm and manually assign a ip address, gateway address and DNS server.

I see two issues here:

• I am having a hard time figuring out how to actually setup the network configuration on the CBL-Mariner vm.
• I need this to be automatable. I don't think I can SSH into the vm and then change network configuration, but I don't know how else I can access the vm through powershell and modify those settings, after I figure out how to do so.

I will be attempting to setup a small DHCP server in a seperate vm, and see if that works. If you have any other suggestions, I would really appreciate it.

[TerryWarwick]

@ericvruder,

By any chance are you deploying EFLOW to a Windows VM that is hosted in Azure? There are limitations to the vSwitch options in an Azure VM and it varies by edition of Windows that you are using. If you are doing this all in Azure VMs, it is my understanding that you will definitely need a DHCP server.

We have confirmed that nodectl.exe does not support internal switch named other than "Default Switch". I believe you can configure the vSwitch using PowerShell. Take a look at Set-VMSwitch. Just note that there could be changes in this area yet as we are still in Public Preview.

Terry

[Thacai]

To add to this issue (i'm collegue of Eric), the DHCP in the default switch in hyper-v is frankly not working(gives us only a IPv6 address not IPv4).

• We've been trying this out on both Windows 10 Enterprise, Version 20H2 and 1909, not azure environment.
• • Bug since version 1903
• • possible workaround (mentioned here: https://borncity.com/win/2019/06/18/windows-10-v1903-ip-resolution-bug-in-hyper-v-switch/) is activating windows feature "Windows Sandbox" and starting this app before hyper-v service, but this appears to only resolve it until the VM is restarted.

We could theoretically as a workaround, setup the "External" vnet ourself, and bridge it with a loopback interface(otherwise host looses network), to allow the "Deploy-EFlow" to continue setup, and afterwards change the VM to our own "Internal" vnet.

Issue with a custom internal vnet, is it requires DHCP, which was included with the default vnet, this adds quite some additional VM's to be managed once this scales up.
Instead i would prefer if in this case we could configure a static ip( as we know the network since we configured the internal vnet ourself) for the VM, does nodectl support this?

Side note: If we we're to experiment a bit more on our end, where can we find the docs on the nodectl.exe tool, primarily in regards to the yaml structures?

[ericvruder]

Just to itterate exactly what we had to do in order to get it to work:

Whenever the guest was incapable of getting an ip, we would do the following:

• shut down the guest
• shut down the hyper-v service
• close any instances of sandbox we had running
• turn on hyper-v
• start an instance of sandbox up
• start up the guest vm

Note: Sometimes, it would be enough to shut down the guest vm, start up an instance of Sandbox, and then start up the guest vm. I have only done this a few times and it hasn't failed yet, but I am hesitant to say that it always works.

Anyways, the previous dance would always lead to an ipv4 being assigned to the guest vm, but there is still an issue with dns name resolution. I am not sure what address is being added to /etc/resolv.conf (in my case,127.0.0.53), but it's not doing anything. We would then SSH into the guest and modify the dns server settings and assign it 8.8.8.8 or 1.1.1.1 - we now have net access with name resolution.

As soon as the guest is restarted, it looses its ip address and we have to start the dance all over again.

Another issue I see with creating our own internal network, besides the DHCP issue, is that the Deploy-Eflow tool does not support that (remember the original issue, can only use 'Default switch' as an internal switch). This would require us to be able to modify the yaml config files. As @Thacai mentioned, if we could provision it with a predefined network configuration and dns servers, that would be amazing.

[ericvruder]

This issue does seem to be associated with our current setup. This works if we run it on another computer, or if we create a vm and get it to run there. Closing this issue.

[PtwHub]

Hi, I landed here since we have a related issue. Maybe someone has a pointer for us: We have EFLOW running on an AzureVM and we can access our .NET-Core website (in an EdgeModule) using Https when logged into the Hyper-V VM via ssh. i.e. It works like a charm in the Hyper-V VM. But, when we open the firewall ports etc. on the Azure-VM -- the usual stuff -- we can't seem to access the https port on the Hyper-V VM from the web. We've searched for port forwarding help on that... Maybe we are missing something obvious to Azure-VM experts? Or is it an Azure-VM-EFLOW issue that we'll need to sit out? Thanks for any tips.

[fcabrera23]

Hi, @PtwHub could you please open a new issue with all the detailed information? Also, please add the answer to these questions?

1. If you RDP to the Azure VM, can you access using the browser to the website?
2. Are you trying to access the website using directly the Azure VM IP?

Thanks,
Francisco

[PtwHub]

Thanks. Will do. ....now looking at https://github.com/Azure-Solutions/azure-quickstart-templates/tree/6d3229350f6b1e4e75adf26746bd82431989699c/301-nested-vms-in-virtual-network

Answers to your Q:

1. Yes, at the 172.31.15.234:433 address works fine in the local browser.
2. We set up the usual dns hostname to the Azure Windows VM and would like to access the https site via that. ... looking for that forwarding/mapping.

[fcabrera23]

@PtwHub,

Thanks for answering my questions. From my understanding, if you can access the module using the local web browser, then you have some issues with the Azure VM traffic redirection. I would suggest creating an issue in the Azure repo, as they probably will be able to assist you better.

Thanks,
Francisco