Skip to content

Conversation

@adelina-t
Copy link
Contributor

In order to support VIPs for container Port Mappings, we should
use PortMappingPolicySetting type from HNSv2 instead of the old
NatPolicy from HNSv1.

Reason for Change:

Kubernetes e2e test: [sig-scheduling] SchedulerPredicates [Serial] validates that there is no conflict between pods with same hostPort but different hostIP and protocol [Conformance] fails on Windows Server 2004 ( as shown here: https://testgrid.k8s.io/sig-windows-containerd#aks-engine-azure-windows-master-2004-containerd-serial-slow ). The reason for this failure is that setting hostIP for port mappings are not supported with Azure-CNI. This PR uses allows setting of VIP by using the PortMappingPolicySetting type instead of NatPolicy.

Notes:

As suggested by @erfrimod , instead of converting From HNSv1 NatPolicy to HNSv2 PortMappingPolicySetting, we should directly unmarshal the policies recieved from the Runtime config into HNSv2 PortMappingPolicySetting. However, if we are to preserve compatibility with HNSv1, a conversion from one to the other is still needed. This is less than ideal and a unified way of approaching this should be introduced for all policy setting types in a subsequent PR.

Copy link
Contributor

@erfrimod erfrimod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@matmerr
Copy link
Member

matmerr commented Oct 14, 2020

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@codecov
Copy link

codecov bot commented Oct 14, 2020

Codecov Report

Merging #689 (c203eb1) into master (b41ae2a) will decrease coverage by 2.96%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #689      +/-   ##
==========================================
- Coverage   41.90%   38.94%   -2.97%     
==========================================
  Files          72       83      +11     
  Lines       10151    10697     +546     
==========================================
- Hits         4254     4166      -88     
- Misses       5426     6033     +607     
- Partials      471      498      +27     

Copy link
Member

@ashvindeodhar ashvindeodhar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adelina,
To confirm: You are converting the policy to hnsv1 to maintain the compatibility. You are saying a unified way is needed to do similar thing for all policy types in general in subsequent PRs?

Type: "NAT",
switch mapping.Protocol {
case "TCP":
protocol = 6
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have consts defined for these as protocolTCP, protocolUDP

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

if err := json.Unmarshal(policy.Data, &endpointPolicy); err != nil {
return false
}
if endpointPolicy.Type == "NAT" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think line 126 is an issue - you changed the Type from NAT to hnsv2.PortMapping in func getPoliciesFromRuntimeCfg.
Here you need to check if type is hnsv2.PortMapping and not NAT

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct! I seem to have missed this :/

jsonPolicies = append(jsonPolicies, serializedOutboundNatPolicy)
}
}
} else if isPolicyTypeOutBoundNAT := IsPolicyTypeNAT(policy); isPolicyTypeOutBoundNAT {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please don't reuse the bool , check is for NAT policy and variable is outboundNAT

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, sorry, this was a editor autocomplete mistake. Fixing :)

"github.com/Azure/azure-container-networking/cns"
"github.com/Azure/azure-container-networking/log"
"github.com/Azure/azure-container-networking/network"
"github.com/Azure/azure-container-networking/network/policy"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also share what all testing has been performed? Just containerd or docker has been tested to make HnsV1 is working as earlier too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please confirm?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specifically the portmapping scenario

@adelina-t
Copy link
Contributor Author

Adelina,
To confirm: You are converting the policy to hnsv1 to maintain the compatibility. You are saying a unified way is needed to do similar thing for all policy types in general in subsequent PRs?

Yes, that is the ideea. Of course, if there are other plans to phase out hnsv1 completely, that would of course simplify things. But with state right now, yes, I believe there should be subsequent PRs to address the other policy types.

@adelina-t adelina-t force-pushed the usePortMappingPolicySettings branch from 5cd9886 to 730e23d Compare October 16, 2020 15:02
@ashvindeodhar
Copy link
Member

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@ashvindeodhar
Copy link
Member

/lgtm @ninzavivek please approve if you are ok and merge once all the checks are completed. Thanks @adelina-t

Copy link
Contributor

@ninzavivek ninzavivek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adelina-t , please mention what all testing has been done - esp docker container with port mappings.

return nil, err
}
natPolicy := hcsshim.NatPolicy{
Type: "NAT",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please use constant here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

"github.com/Azure/azure-container-networking/cns"
"github.com/Azure/azure-container-networking/log"
"github.com/Azure/azure-container-networking/network"
"github.com/Azure/azure-container-networking/network/policy"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specifically the portmapping scenario

In order to support VIPs for container Port Mappings, we should
use PortMappingPolicySetting type from HNSv2 instead of the old
NatPolicy from HNSv1.
@ashvindeodhar
Copy link
Member

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants