Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flexible pipeline - #1 Architecture #3035

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions pkg/agent/openflow/egress.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
// Copyright 2021 Antrea Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package openflow

import (
"antrea.io/antrea/pkg/agent/openflow/cookie"
binding "antrea.io/antrea/pkg/ovs/openflow"
)

type featureEgress struct {
cookieAllocator cookie.Allocator

snatFlowCache *flowCategoryCache
hostNetworkingFlows []binding.Flow

enableProxy bool
}

func (c *featureEgress) getFeatureID() featureID {
return Egress
}

func newFeatureEgress(cookieAllocator cookie.Allocator, enableProxy bool) feature {
return &featureEgress{
snatFlowCache: newFlowCategoryCache(),
cookieAllocator: cookieAllocator,
enableProxy: enableProxy,
}
}
214 changes: 214 additions & 0 deletions pkg/agent/openflow/feature_templates.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,214 @@
// Copyright 2021 Antrea Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package openflow

import (
binding "antrea.io/antrea/pkg/ovs/openflow"
"antrea.io/antrea/pkg/util/runtime"
)

func (c *featurePodConnectivity) getTemplate(protocol ofProtocol) *pipelineTemplate {
var template *pipelineTemplate
if protocol == ofProtocolIP {
template = &pipelineTemplate{
stageTables: map[binding.StageID][]tableRequest{
binding.ClassifierStage: {
tableRequest{ClassifierTable, 0x7f},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

anywhere we can learn why here is setting as 0x7f? or will there be any update for existing ovs-pipeline design doc? https://github.com/antrea-io/antrea/blob/main/docs/design/ovs-pipeline.md

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. How the table priorities are decided? Should we define constants?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The order of table within a stage is decided by this priority. We don't need to define constants because priorities should be the same within a stage, otherwise the order of tables cannot be decided by priority. @jianjuns @luolanzone

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not understand your explanation. How 0x7f, 0x6f, 0x7d in this file are decided? If I add a new table later, how I decide the priority value?

},
binding.ValidationStage: {
tableRequest{SpoofGuardTable, 0x7f},
},
binding.ConntrackStateStage: {
tableRequest{ConntrackTable, 0x7f},
tableRequest{ConntrackStateTable, 0x6f},
},
binding.RoutingStage: {
tableRequest{L3ForwardingTable, 0x7f},
tableRequest{L3DecTTLTable, 0x7d},
},
binding.SwitchingStage: {
tableRequest{L2ForwardingCalcTable, 0x7f},
},
binding.ConntrackStage: {
tableRequest{ConntrackCommitTable, 0x7f},
},
binding.OutputStage: {
tableRequest{L2ForwardingOutTable, 0x7f},
},
},
}
for _, ipProtocol := range c.ipProtocols {
if ipProtocol == binding.ProtocolIPv6 {
template.stageTables[binding.ValidationStage] = append(template.stageTables[binding.ValidationStage], tableRequest{IPv6Table, 0x7e})
break
}
}
} else if protocol == ofProtocolARP {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think here you can use switch to make the whole structure more clear.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, updated in #3058

template = &pipelineTemplate{
stageTables: map[binding.StageID][]tableRequest{
binding.ValidationStage: {
tableRequest{ARPSpoofGuardTable, 0x7f},
},
binding.OutputStage: {
tableRequest{ARPResponderTable, 0x7f},
},
},
}
}

return template
}

func (c *featureNetworkPolicy) getTemplate(protocol ofProtocol) *pipelineTemplate {
var template *pipelineTemplate
if protocol != ofProtocolIP {
return template
}
template = &pipelineTemplate{
stageTables: map[binding.StageID][]tableRequest{
binding.ValidationStage: {
tableRequest{SpoofGuardTable, 0x7f},
},
binding.ConntrackStateStage: {
tableRequest{SNATConntrackTable, 0xff},
},
binding.EgressSecurityStage: {
tableRequest{EgressRuleTable, 0x7f},
tableRequest{EgressDefaultTable, 0x7e},
tableRequest{EgressMetricTable, 0x7d},
},
binding.RoutingStage: {
tableRequest{L3ForwardingTable, 0x7f},
},
binding.IngressSecurityStage: {
tableRequest{IngressRuleTable, 0x7f},
tableRequest{IngressDefaultTable, 0x7e},
tableRequest{IngressMetricTable, 0x7d},
},
binding.ConntrackStage: {
tableRequest{ConntrackCommitTable, 0x7f},
},
},
}
if c.enableAntreaPolicy {
template.stageTables[binding.EgressSecurityStage] = append(template.stageTables[binding.EgressSecurityStage],
tableRequest{AntreaPolicyEgressRuleTable, 0x8f},
)
template.stageTables[binding.IngressSecurityStage] = append(template.stageTables[binding.IngressSecurityStage],
tableRequest{AntreaPolicyIngressRuleTable, 0x8f},
)
}
return template
}

func (c *featureService) getTemplate(protocol ofProtocol) *pipelineTemplate {
var template *pipelineTemplate
if protocol != ofProtocolIP {
return template
}
if c.enableProxy {
template = &pipelineTemplate{
stageTables: map[binding.StageID][]tableRequest{
binding.ConntrackStateStage: {
tableRequest{SNATConntrackTable, 0xff},
},
binding.PreRoutingStage: {
tableRequest{SessionAffinityTable, 0x7f},
tableRequest{ServiceLBTable, 0x7e},
tableRequest{EndpointDNATTable, 0x7d},
},
binding.RoutingStage: {
tableRequest{L3ForwardingTable, 0x7f},
tableRequest{ServiceHairpinMarkTable, 0x7e},
},
binding.PostRoutingStage: {
tableRequest{SNATConntrackCommitTable, 0x7e},
},
binding.ConntrackStage: {
tableRequest{ConntrackCommitTable, 0x7f},
},
binding.OutputStage: {
tableRequest{L2ForwardingOutTable, 0x7f},
},
},
}
if runtime.IsWindowsPlatform() {
template.stageTables[binding.ValidationStage] = append(template.stageTables[binding.ValidationStage], tableRequest{UplinkTable, 0x8f})
}
if c.proxyAll {
template.stageTables[binding.PreRoutingStage] = append(template.stageTables[binding.PreRoutingStage], tableRequest{NodePortProbeTable, 0x8f})
}
} else {
template = &pipelineTemplate{
stageTables: map[binding.StageID][]tableRequest{
binding.PreRoutingStage: {
tableRequest{DNATTable, 0x7f},
},
},
}
}
return template
}

func (c *featureEgress) getTemplate(protocol ofProtocol) *pipelineTemplate {
var template *pipelineTemplate
if protocol != ofProtocolIP {
return template
}
template = &pipelineTemplate{
stageTables: map[binding.StageID][]tableRequest{
binding.RoutingStage: {
tableRequest{L3ForwardingTable, 0x7f},
},
binding.PostRoutingStage: {
tableRequest{SNATTable, 0x8f},
},
},
}
return template
}

func (c *featureTraceflow) getTemplate(protocol ofProtocol) *pipelineTemplate {
var template *pipelineTemplate
if protocol != ofProtocolIP {
return template
}
template = &pipelineTemplate{}
return template
}

func (c *featureVMConnectivity) getTemplate(protocol ofProtocol) *pipelineTemplate {
var template *pipelineTemplate
if protocol != ofProtocolIP {
return template
}
template = &pipelineTemplate{
stageTables: map[binding.StageID][]tableRequest{
binding.ClassifierStage: {
tableRequest{ClassifierTable, 0x7f},
},
binding.ConntrackStateStage: {
tableRequest{ConntrackStateTable, 0x7f},
},
binding.ConntrackStage: {
tableRequest{ConntrackTable, 0x7f},
},
binding.OutputStage: {
tableRequest{L2ForwardingOutTable, 0x7f},
},
},
}
return template
}
69 changes: 50 additions & 19 deletions pkg/agent/openflow/fields.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,32 +34,49 @@ var (
FromLocalRegMark = binding.NewRegMark(PktSourceField, 2)
FromUplinkRegMark = binding.NewRegMark(PktSourceField, 4)
FromBridgeRegMark = binding.NewRegMark(PktSourceField, 5)
// reg0[16]: Mark to indicate the ofPort number of an interface is found.
OFPortFoundRegMark = binding.NewOneBitRegMark(0, 16, "OFPortFound")
// reg0[18]: Mark to indicate the packet needs DNAT to virtual IP.
// reg0[4..7]: Field to mark the packet destination. Marks in this field include,
// - 0: to local Pod
// - 1: to remote Node
// - 2: to external
PktDestinationField = binding.NewRegField(0, 4, 7, "PacketDestination")
ToTunnelRegMark = binding.NewRegMark(PktDestinationField, 0)
ToGatewayRegMark = binding.NewRegMark(PktDestinationField, 1)
ToLocalRegMark = binding.NewRegMark(PktDestinationField, 2)
ToUplinkRegMark = binding.NewRegMark(PktDestinationField, 4)
PacketUnionField = binding.NewRegField(0, 0, 7, "PacketUnion")
GatewayHairpinRegMark = binding.NewRegMark(PacketUnionField, (1<<4)|1)
// reg0[8]: Mark to indicate the ofPort number of an interface is found.
OFPortFoundRegMark = binding.NewOneBitRegMark(0, 8, "OFPortFound")
// reg0[9]: Mark to indicate the packet needs DNAT to virtual IP.
// If a packet uses HairpinRegMark, it will be output to the port where it enters OVS pipeline in L2ForwardingOutTable.
HairpinRegMark = binding.NewOneBitRegMark(0, 18, "Hairpin")
// reg0[19]: Mark to indicate the packet's MAC address needs to be rewritten.
RewriteMACRegMark = binding.NewOneBitRegMark(0, 19, "RewriteMAC")
// reg0[20]: Mark to indicate the packet is denied(Drop/Reject).
CnpDenyRegMark = binding.NewOneBitRegMark(0, 20, "CNPDeny")
// reg0[21..22]: Field to indicate disposition of Antrea Policy. It could have more bits to support more disposition
HairpinRegMark = binding.NewOneBitRegMark(0, 9, "Hairpin")
// reg0[10]: Field to indicate that which IP should be used for hairpin connections.
SNATWithGatewayIP = binding.NewOneBitRegMark(0, 10, "SNATWithGatewayIP")
SNATWithVirtualIP = binding.NewOneBitZeroRegMark(0, 10, "SNATWithVirtualIP")
HairpinSNATUnionField = binding.NewRegField(0, 9, 10, "HairpinSNATUnion")
HairpinSNATWithVirtualIP = binding.NewRegMark(HairpinSNATUnionField, 1)
HairpinSNATWithGatewayIP = binding.NewRegMark(HairpinSNATUnionField, 3)
// reg0[11]: Mark to indicate the packet's MAC address needs to be rewritten.
RewriteMACRegMark = binding.NewOneBitRegMark(0, 11, "RewriteMAC")
// reg0[12]: Mark to indicate the packet is denied(Drop/Reject).
CnpDenyRegMark = binding.NewOneBitRegMark(0, 12, "CNPDeny")
// reg0[13..14]: Field to indicate disposition of Antrea Policy. It could have more bits to support more disposition
// that Antrea policy support in the future.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/Antrea policy/Antrea Policy/

// Marks in this field include,
// - 0b00: allow
// - 0b01: drop
// - 0b10: reject
APDispositionField = binding.NewRegField(0, 21, 22, "APDisposition")
APDispositionField = binding.NewRegField(0, 13, 14, "APDisposition")
DispositionAllowRegMark = binding.NewRegMark(APDispositionField, DispositionAllow)
DispositionDropRegMark = binding.NewRegMark(APDispositionField, DispositionDrop)
DispositionRejRegMark = binding.NewRegMark(APDispositionField, DispositionRej)
// reg0[24..27]: Field to indicate the reasons of sending packet to the controller.
// reg0[15..18]: Field to indicate the reasons of sending packet to the controller.
// Marks in this field include,
// - 0b0001: logging
// - 0b0010: reject
// - 0b0100: deny (used by Flow Exporter)
// - 0b1000: DNS packet (used by FQDN)
CustomReasonField = binding.NewRegField(0, 24, 27, "PacketInReason")
CustomReasonField = binding.NewRegField(0, 15, 18, "PacketInReason")
CustomReasonLoggingRegMark = binding.NewRegMark(CustomReasonField, CustomReasonLogging)
CustomReasonRejectRegMark = binding.NewRegMark(CustomReasonField, CustomReasonReject)
CustomReasonDenyRegMark = binding.NewRegMark(CustomReasonField, CustomReasonDeny)
Expand All @@ -68,8 +85,6 @@ var (
// reg1(NXM_NX_REG1)
// Field to cache the ofPort of the OVS interface where to output packet.
TargetOFPortField = binding.NewRegField(1, 0, 31, "TargetOFPort")
// ToGatewayRegMark marks that the output interface is Antrea gateway.
ToGatewayRegMark = binding.NewRegMark(TargetOFPortField, config.HostGatewayOFPort)
// ToBridgeRegMark marks that the output interface is OVS bridge.
ToBridgeRegMark = binding.NewRegMark(TargetOFPortField, config.BridgeOFPort)

Expand Down Expand Up @@ -103,16 +118,20 @@ var (
NodePortAddressField = binding.NewRegField(4, 19, 19, "NodePortAddress")
// ToNodePortAddressRegMark marks that the Service type as NodePort.
ToNodePortAddressRegMark = binding.NewRegMark(NodePortAddressField, 0b1)
// reg4[20]: Field to mark that whether the packet of Service NodePort/LoadBalancer from gateway requires SNAT.
ServiceSNATField = binding.NewRegField(4, 20, 20, "ServiceSNAT")
// ServiceNeedSNATRegMark marks that the packet of Service NodePort/LoadBalancer requires SNAT.
ServiceNeedSNATRegMark = binding.NewRegMark(ServiceSNATField, 0b1)
// reg4[16..19]: Field to store the union value of Endpoint state and the mark of whether Service type is NodePort.
NodePortUnionField = binding.NewRegField(4, 16, 19, "NodePortUnion")
// reg4[21]: Mark to indicate the packet is from local AntreaFlexibleIPAM Pod.
// NotAntreaFlexibleIPAMRegMark will be used with RewriteMACRegMark, thus the reg id must not be same due to the limitation of ofnet library.
AntreaFlexibleIPAMRegMark = binding.NewOneBitRegMark(4, 21, "AntreaFlexibleIPAM")
NotAntreaFlexibleIPAMRegMark = binding.NewOneBitZeroRegMark(4, 21, "AntreaFlexibleIPAM")
// reg4[22..23]: Field to store the state of a connection of Service NodePort/LoadBalancer from gateway which
// requires SNAT or not.
// - 0b01: connection requires SNAT and is not marked with a ct mark.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure which one is correct but I suppose all ct mark string should be the same, either CT mark or ct mark in comments.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe CT mark.

// - 0b11: connection requires SNAT and is marked with a ct mark.
ServiceSNATStateField = binding.NewRegField(4, 22, 23, "ServiceSNAT")
NotRequireSNATRegMark = binding.NewRegMark(ServiceSNATStateField, 0b00)
RequireSNATRegMark = binding.NewRegMark(ServiceSNATStateField, 0b01)
CTMarkedSNATRegMark = binding.NewRegMark(ServiceSNATStateField, 0b11)

// reg5(NXM_NX_REG5)
// Field to cache the Egress conjunction ID hit by TraceFlow packet.
Expand Down Expand Up @@ -141,12 +160,24 @@ var (

// Mark to indicate the connection is initiated through the host gateway interface
// (i.e. for which the first packet of the connection was received through the gateway).
// This CT mark is only used in CtZone / CtZoneV6.
FromGatewayCTMark = binding.NewCTMark(0b1, 1, 1)
// Mark to indicate DNAT is performed on the connection for Service.
// This CT mark is both used in CtZone / CtZoneV6 and SNATCtZone / SNATCtZoneV6.
ServiceCTMark = binding.NewCTMark(0b1, 2, 2)
// Mark to indicate the connection is initiated through the host bridge interface
// (i.e. for which the first packet of the connection was received through the bridge).
FromBridgeCTMark = binding.NewCTMark(0x1, 3, 3)
// This CT mark is only used in CtZone / CtZoneV6.
FromBridgeCTMark = binding.NewCTMark(0xb1, 3, 3)
// Mark to indicate SNAT should be performed on the connection for Service.
// This CT mark is only used in CtZone / CtZoneV6.
ServiceSNATCTMark = binding.NewCTMark(0b1, 4, 4)
// Mark to indicate the connection is hairpin.
// This CT mark is only used in SNATCtZone / SNATCtZoneV6.
HairpinCTMark = binding.NewCTMark(0b1, 5, 5)
// Mark to indicate the connection is hairpin and Service.
// This CT mark is only used in SNATCtZone / SNATCtZoneV6.
UnionHairpinServiceCTMark = binding.NewCTMark(0b11, 4, 5)
)

// Fields using CT label.
Expand Down
Loading