Change append to insert for iptables rules

[eric]

Updates how iptables FORWARD rules are applied to accommodate an existing final DROP rule for the chain.

I am running CentOS 7.6 with firewalld enabled and found that all of the iptables entries that kube-router were adding were not taking effect due to where they were located.

This follows the convention of docker's libnetwork and uses -I instead of -A for adding rules to the FORWARD chain.

For reference, the output from iptables:

# iptables -t filter -L FORWARD -v
Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 FORWARD_direct  all  --  any    any     anywhere             anywhere            
    0     0 FORWARD_IN_ZONES_SOURCE  all  --  any    any     anywhere             anywhere            
    0     0 FORWARD_IN_ZONES  all  --  any    any     anywhere             anywhere            
    0     0 FORWARD_OUT_ZONES_SOURCE  all  --  any    any     anywhere             anywhere            
    0     0 FORWARD_OUT_ZONES  all  --  any    any     anywhere             anywhere            
    0     0 DROP       all  --  any    any     anywhere             anywhere             ctstate INVALID
    0     0 REJECT     all  --  any    any     anywhere             anywhere             reject-with icmp-host-prohibited
    0     0 ACCEPT     all  --  kube-bridge any     anywhere             anywhere             /* allow outbound traffic from pods */
    0     0 ACCEPT     all  --  any    kube-bridge  anywhere             anywhere             /* allow inbound traffic to pods */
    0     0 ACCEPT     all  --  any    eno1    anywhere             anywhere             /* allow outbound node port traffic on node interface with which node ip is associated */

[murali-reddy]

@eric Thanks for your PR. Which component is adding below rule?

 0     0 REJECT     all  --  any    any     anywhere             anywhere             reject-with icmp-host-prohibited

It overrides default policy Chain FORWARD (policy DROP 0 packets, 0 bytes) so wonder it is needed at all.

Reason it was Append instead of Insert was to give network policies rules to give precedence. So there is conflict here. Changing the order here https://github.com/cloudnativelabs/kube-router/blob/v0.2.3/pkg/cmd/kube-router.go#L130-L157 gives network policies that will be created takes precedence. But still the problem remains if we have rules already existing, then in which case rules are not added/inserted.

[eric]

It is added by firewalld on CENTOS by default.

[murali-reddy]

@eric Insert over Append is needed anyway. Change looks good. I will merge the PR.

Not sure how would restart firewalld behave, does it always append the DROP rule? Depending on which there may still be race condition.