T5729: firewall: multiple backports

[nicolas-fort]

Change Summary

Multiple backports that were missing in sagitta

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Code style update (formatting, renaming)
• Refactoring (no functional changes)
• Migration from an old Vyatta component to vyos-1x, please link to related PR inside obsoleted component
• Other (please describe):
  Backports

Related Task(s)

• https://vyos.dev/T5729
• https://vyos.dev/T5616
• https://vyos.dev/T5590

Related PR(s)

Original PR for 1.5:

• T5729: firewall: switch to valueless in  #2471
• T5590: firewall log rule: fix order which rule are processed #2283
• T5616: firewall and policy: add option to be able to match firewall marks #2314

Component(s) name

firewall
policy-route
nat
nat66

Proposed changes

Multiple backports that were missing in saggita:

• https://vyos.dev/T5729 --> Move to valules 'log' and 'state'
• https://vyos.dev/T5616 --> Add firewall marks matcher in fwall and policy-route
• https://vyos.dev/T5590 --> Fix log parsing in ruleset

How to test

vyos@Sagitta-Valueless# set firewall ipv4 name FOO rule 10 action drop 
[edit]
vyos@Sagitta-Valueless# set firewall ipv4 name FOO rule 10 log-options group 4
[edit]
vyos@Sagitta-Valueless# commit

log-options defined, but log is not enable

[[firewall]] failed
Commit failed
[edit]
vyos@Sagitta-Valueless# set firewall ipv4 name FOO rule 10 log
[edit]
vyos@Sagitta-Valueless# commit
....
....
vyos@Sagitta-Valueless# run show config comm | grep "policy\|firewall"
set firewall ipv4 name FOO rule 10 action 'drop'
set firewall ipv4 name FOO rule 10 log
set firewall ipv4 name FOO rule 10 log-options group '4'
set firewall ipv4 name FOO rule 10 protocol 'icmp'
set firewall ipv4 name FOO rule 10 source address '198.51.100.55'
set firewall ipv4 name FOO rule 20 action 'drop'
set firewall ipv4 name FOO rule 20 mark '22334'
set firewall ipv6 name BAR rule 10 action 'reject'
set firewall ipv6 name BAR rule 10 state 'invalid'
set firewall ipv6 name BAR rule 20 action 'accept'
set firewall ipv6 name BAR rule 20 log
set firewall ipv6 name BAR rule 20 state 'related'
set firewall ipv6 name BAR rule 20 state 'new'
set firewall ipv6 name BAR rule 20 state 'established'
set policy route PBR_4 rule 10 action 'accept'
set policy route PBR_4 rule 10 log
set policy route PBR_4 rule 10 mark '9988'
set policy route PBR_4 rule 20 set mark '7777'
set policy route PBR_4 rule 20 state 'new'
set policy route PBR_4 rule 20 state 'related'
[edit]
vyos@Sagitta-Valueless# 

And generated chains:

vyos@Sagitta-Valueless# sudo nft -s list chain ip vyos_filter NAME_FOO
table ip vyos_filter {
        chain NAME_FOO {
                meta l4proto icmp ip saddr 198.51.100.55 log prefix "[ipv4-NAM-FOO-10-D]" log group 4 counter drop comment "NAM-FOO-10"
                meta mark 0x0000573e counter drop comment "NAM-FOO-20"
                counter drop comment "FOO default-action drop"
        }
}
[edit]
vyos@Sagitta-Valueless# sudo nft -s list chain ip6 vyos_filter NAME6_BAR
table ip6 vyos_filter {
        chain NAME6_BAR {
                ct state invalid counter reject comment "NAM-BAR-10"
                ct state { established, related, new } log prefix "[ipv6-NAM-BAR-20-A]" counter accept comment "NAM-BAR-20"
                counter drop comment "BAR default-action drop"
        }
}
[edit]
vyos@Sagitta-Valueless# 

vyos@Sagitta-Valueless# sudo nft -s list chain ip vyos_mangle VYOS_PBR_UD_PBR_4
table ip vyos_mangle {
        chain VYOS_PBR_UD_PBR_4 {
                meta mark 0x00002704 log prefix "[ipv4-route-PBR_4-10-A]" counter accept comment "route-PBR_4-10"
                ct state { related, new } counter meta mark set 0x00001e61 return comment "route-PBR_4-20"
        }
}
[edit]

Smoketest result

### SMOkETESTS:
root@Sagitta-Valueless:/usr/libexec/vyos/tests/smoke/cli# ./test_firewall.py 
test_geoip (__main__.TestFirewall.test_geoip) ... Updating GeoIP. Please wait...
Updating GeoIP. Please wait...
ok
test_groups (__main__.TestFirewall.test_groups) ... ok
test_ipv4_advanced (__main__.TestFirewall.test_ipv4_advanced) ... ok
test_ipv4_basic_rules (__main__.TestFirewall.test_ipv4_basic_rules) ... ok
test_ipv4_mask (__main__.TestFirewall.test_ipv4_mask) ... ok
test_ipv4_state_and_status_rules (__main__.TestFirewall.test_ipv4_state_and_status_rules) ... ok
test_ipv6_advanced (__main__.TestFirewall.test_ipv6_advanced) ... ok
test_ipv6_basic_rules (__main__.TestFirewall.test_ipv6_basic_rules) ... ok
test_ipv6_mask (__main__.TestFirewall.test_ipv6_mask) ... ok
test_nested_groups (__main__.TestFirewall.test_nested_groups) ... 
Group "smoketest_network1" has a circular reference

ok
test_source_validation (__main__.TestFirewall.test_source_validation) ... ok
test_sysfs (__main__.TestFirewall.test_sysfs) ... ok
test_zone_basic (__main__.TestFirewall.test_zone_basic) ... ok

----------------------------------------------------------------------
Ran 13 tests in 29.726s

OK
root@Sagitta-Valueless:/usr/libexec/vyos/tests/smoke/cli# ./test_policy_route.py 
test_pbr_group (__main__.TestPolicyRoute.test_pbr_group) ... ok
test_pbr_mark (__main__.TestPolicyRoute.test_pbr_mark) ... ok
test_pbr_mark_connection (__main__.TestPolicyRoute.test_pbr_mark_connection) ... ok
test_pbr_matching_criteria (__main__.TestPolicyRoute.test_pbr_matching_criteria) ... ok
test_pbr_table (__main__.TestPolicyRoute.test_pbr_table) ... ok

----------------------------------------------------------------------
Ran 5 tests in 15.148s

OK
root@Sagitta-Valueless:/usr/libexec/vyos/tests/smoke/cli# 
root@Sagitta-Valueless:/usr/libexec/vyos/tests/smoke/cli# ./test_nat.py 
test_dnat (__main__.TestNAT.test_dnat) ... ok
test_dnat_negated_addresses (__main__.TestNAT.test_dnat_negated_addresses) ... ok
test_dnat_redirect (__main__.TestNAT.test_dnat_redirect) ... ok
test_dnat_without_translation_address (__main__.TestNAT.test_dnat_without_translation_address) ... ok
test_nat_balance (__main__.TestNAT.test_nat_balance) ... ok
test_nat_no_rules (__main__.TestNAT.test_nat_no_rules) ... ok
test_snat (__main__.TestNAT.test_snat) ... ok
test_snat_groups (__main__.TestNAT.test_snat_groups) ... ok
test_snat_required_translation_address (__main__.TestNAT.test_snat_required_translation_address) ... 
Source NAT configuration error in rule 5: translation requires address
and/or port

ok
test_static_nat (__main__.TestNAT.test_static_nat) ... ok

----------------------------------------------------------------------
Ran 10 tests in 17.796s

OK
root@Sagitta-Valueless:/usr/libexec/vyos/tests/smoke/cli# 
root@Sagitta-Valueless:/usr/libexec/vyos/tests/smoke/cli# 
root@Sagitta-Valueless:/usr/libexec/vyos/tests/smoke/cli# ./test_nat66.py 
test_destination_nat66 (__main__.TestNAT66.test_destination_nat66) ... ok
test_destination_nat66_prefix (__main__.TestNAT66.test_destination_nat66_prefix) ... ok
test_destination_nat66_protocol (__main__.TestNAT66.test_destination_nat66_protocol) ... ok
test_destination_nat66_without_translation_address (__main__.TestNAT66.test_destination_nat66_without_translation_address) ... ok
test_nat66_no_rules (__main__.TestNAT66.test_nat66_no_rules) ... ok
test_source_nat66 (__main__.TestNAT66.test_source_nat66) ... ok
test_source_nat66_address (__main__.TestNAT66.test_source_nat66_address) ... ok
test_source_nat66_protocol (__main__.TestNAT66.test_source_nat66_protocol) ... ok
test_source_nat66_required_translation_prefix (__main__.TestNAT66.test_source_nat66_required_translation_prefix) ... 
Source NAT66 configuration error in rule 5: translation address not
specified


Source NAT66 configuration error in rule 5: translation address not
specified

ok

----------------------------------------------------------------------
Ran 9 tests in 14.977s

OK
root@Sagitta-Valueless:/usr/libexec/vyos/tests/smoke/cli# 

Checklist:

• I have read the CONTRIBUTING document
• I have linked this PR to one or more Phabricator Task(s)
• I have run the components SMOKETESTS if applicable
• My commit headlines contain a valid Task id
• My change requires a change to the documentation
• I have updated the documentation accordingly

[nicolas-fort]

There's an error in migration script. Do not merge yet, I'll submit a patch

[sever-sever]

There's an error in migration script. Do not merge yet, I'll submit a patch

Convert it to draft

[nicolas-fort]

PR Ready