t3c: mustache template DS parameter for remap.config lines

[traeak]

This allows remap.config line to be reordered, modified, or plain hard coded set by specifying a DS parameter with a custom mustache template. (github.com/cbroglie/mustache)

The DS parameter is Config File: 'remap.config' and the Name one of:

• line_template.first
• line_template.inner
• line_template.last

The corresponding default values are as follows:

const DefaultFirstRemapConfigTemplateString = map {{{Source}}} {{{Destination}}} {{{Strategy}}} {{{Dscp}}} {{{HeaderRewrite}}} {{{DropQstring}}} {{{Signing}}} {{{RegexRemap}}} {{{Cachekey}}} {{{RangeRequests}}} {{{Pacing}}} {{{RawText}}}
const DefaultLastRemapConfigTemplateString = map {{{Source}}} {{{Destination}}} {{{Strategy}}} {{{HeaderRewrite}}} {{{Cachekey}}} {{{RangeRequests}}} {{{RawText}}}
const DefaultInnerRemapConfigTemplateString = DefaultLastRemapConfigTemplateString

Note this field may be used/abused to hard set the entire remap.config line for the given delivery service.

---

Which Traffic Control components are affected by this PR?

• Documentation
• Traffic Control Cache Config (t3c, formerly ORT)
• Traffic Portal

What is the best way to verify this PR?

If this is a bugfix, which Traffic Control versions contained the bug?

PR submission checklist

• This PR has tests
• This PR has documentation
• This PR has a CHANGELOG.md entry
• This PR DOES NOT FIX A SERIOUS SECURITY VULNERABILITY (see the Apache Software Foundation's security guidelines for details)

[ocket8888]

Why did you decide on this library? The one linked to on the mustache website is cbroclie/mustache.

[traeak]

Why did you decide on this library? The one linked to on the mustache website is cbroclie/mustache.

It was one I had already used for some internal work. I'll look at swapping out for this one. This really is just playing around, I made it draft for others to look at.

I changed over to using the suggested mustache lib.

[codecov]

Codecov Report

Merging #7155 (700a0ed) into master (c4444af) will increase coverage by 0.08%.
The diff coverage is 84.00%.

❗ Current head 700a0ed differs from pull request most recent head 0c75f67. Consider uploading reports for the commit 0c75f67 to get more accurate results

@@             Coverage Diff              @@
##             master    #7155      +/-   ##
============================================
+ Coverage     28.23%   28.31%   +0.08%     
  Complexity       98       98              
============================================
  Files           617      617              
  Lines         69576    69671      +95     
  Branches         90       90              
============================================
+ Hits          19642    19726      +84     
- Misses        48120    48125       +5     
- Partials       1814     1820       +6     

Flag	Coverage Δ
golib_unit	53.18% <84.00%> (+0.30%)	⬆️
grove_unit	4.60% <ø> (ø)
t3c_generate_unit	24.96% <ø> (ø)
traffic_monitor_unit	20.43% <ø> (ø)
traffic_ops_integration	69.29% <ø> (ø)
traffic_ops_unit	19.67% <ø> (ø)
traffic_stats_unit	10.41% <ø> (ø)
v3	57.68% <ø> (ø)
v4	79.09% <ø> (ø)
v5	78.53% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
lib/go-atscfg/remapdotconfig.go	76.80% <84.00%> (+0.72%)	⬆️
lib/go-atscfg/headerrewritedotconfig.go	57.14% <0.00%> (+1.61%)	⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[zrhoffman]

Looks close! Is fixing the failing integration test possible?

--- FAIL: TestT3cGit (5.63s)
--- FAIL: TestT3cBadassAndSyncDs (11.31s)
--- FAIL: TestT3cBadassAndSyncDs/check_diff_of_remap.config_between_baseline_and_badass-generated (0.00s)
    t3c_mode_test.go:84: /opt/trafficserver/etc/trafficserver/remap.config and baseline-configs/remap.config differ:

diff --git a/baseline-configs/remap.config b/opt/trafficserver/etc/trafficserver/remap.config
index 6cee4fa..c0e5480 100644
--- a/baseline-configs/remap.config
+++ b/opt/trafficserver/etc/trafficserver/remap.config
@@ -1,5 +1,6 @@
-# do not edit - generated for atlanta-edge-03 by  () on fri nov 13 18:49:10 utc 2020
-map http://atlanta-edge-03.ds-origin-ip-0.test.cdn1.net/     http://192.0.2.1/ @plugin=header_rewrite.so @pparam=dscp/set_dscp_40.config @plugin=header_rewrite.so @pparam=hdr_rw_first_ds-origin-ip-0.config @plugin=cachekey.so @pparam=--separator= @pparam=--remove-all-params=true @pparam=--remove-path=true @pparam=--capture-prefix-uri=/^([^?]*)/$1/ # ds 'ds-origin-ip-0' topology 'mso-topology'
-map https://atlanta-edge-03.ds-origin-ip-0.test.cdn1.net/     http://192.0.2.1/ @plugin=header_rewrite.so @pparam=dscp/set_dscp_40.config @plugin=header_rewrite.so @pparam=hdr_rw_first_ds-origin-ip-0.config @plugin=cachekey.so @pparam=--separator= @pparam=--remove-all-params=true @pparam=--remove-path=true @pparam=--capture-prefix-uri=/^([^?]*)/$1/ # ds 'ds-origin-ip-0' topology 'mso-topology'
-map http://atlanta-edge-03.ds-top.test.cdn1.net/     http://origin.topology.example.net/ @plugin=header_rewrite.so @pparam=dscp/set_dscp_40.config @plugin=header_rewrite.so @pparam=hdr_rw_first_ds-top.config @plugin=cachekey.so @pparam=--separator= @pparam=--remove-all-params=true @pparam=--remove-path=true @pparam=--capture-prefix-uri=/^([^?]*)/$1/ # ds 'ds-top' topology 'mso-topology'
-map https://atlanta-edge-03.ds-top.test.cdn1.net/     http://origin.topology.example.net/ @plugin=header_rewrite.so @pparam=dscp/set_dscp_40.config @plugin=header_rewrite.so @pparam=hdr_rw_first_ds-top.config @plugin=cachekey.so @pparam=--separator= @pparam=--remove-all-params=true @pparam=--remove-path=true @pparam=--capture-prefix-uri=/^([^?]*)/$1/ # ds 'ds-top' topology 'mso-topology'
+# do not edit - generated for atlanta-edge-03 by t3c-generate 7.1.0..8a9a3388 from https://to_server:443 ips (172.18.0.3:443) on 2023-01-05t16:32:14.150696352z
+
+map http://atlanta-edge-03.ds-origin-ip-0.test.cdn1.net/ http://192.0.2.1/ @plugin=header_rewrite.so @pparam=dscp/set_dscp_40.config @plugin=header_rewrite.so @pparam=hdr_rw_first_ds-origin-ip-0.config @plugin=cachekey.so @pparam=--separator= @pparam=--remove-all-params=true @pparam=--remove-path=true @pparam=--capture-prefix-uri=/^([^?]*)/$1/    # ds 'ds-origin-ip-0' topology 'mso-topology'
+map https://atlanta-edge-03.ds-origin-ip-0.test.cdn1.net/ http://192.0.2.1/ @plugin=header_rewrite.so @pparam=dscp/set_dscp_40.config @plugin=header_rewrite.so @pparam=hdr_rw_first_ds-origin-ip-0.config @plugin=cachekey.so @pparam=--separator= @pparam=--remove-all-params=true @pparam=--remove-path=true @pparam=--capture-prefix-uri=/^([^?]*)/$1/    # ds 'ds-origin-ip-0' topology 'mso-topology'
+map http://atlanta-edge-03.ds-top.test.cdn1.net/ http://origin.topology.example.net/ @plugin=header_rewrite.so @pparam=dscp/set_dscp_40.config @plugin=header_rewrite.so @pparam=hdr_rw_first_ds-top.config @plugin=cachekey.so @pparam=--separator= @pparam=--remove-all-params=true @pparam=--remove-path=true @pparam=--capture-prefix-uri=/^([^?]*)/$1/    # ds 'ds-top' topology 'mso-topology'
+map https://atlanta-edge-03.ds-top.test.cdn1.net/ http://origin.topology.example.net/ @plugin=header_rewrite.so @pparam=dscp/set_dscp_40.config @plugin=header_rewrite.so @pparam=hdr_rw_first_ds-top.config @plugin=cachekey.so @pparam=--separator= @pparam=--remove-all-params=true @pparam=--remove-path=true @pparam=--capture-prefix-uri=/^([^?]*)/$1/    # ds 'ds-top' topology 'mso-topology'

[zrhoffman]

Very close, there's just

• 2 spaces between @pparam=hdr_rw_first_ds-origin-ip-0.config and @plugin=cachekey.so in the generated remap.config
• 2 spaces between @pparam=--capture-prefix-uri=/^([^?]*)/$1/ and # in the generated remap.config

If you want to just add those spaces to the reference, the test would pass and we could call it good

--- FAIL: TestT3cBadassAndSyncDs/check_diff_of_remap.config_between_baseline_and_badass-generated (0.00s)
    t3c_mode_test.go:84: /opt/trafficserver/etc/trafficserver/remap.config and baseline-configs/remap.config differ:

diff --git a/baseline-configs/remap.config b/opt/trafficserver/etc/trafficserver/remap.config
index 0352abf..6df6491 100644
--- a/baseline-configs/remap.config
+++ b/opt/trafficserver/etc/trafficserver/remap.config
@@ -1,5 +1,6 @@
-# do not edit - generated for atlanta-edge-03 by  () on fri nov 13 18:49:10 utc 2020
-map http://atlanta-edge-03.ds-origin-ip-0.test.cdn1.net/ http://192.0.2.1/ @plugin=header_rewrite.so @pparam=dscp/set_dscp_40.config @plugin=header_rewrite.so @pparam=hdr_rw_first_ds-origin-ip-0.config @plugin=cachekey.so @pparam=--separator= @pparam=--remove-all-params=true @pparam=--remove-path=true @pparam=--capture-prefix-uri=/^([^?]*)/$1/ # ds 'ds-origin-ip-0' topology 'mso-topology'
-map https://atlanta-edge-03.ds-origin-ip-0.test.cdn1.net/ http://192.0.2.1/ @plugin=header_rewrite.so @pparam=dscp/set_dscp_40.config @plugin=header_rewrite.so @pparam=hdr_rw_first_ds-origin-ip-0.config @plugin=cachekey.so @pparam=--separator= @pparam=--remove-all-params=true @pparam=--remove-path=true @pparam=--capture-prefix-uri=/^([^?]*)/$1/ # ds 'ds-origin-ip-0' topology 'mso-topology'
-map http://atlanta-edge-03.ds-top.test.cdn1.net/ http://origin.topology.example.net/ @plugin=header_rewrite.so @pparam=dscp/set_dscp_40.config @plugin=header_rewrite.so @pparam=hdr_rw_first_ds-top.config @plugin=cachekey.so @pparam=--separator= @pparam=--remove-all-params=true @pparam=--remove-path=true @pparam=--capture-prefix-uri=/^([^?]*)/$1/ # ds 'ds-top' topology 'mso-topology'
-map https://atlanta-edge-03.ds-top.test.cdn1.net/ http://origin.topology.example.net/ @plugin=header_rewrite.so @pparam=dscp/set_dscp_40.config @plugin=header_rewrite.so @pparam=hdr_rw_first_ds-top.config @plugin=cachekey.so @pparam=--separator= @pparam=--remove-all-params=true @pparam=--remove-path=true @pparam=--capture-prefix-uri=/^([^?]*)/$1/ # ds 'ds-top' topology 'mso-topology'
+# do not edit - generated for atlanta-edge-03 by t3c-generate 7.1.0..c3af4984 from https://to_server:443 ips (172.18.0.3:443) on 2023-01-05t21:01:49.85368428z
+
+map http://atlanta-edge-03.ds-origin-ip-0.test.cdn1.net/ http://192.0.2.1/ @plugin=header_rewrite.so @pparam=dscp/set_dscp_40.config @plugin=header_rewrite.so @pparam=hdr_rw_first_ds-origin-ip-0.config  @plugin=cachekey.so @pparam=--separator= @pparam=--remove-all-params=true @pparam=--remove-path=true @pparam=--capture-prefix-uri=/^([^?]*)/$1/  # ds 'ds-origin-ip-0' topology 'mso-topology'
+map https://atlanta-edge-03.ds-origin-ip-0.test.cdn1.net/ http://192.0.2.1/ @plugin=header_rewrite.so @pparam=dscp/set_dscp_40.config @plugin=header_rewrite.so @pparam=hdr_rw_first_ds-origin-ip-0.config  @plugin=cachekey.so @pparam=--separator= @pparam=--remove-all-params=true @pparam=--remove-path=true @pparam=--capture-prefix-uri=/^([^?]*)/$1/  # ds 'ds-origin-ip-0' topology 'mso-topology'
+map http://atlanta-edge-03.ds-top.test.cdn1.net/ http://origin.topology.example.net/ @plugin=header_rewrite.so @pparam=dscp/set_dscp_40.config @plugin=header_rewrite.so @pparam=hdr_rw_first_ds-top.config  @plugin=cachekey.so @pparam=--separator= @pparam=--remove-all-params=true @pparam=--remove-path=true @pparam=--capture-prefix-uri=/^([^?]*)/$1/  # ds 'ds-top' topology 'mso-topology'
+map https://atlanta-edge-03.ds-top.test.cdn1.net/ http://origin.topology.example.net/ @plugin=header_rewrite.so @pparam=dscp/set_dscp_40.config @plugin=header_rewrite.so @pparam=hdr_rw_first_ds-top.config  @plugin=cachekey.so @pparam=--separator= @pparam=--remove-all-params=true @pparam=--remove-path=true @pparam=--capture-prefix-uri=/^([^?]*)/$1/  # ds 'ds-top' topology 'mso-topology'

[zrhoffman]

Looks good, thanks!