Filebeat input httpjson getRFC5988Link parsing failure

[sf-sharris]

The getRFC5988Link value template function will fail to parse Link strings in which there is no whitespace between the link's elements. As far as I understand the spec, whitespace is not required and there is at least one vendor producing pagination links without any whitespace.

For confirmed bugs, please report:

• Version: Filebeat 8.6.1 (x-pack) and prior versions
• Operating System: Any
• Discuss Forum URL:
• Steps to Reproduce:

Alter an existing test case removing whitespace

$ git diff value_tpl_test.go
diff --git a/x-pack/filebeat/input/httpjson/value_tpl_test.go b/x-pack/filebeat/input/httpjson/value_tpl_test.go
index c5332a9658..b989e2128b 100644
--- a/x-pack/filebeat/input/httpjson/value_tpl_test.go
+++ b/x-pack/filebeat/input/httpjson/value_tpl_test.go
@@ -214,8 +214,8 @@ func TestValueTpl(t *testing.T) {
                                lastResponse: newTestResponse(
                                        nil,
                                        http.Header{"Link": []string{
-                                               `<https://example.com/api/v1/users?after=00ubfjQEMYBLRUWIEDKK>; title="Page 3"; rel="next"`,      
-                                               `<https://example.com/api/v1/users?before=00ubfjQEMYBLRUWIEDKK>; title="Page 1"; rel="previous"`, 
+                                               `<https://example.com/api/v1/users?after=00ubfjQEMYBLRUWIEDKK>;title="Page 3";rel="next"`,        
+                                               `<https://example.com/api/v1/users?before=00ubfjQEMYBLRUWIEDKK>;title="Page 1";rel="previous"`,   
                                        }},
                                        "",
                                ),

Run test cases

#/go/src/github.com/elastic/beats/x-pack/filebeat/input/httpjson# go test -v . -run TestValueTpl/func_getRFC5988Link_multiple_rel_as_separate_strings_matches
=== RUN   TestValueTpl
=== RUN   TestValueTpl/func_getRFC5988Link_multiple_rel_as_separate_strings_matches
    value_tpl_test.go:680:
                Error Trace:    /go/src/github.com/elastic/beats/x-pack/filebeat/input/httpjson/value_tpl_test.go:680
                Error:          Not equal:
                                expected: "https://example.com/api/v1/users?before=00ubfjQEMYBLRUWIEDKK"
                                actual  : ""

                                Diff:
                                --- Expected
                                +++ Actual
                                @@ -1 +1 @@
                                -https://example.com/api/v1/users?before=00ubfjQEMYBLRUWIEDKK
                                +
                Test:           TestValueTpl/func_getRFC5988Link_multiple_rel_as_separate_strings_matches
    value_tpl_test.go:682:
                Error Trace:    /go/src/github.com/elastic/beats/x-pack/filebeat/input/httpjson/value_tpl_test.go:682
                Error:          Received unexpected error:
                                the template result is empty
                Test:           TestValueTpl/func_getRFC5988Link_multiple_rel_as_separate_strings_matches
--- FAIL: TestValueTpl (0.00s)
    --- FAIL: TestValueTpl/func_getRFC5988Link_multiple_rel_as_separate_strings_matches (0.00s)
FAIL
FAIL    github.com/elastic/beats/v7/x-pack/filebeat/input/httpjson      0.008s
FAIL

Potential Fix

beats/x-pack/filebeat/input/httpjson/value_tpl.go

Line 231 in 37ddd1e

var regexpLinkRel = regexp.MustCompile(`<(.*)>;.*\srel\="?([^;"]*)`)

-var regexpLinkRel = regexp.MustCompile(`<(.*)>;.*\srel\="?([^;"]*)`)
+var regexpLinkRel = regexp.MustCompile(`<(.*)>;.*rel\="?([^;"]*)`)

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)