tenable_sc.vulnerability: Fix API sort order

[kcreddy]

Proposed commit message

tenable_sc.vulnerability: Fix API sort order

The Tenable /analysis endpoint doesn't sort the data 
using any order. So remove the existing "sort_direction" and 
"sortField" options. Instead use the "lastSeen" inside 
"filters" using epoch timestamp[1] to move the cursor. An 
"Additional Look-back Time" option is also added to fetch 
late arriving data.

Other changes:
- Add "HTTP Client Timeout" option to input templates.
  This configuration helps avoid API timeouts.
- Update default interval to 4 hours in vulnerability 
  data streams to avoid frequent polls as recommended 
  by Tenable.

[1]: https://developer.tenable.com/docs/sc-integrations#data-exports

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

Updated system tests with pagination paths and assert.hit_count should pass.

--- Test results for package: tenable_sc - START ---
╭────────────┬───────────────┬───────────┬───────────┬────────┬───────────────╮
│ PACKAGE    │ DATA STREAM   │ TEST TYPE │ TEST NAME │ RESULT │  TIME ELAPSED │
├────────────┼───────────────┼───────────┼───────────┼────────┼───────────────┤
│ tenable_sc │ vulnerability │ system    │ default   │ PASS   │ 39.414923208s │
╰────────────┴───────────────┴───────────┴───────────┴────────┴───────────────╯
--- Test results for package: tenable_sc - END   ---
Done

Related issues

• Closes tenable_sc.vulnerability: API sort order #17164

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 59eec96

History

• 💚 Build #37134 succeeded 344a9ca
• 💚 Build #37132 succeeded b73ce60

cc @kcreddy

[ShourieG]

LGTM

[elastic-vault-github-plugin-prod]

Package tenable_sc - 2.1.0 containing this change is available at https://epr.elastic.co/package/tenable_sc/2.1.0/