Add headless header and status matchers

[dogancanbakir]

Proposed changes

This PR adds header and status matchers for the headless mode. Closes #3715.

Command:

$ nuclei -u http://www.example.com -t headless_test.yaml -headless

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v2.9.6

                projectdiscovery.io

[INF] Current nuclei version: v2.9.6 (latest)
[INF] Current nuclei-templates version: v9.5.2 (latest)
[INF] New templates added in latest release: 50
[INF] Templates loaded for current scan: 1
[INF] Targets loaded for current scan: 1
[headless-header-test] [headless] [critical] http://www.example.com

Test template:

id: headless-header-test

info:
  name: Testing headless matchers
  author: forgedhallpass
  severity: critical
  tags: headless

headless:
  - steps:
      - args:
          url: "{{BaseURL}}"
        action: navigate
      - action: waitload

    matchers-condition: and
    matchers:
      - type: word
        part: header
        words:
          - "text/html"
      - type: status
        status:
          - 200

Checklist

• Pull request is created against the dev branch
• All checks passed (lint, unit/integration/regression tests etc.) with my changes
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)

[ehsandeep]

id: headless-header-status-test

info:
  name: headless header + status test
  author: pdteam
  severity: info

headless:
  - steps:
      - args:
          url: "{{BaseURL}}"
        action: navigate
      - action: waitload

    matchers-condition: and
    matchers:
      - type: word
        part: header
        words:
          - text/plain

      - type: status
        status:
          - 200

echo https://scanme.sh | ./nuclei -t test.yaml -headless -silent

[headless-header-status-test] [headless] [info] https://scanme.sh

We can add the above test as part of the integration test for headless.

[ehsandeep]

Looks good!

One more thing, either we can create a follow-up ticket it or consider cover in this PR.

Similar to http engine, we can expose header key: values as a variable to use with dsl matchers / extractors.

Here is an example from existing http engine -

id: basic-example

info:
  name: Test HTTP Template
  author: pdteam
  severity: info

http:
  - method: GET
    path:
      - "{{BaseURL}}"

    extractors:
      - type: dsl
        dsl:
          - content_type

echo scanme.sh | ./nuclei -t aa.yaml -v -svd
                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v2.9.6

		projectdiscovery.io

[INF] Current nuclei version: v2.9.6 (latest)
[INF] Current nuclei-templates version: v9.5.2 (latest)
[INF] New templates added in latest release: 50
[INF] Templates loaded for current scan: 1
[INF] Targets loaded for current scan: 1
[INF] Running httpx on input host
[INF] Found 1 URL from httpx
[DBG] Final Protocol request variables: 
	1. File => 
	2. BaseURL => https://scanme.sh
	3. Host => scanme.sh
	4. RootURL => https://scanme.sh
	5. DN => scanme
	6. TLD => sh
	7. Input => https://scanme.sh
	8. SD => 
	9. Hostname => scanme.sh
	10. ip => 
	11. Scheme => https
	12. Path => 
	13. Port => 443
	14. RDN => scanme.sh
	15. FQDN => scanme.sh

[VER] [basic-example] Sent HTTP request to https://scanme.sh
[DBG] Protocol response variables: 
	1. BaseURL => https://scanme.sh
	2. Port => 443
	3. RDN => scanme.sh
	4. curl-command => curl -X 'GET' -d '' -H 'A .... 7.36' 'https://scanme.sh'
	5. content_type => text/plain; charset=utf-8
	6. template-info => {Test HTTP Template pdteam    {info} map[] <nil> }
	7. Host => scanme.sh
	8. File => 
	9. template-path => /Users/geekboy/Github/nuclei/v2/cmd/nuclei/aa.yaml
	10. host => https://scanme.sh
	11. Path => 
	12. RootURL => https://scanme.sh
	13. header => HTTP/1.1 200 OK  Connecti .... Jun 2023 09:55:38 GMT    
	14. request => GET / HTTP/1.1  Host: sca .... Accept-Encoding: gzip    
	15. SD => 
	16. Scheme => https
	17. Input => https://scanme.sh
	18. FQDN => scanme.sh
	19. status_code => 200
	20. duration => 0.2552055
	21. interactsh-server => 
	22. TLD => sh
	23. date => Thu, 08 Jun 2023 09:55:38 GMT
	24. type => http
	25. matched => https://scanme.sh
	26. content_length => 2
	27. template-id => basic-example
	28. response => HTTP/1.1 200 OK  Connecti .... n 2023 09:55:38 GMT    ok
	29. all_headers => HTTP/1.1 200 OK  Connecti .... Jun 2023 09:55:38 GMT    
	30. DN => scanme
	31. Hostname => scanme.sh
	32. ip => 128.199.158.128
	33. body => ok

[basic-example] [http] [info] https://scanme.sh [text/plain; charset=utf-8]

[dogancanbakir]

@ehsandeep, Can you create a separate issue for better clarity?

[ehsandeep]

@ehsandeep, Can you create a separate issue for better clarity?

#3796

[ShubhamRasal]

lgtm 🚀