Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XPath matcher support #4087

Merged
merged 5 commits into from Aug 25, 2023
Merged

Conversation

denandz
Copy link
Contributor

@denandz denandz commented Aug 24, 2023

Proposed changes

This pull request adds XPath matcher support. The same libraries as used by the extractor functionality are re-used here, so we have no new dependencies. Closes #4086

The following template and output shows the new XPath matcher support:

id: title-check

info:
  name: title-check
  author: test
  severity: critical

http:
  - raw:
      - |
        GET / HTTP/1.1
        Host: {{Hostname}}

    matchers:
      - type: xpath
        part: body
        xpath:
          - "/html/head/title[contains(text(), 'Example Domain')]"
:~$ nuclei -t xpath-test.yaml -u http://example.com

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v2.9.12

		projectdiscovery.io

[INF] Current nuclei version: v2.9.12 (latest)
[INF] Current nuclei-templates version: v9.6.2 (latest)
[INF] New templates added in latest release: 61
[INF] Templates loaded for current scan: 1
[INF] Targets loaded for current scan: 1
[title-check] [http] [critical] http://example.com/

The validation logic has been extended to return information about XPath syntax errors to end users, rather than just silently failing.

:~$ ./nuclei -t xpath-brokentest.yaml -validate

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v2.9.12

		projectdiscovery.io

[ERR] Error occurred parsing template /home/doi/xpath-brokentest.yaml: could not compile request: could not compile operators: could not compile matcher: /html/head/title[contains(text(), 'Example Domain'))] has an invalid token
[FTL] Could not validate templates: errors occurred during template validation

Checklist

  • Pull request is created against the dev branch
  • All checks passed (lint, unit/integration/regression tests etc.) with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

@ehsandeep ehsandeep linked an issue Aug 25, 2023 that may be closed by this pull request
@ehsandeep ehsandeep added the Type: Enhancement Most issues will probably ask for additions or changes. label Aug 25, 2023
Copy link
Member

@tarunKoyalwar tarunKoyalwar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm !

Awesome Work @denandz !!!

$ go run . -u example.com -t ~/test-templates/xpath-matcher.yaml -ms              

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v2.9.12

		projectdiscovery.io

[INF] Current nuclei version: v2.9.12 (latest)
[INF] Current nuclei-templates version: v9.6.2 (latest)
[INF] New templates added in latest release: 61
[INF] Templates loaded for current scan: 1
[INF] Targets loaded for current scan: 1
[INF] Running httpx on input host
[INF] Found 1 URL from httpx
[title-check] [matched] [http] [critical] https://example.com/

@ehsandeep ehsandeep merged commit f520d7e into projectdiscovery:dev Aug 25, 2023
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Enhancement Most issues will probably ask for additions or changes.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

XPath matcher support
3 participants