fix showing multiple failure matches per template on -ms set #3770

RamanaReddy0M · 2023-06-01T02:07:44Z

#2693 Bug in matcher-status option

Checklist

Pull request is created against the dev branch
All checks passed (lint, unit/integration/regression tests etc.) with my changes
I have added tests that prove my fix is effective or that my feature works
I have added necessary documentation (if appropriate)

add integration test

tarunKoyalwar

requesting minor changes + add new flag to control behaviour of matcher status at template scope vs per request

v2/pkg/protocols/common/executer/executer.go

tarunKoyalwar

-msr should implicitly enable -ms to avoid conflicting scenarios

$ ./nuclei -u example.com -id CVE-2021-29490  -msr   

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v2.9.6

		projectdiscovery.io

[WRN] Found 3 templates loaded with deprecated protocol syntax, update before v2.9.5 for continued support.
[INF] Current nuclei version: v2.9.6 (latest)
[INF] Current nuclei-templates version: v9.5.2 (latest)
[INF] New templates added in latest release: 50
[INF] Templates loaded for current scan: 1
[INF] Targets loaded for current scan: 1
[INF] Running httpx on input host
[INF] Found 1 URL from httpx
[CVE-2021-29490] [http] [medium] https://example.com
[CVE-2021-29490] [http] [medium] https://example.com
[INF] No results found. Better luck next time!

^ output shows that target is vulnerable but actually i.e matcher status (and status is omitted)

suggesting updating PR description with POC . ex: command to run to test implementation

v2/cmd/integration-test/http.go

v2/pkg/output/output.go

v2/pkg/protocols/common/executer/executer.go

…3777) Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3.4.0 to 3.5.0. - [Release notes](https://github.com/golangci/golangci-lint-action/releases) - [Commits](golangci/golangci-lint-action@v3.4.0...v3.5.0) --- updated-dependencies: - dependency-name: golangci/golangci-lint-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [github.com/xanzy/go-gitlab](https://github.com/xanzy/go-gitlab) from 0.83.0 to 0.84.0. - [Changelog](https://github.com/xanzy/go-gitlab/blob/master/releases_test.go) - [Commits](xanzy/go-gitlab@v0.83.0...v0.84.0) --- updated-dependencies: - dependency-name: github.com/xanzy/go-gitlab dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…3780) Bumps [github.com/spf13/cast](https://github.com/spf13/cast) from 1.5.0 to 1.5.1. - [Release notes](https://github.com/spf13/cast/releases) - [Commits](spf13/cast@v1.5.0...v1.5.1) --- updated-dependencies: - dependency-name: github.com/spf13/cast dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…to 0.0.28 in /v2 (#3779) * chore(deps): bump github.com/projectdiscovery/fastdialer in /v2 Bumps [github.com/projectdiscovery/fastdialer](https://github.com/projectdiscovery/fastdialer) from 0.0.26 to 0.0.28. - [Release notes](https://github.com/projectdiscovery/fastdialer/releases) - [Commits](projectdiscovery/fastdialer@v0.0.26...v0.0.28) --- updated-dependencies: - dependency-name: github.com/projectdiscovery/fastdialer dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Bump retryabledns to 0.28 * Update the retryabledns --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>

* deprecatedProtocolNameTemplates * use syncLock * fix lint error * change version in deprecated warning msg * comment asnmap expand unit test --------- Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io> Co-authored-by: Tarun Koyalwar <45962551+tarunKoyalwar@users.noreply.github.com>

* Basic headless fuzzing * Remove debug statements * Add integration tests * Update template * Fix recognize payload value in matcher * Update tempalte * use req.SetURL() --------- Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>

…:robot:

* add headless header and status matchers * rename headers as header * add integration test for header+status * fix typo

Bumps golang from 1.20.4-alpine to 1.20.5-alpine. --- updated-dependencies: - dependency-name: golang dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [github.com/go-playground/validator/v10](https://github.com/go-playground/validator) from 10.11.2 to 10.14.1. - [Release notes](https://github.com/go-playground/validator/releases) - [Commits](go-playground/validator@v10.11.2...v10.14.1) --- updated-dependencies: - dependency-name: github.com/go-playground/validator/v10 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [github.com/projectdiscovery/rawhttp](https://github.com/projectdiscovery/rawhttp) from 0.1.11 to 0.1.13. - [Release notes](https://github.com/projectdiscovery/rawhttp/releases) - [Commits](projectdiscovery/rawhttp@v0.1.11...v0.1.13) --- updated-dependencies: - dependency-name: github.com/projectdiscovery/rawhttp dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…/v2 (#3812) Bumps [github.com/go-git/go-git/v5](https://github.com/go-git/go-git) from 5.6.1 to 5.7.0. - [Release notes](https://github.com/go-git/go-git/releases) - [Commits](go-git/go-git@v5.6.1...v5.7.0) --- updated-dependencies: - dependency-name: github.com/go-git/go-git/v5 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [github.com/projectdiscovery/hmap](https://github.com/projectdiscovery/hmap) from 0.0.11 to 0.0.13. - [Release notes](https://github.com/projectdiscovery/hmap/releases) - [Commits](projectdiscovery/hmap@v0.0.11...v0.0.13) --- updated-dependencies: - dependency-name: github.com/projectdiscovery/hmap dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

RamanaReddy0M · 2023-06-21T13:46:34Z

matcher-status

✗ ./nuclei -u example.com -id CVE-2021-29490  -ms 

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v2.9.6

                projectdiscovery.io

[INF] Current nuclei version: v2.9.6 (latest)
[INF] Current nuclei-templates version: v9.5.3 (latest)
[INF] New templates added in latest release: 82
[INF] Templates loaded for current scan: 1
[INF] Targets loaded for current scan: 1
[INF] Running httpx on input host
[INF] Found 1 URL from httpx
[CVE-2021-29490] [failed] [http] [medium] https://example.com

matcher status per request

✗ ./nuclei -u example.com -id CVE-2021-29490  -msr

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v2.9.6

                projectdiscovery.io

[INF] Current nuclei version: v2.9.6 (latest)
[INF] Current nuclei-templates version: v9.5.3 (latest)
[INF] New templates added in latest release: 82
[INF] Templates loaded for current scan: 1
[INF] Targets loaded for current scan: 1
[INF] Running httpx on input host
[INF] Found 1 URL from httpx
[CVE-2021-29490] [failed] [http] [medium] https://example.com
[CVE-2021-29490] [failed] [http] [medium] https://example.com

* adding random tls impersonate * dep update --------- Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>

* use templateman enhance api to populate cve info * rename cve-annotate => tmc add additional flags to format, lint and enhance template using templateman apis * minior changes * remove duplicate code * misc update * Add validate and error log option * print if updated * print format and enhance only if updated * make max-request optional * fix reference unmarshal error * fix removing self-contained tag --------- Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com> Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io> Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>

tarunKoyalwar

lgtm !

when using `-msr` (matcher status per request)

$ echo https://example.com | ./nuclei -tags oast -msr -silent | sort | uniq -c | sort -r
   7 [CVE-2023-27350] [failed] [http] [critical] https://example.com
   5 [request-based-interaction] [failed] [http] [info] https://example.com
   4 [CVE-2022-22965] [failed] [http] [critical] https://example.com
   3 [umbraco-base-ssrf] [failed] [http] [medium] https://example.com
   3 [fastjson-1-2-68-rce] [failed] [http] [critical] https://example.com
   3 [CVE-2022-22947] [failed] [http] [critical] https://example.com
   3 [CVE-2022-21587] [failed] [http] [critical] https://example.com
   3 [CVE-2020-28976] [failed] [http] [medium] https://example.com
   2 [fastjson-1-2-24-rce] [failed] [http] [critical] https://example.com
   2 [apache-solr-log4j-rce] [failed] [http] [critical] https://example.com
   2 [CVE-2022-31814] [failed] [http] [critical] https://example.com
   2 [CVE-2022-26134] [failed] [http] [critical] https://example.com
   2 [CVE-2022-24112] [failed] [http] [critical] https://example.com
   2 [CVE-2022-22972] [failed] [http] [critical] https://example.com
   2 [CVE-2021-44228] [failed] [http] [critical] https://example.com
   2 [CVE-2021-29490] [failed] [http] [medium] https://example.com
   2 [CVE-2021-21881] [failed] [http] [critical] https://example.com
   2 [CVE-2021-1497] [failed] [http] [critical] https://example.com
   2 [CVE-2020-28188] [failed] [http] [critical] https://example.com
   2 [CVE-2020-25506] [failed] [http] [critical] https://example.com
   2 [CVE-2020-17456] [failed] [http] [critical] https://example.com
   2 [CVE-2019-20224] [failed] [http] [high] https://example.com
   2 [CVE-2019-12988] [failed] [http] [critical] https://example.com
   2 [CVE-2019-12987] [failed] [http] [critical] https://example.com
   2 [CVE-2019-12986] [failed] [http] [critical] https://example.com
   2 [CVE-2019-12985] [failed] [http] [critical] https://example.com
   2 [CVE-2018-10818] [failed] [http] [critical] https://example.com
   2 [CVE-2018-10562] [failed] [http] [critical] https://example.com
   2 [CVE-2017-10271] [failed] [http] [high] https://example.com
   1 [zimbra-preauth-ssrf] [failed] [http] [critical] https://example.com
   1 [xenmobile-server-log4j] [failed] [http] [critical] https://example.com
   1 [wp-xmlrpc-pingback-detection] [failed] [http] [info] https://example.com
   1 [wordpress-ssrf-oembed] [failed] [http] [medium] https://example.com

when using `-ms` match (matcher status)

$  echo https://example.com | ./nuclei -tags oast -ms -silent | sort | uniq -c | sort -r
   1 [zimbra-preauth-ssrf] [failed] [http] [critical] https://example.com
   1 [xenmobile-server-log4j] [failed] [http] [critical] https://example.com
   1 [wp-xmlrpc-pingback-detection] [failed] [http] [info] https://example.com
   1 [wordpress-ssrf-oembed] [failed] [http] [medium] https://example.com
   1 [webpagetest-ssrf] [failed] [http] [high] https://example.com
   1 [vrealize-operations-log4j-rce] [failed] [http] [critical] https://example.com
   1 [vmware-vcenter-ssrf] [failed] [http] [critical] https://example.com
   1 [vmware-vcenter-log4j-jndi-rce] [failed] [http] [critical] https://example.com
   1 [vmware-siterecovery-log4j-rce] [failed] [http] [critical] https://example.com
   1 [vmware-operation-manager-log4j] [failed] [http] [critical] https://example.com
   1 [vmware-nsx-log4j] [failed] [http] [critical] https://example.com
   1 [vmware-horizon-log4j-jndi-rce] [failed] [http] [critical] https://example.com
   1 [vmware-hcx-log4j] [failed] [http] [critical] https://example.com
   1 [unifi-network-log4j-rce] [failed] [http] [critical] https://example.com
   1 [unauth-ztp-ping] [failed] [http] [high] https://example.com
   1 [umbraco-base-ssrf] [failed] [http] [medium] https://example.com
   1 [tls-sni-proxy] [failed] [http] [info] https://example.com
   1 [targa-camera-ssrf] [failed] [http] [high] https://example.com
   1 [ssrf-via-oauth-misconfig] [failed] [http] [medium] https://example.com
   1 [springboot-log4j-rce] [failed] [http] [critical] https://example.com

Note:

-msr does not show matchers status of each matcher ❌
it shows matcher status if there are multiple requests 👍

ehsandeep

./nuclei -pt tcp -tags cve -ms -u 1.1.1.1

not working with network protocol

RamanaReddy0M · 2023-06-27T18:18:51Z

✗ ./nuclei -pt tcp -tags cve -ms -u 1.1.1.1 -silent
[CVE-2022-31793] [failed] [tcp] [high] 1.1.1.1
[CVE-2018-2628] [failed] [tcp] [critical] 1.1.1.1
[CVE-2016-2004] [failed] [tcp] [critical] 1.1.1.1
[CVE-2022-24706] [failed] [tcp] [critical] 1.1.1.1
[CVE-2015-3306] [failed] [tcp] [high] 1.1.1.1
[CVE-2017-3881] [failed] [tcp] [critical] 1.1.1.1
[CVE-2023-33246] [failed] [tcp] [critical] 1.1.1.1
[CVE-2020-1938] [failed] [tcp] [critical] 1.1.1.1
[CVE-2016-3510] [failed] [tcp] [critical] 1.1.1.1
[CVE-2021-44521] [failed] [tcp] [critical] 1.1.1.1
[CVE-2001-1473] [failed] [tcp] [high] 1.1.1.1
[CVE-2020-7247] [failed] [tcp] [critical] 1.1.1.1
[CVE-2018-2893] [failed] [tcp] [critical] 1.1.1.1
[CVE-2022-0543] [failed] [tcp] [critical] 1.1.1.1
[CVE-2011-2523] [failed] [tcp] [critical] 1.1.1.1

ehsandeep

-ms option working as expected, we can remove -msr option as this feature is focused on template level match / fail status only.

remove -msr option

Note:

-ms option does not consider the template failed to execute because of missing dynamic variable as failed.
nuclei -t http/cves/2023/CVE-2023-29084.yaml -u example.com -ms -v

RamanaReddy0M added 2 commits June 1, 2023 07:26

fix showing multiple failure matchers per template

8b45bf1

add integration test

Merge branch 'dev' into issue-2693-bug-in-matcher-status

c7b5009

RamanaReddy0M self-assigned this Jun 1, 2023

RamanaReddy0M linked an issue Jun 1, 2023 that may be closed by this pull request

Bug in matcher-status option #2693

Closed

RamanaReddy0M changed the title ~~Issue 2693 bug in matcher status~~ fix showing multiple failure matches per template on -ms set Jun 1, 2023

exclude AS134029 from unit test

5430b8d

RamanaReddy0M requested a review from tarunKoyalwar June 1, 2023 03:26

Merge branch 'dev' into issue-2693-bug-in-matcher-status

1dd2c7a

tarunKoyalwar requested changes Jun 5, 2023

View reviewed changes

v2/pkg/protocols/common/executer/executer.go Outdated Show resolved Hide resolved

RamanaReddy0M added 2 commits June 6, 2023 02:14

Add flag for match status per request

84c231f

Merge branch 'dev' into issue-2693-bug-in-matcher-status

bb1afaa

RamanaReddy0M requested a review from tarunKoyalwar June 5, 2023 21:03

tarunKoyalwar requested changes Jun 13, 2023

View reviewed changes

v2/cmd/integration-test/http.go Outdated Show resolved Hide resolved

v2/pkg/output/output.go Outdated Show resolved Hide resolved

v2/pkg/protocols/common/executer/executer.go Outdated Show resolved Hide resolved

v2/pkg/protocols/common/executer/executer.go Outdated Show resolved Hide resolved

ehsandeep added the Status: Revision Needed Submitter of PR needs to revise the PR related to the issue. label Jun 19, 2023

dependabot bot and others added 16 commits June 21, 2023 12:39

enable no-httpx when passive scan is launched (#3789)

1da6a13

Issue 3339 headless fuzz (#3790)

a4b6236

* Basic headless fuzzing * Remove debug statements * Add integration tests * Update template * Fix recognize payload value in matcher * Update tempalte * use req.SetURL() --------- Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>

Auto Generate Syntax Docs + JSONSchema [Fri Jun 9 00:23:32 UTC 2023] …

1ced070

…:robot:

Add headless header and status matchers (#3794)

b5d6c39

* add headless header and status matchers * rename headers as header * add integration test for header+status * fix typo

Using safe dereferencing

a293d7b

adding comment

fd4f881

RamanaReddy0M added 6 commits June 21, 2023 12:39

update goflags dependency

9b0f7a4

update goflags dependency

7798802

bump goflags v0.1.9 => v0.1.10

4c6da7a

resovle merge conflicts

ca1df65

handle failure matcher flags logic at executor itself

7fa322b

add integration test to matcher status per request

ce2dfcf

RamanaReddy0M requested a review from ehsandeep June 21, 2023 13:58

Mzack9999 and others added 2 commits June 21, 2023 19:39

Adding random tls impersonate (#3844)

05d0c1f

* adding random tls impersonate * dep update --------- Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>

RamanaReddy0M removed the Status: Revision Needed Submitter of PR needs to revise the PR related to the issue. label Jun 21, 2023

resolve merge conflicts

23cc7f1

RamanaReddy0M requested a review from tarunKoyalwar June 23, 2023 06:54

tarunKoyalwar approved these changes Jun 26, 2023

View reviewed changes

ehsandeep requested changes Jun 26, 2023

View reviewed changes

tarunKoyalwar mentioned this pull request Jun 27, 2023

enable asnmap test #3881

Merged

4 tasks

Base automatically changed from dev to v3-beta June 27, 2023 16:29

ehsandeep changed the base branch from v3-beta to dev June 27, 2023 16:34

RamanaReddy0M added 2 commits June 27, 2023 22:49

fix matcher status with network protocol

eb3ee1c

fix test

83761e7

RamanaReddy0M requested a review from ehsandeep June 27, 2023 18:19

ehsandeep requested changes Jun 30, 2023

View reviewed changes

ehsandeep added Status: Revision Needed Submitter of PR needs to revise the PR related to the issue. Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors. labels Jun 30, 2023

remove -msr flag

65a31c1

ehsandeep approved these changes Jun 30, 2023

View reviewed changes

ehsandeep merged commit 6707bc7 into dev Jun 30, 2023
11 checks passed

ehsandeep deleted the issue-2693-bug-in-matcher-status branch June 30, 2023 18:02

ehsandeep removed the Status: Revision Needed Submitter of PR needs to revise the PR related to the issue. label Jun 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix showing multiple failure matches per template on -ms set #3770

fix showing multiple failure matches per template on -ms set #3770

RamanaReddy0M commented Jun 1, 2023 •

edited

tarunKoyalwar left a comment

tarunKoyalwar left a comment

RamanaReddy0M commented Jun 21, 2023

tarunKoyalwar left a comment

ehsandeep left a comment •

edited by RamanaReddy0M

RamanaReddy0M commented Jun 27, 2023

ehsandeep left a comment •

edited

fix showing multiple failure matches per template on -ms set #3770

fix showing multiple failure matches per template on -ms set #3770

Conversation

RamanaReddy0M commented Jun 1, 2023 • edited

Checklist

tarunKoyalwar left a comment

Choose a reason for hiding this comment

tarunKoyalwar left a comment

Choose a reason for hiding this comment

RamanaReddy0M commented Jun 21, 2023

tarunKoyalwar left a comment

Choose a reason for hiding this comment

when using -msr (matcher status per request)

when using -ms match (matcher status)

Note:

ehsandeep left a comment • edited by RamanaReddy0M

Choose a reason for hiding this comment

RamanaReddy0M commented Jun 27, 2023

ehsandeep left a comment • edited

Choose a reason for hiding this comment

RamanaReddy0M commented Jun 1, 2023 •

edited

when using `-msr` (matcher status per request)

when using `-ms` match (matcher status)

ehsandeep left a comment •

edited by RamanaReddy0M

ehsandeep left a comment •

edited