Timeout issue

[mastercho]

Nuclei version: 3.2.8

Current Behavior:

When try to run sql injection time based templates which is not with raw request and don't have @timeout in template its get

[WRN] [CVE-2023-6063] Could not execute step: [:RUNTIME] got following errors while executing flow <- [:RUNTIME] failed to execute http:1 protocol <- [:RUNTIME] got err while executing https://xxx.com/wp-login.php <- context deadline exceeded
[17:50]

Tried to reproduce on localhost instance with custom template and its gets same timeout error and it shouldn't. Tried older version before #5148 and its works. Something might be wrong in PR. Also when debugging in BURP i can see request is not timeout its normal and complete request but for some reason its throws error in nuclei.

Expected Behavior:

To not give runtime error

Steps To Reproduce:

You can try on CVE-2023-6063

[mastercho]

Seems @ritikchaddha updated template on my advice but you got the idea where is the issue, yet still can be tested on old version of template before this commit projectdiscovery/nuclei-templates@57776fd

[ehsandeep]

@mastercho @timeout is feature of raw request, so it won't work with base request format, so it is expected, we suggest to use raw format when you are looking to overwrite default nuclei timeout via template, do you face any specific blocker to not use raw format for such cases?

[mastercho]

@mastercho @timeout is feature of raw request, so it won't work with base request format, so it is expected, we suggest to use raw format when you are looking to overwrite default nuclei timeout via template, do you face any specific blocker to not use raw format for such cases?

I'm surprised that you don't know about the case when 90% of your team already know...

Yes i can't use raw in my case because payload have "

such

archive?id=1&page_type=category&rewrite=news&subpage_type=post" AND (SELECT 5728 FROM (SELECT(SLEEP(5)))AuDU)-- lafl

URLEncode makes payload to not execute so we can't use URLEncode.

According to whole PD Template team we can't escape " in request too, so only way is to use method GET with the path but seems on recent PR #5148 you did something which cause

Could not execute step: [:RUNTIME] got err while executing (all details was provided to every PD template team and @tarunKoyalwar in discord before the issue in Github, but seems you are not discussing issues in Slack)

Then i went and check SQLi Time Based templates who are not using RAW request (and RAW request who dont have @timeout) and issues exists on them too.

[ehsandeep]

@mastercho there are over 30 OSS projects with hundreds of issues, so it's understandable that no team member can have context for all issues at all times. To keep it simple and avoid unnecessary back-and-forth, please include all relevant information directly in the issue itself to review instead of referencing offline conversations.

About the example you shared, I can see it works by replacing space with + sign.

id: basic-raw-example

info:
  name: Test RAW Template
  author: pdteam
  severity: info

http:
  - raw:
      - |
        GET /archive?id=1&page_type=category&rewrite=news&subpage_type=post"+AND+(SELECT+5728+FROM+(SELECT(SLEEP(5)))AuDU)--+lafl HTTP/1.1
        Host: {{Hostname}}
        Origin: {{BaseURL}}

    matchers:
      - type: word
        words:
          - "Test is test matcher text"
        negative: true

If this doesn't work for you, can you share the actual example / template that did not worked with raw format?

[mastercho]

+ sign solved issue with template which i was creating but this doesn't solve issue in all affected time based sql injection templates who are broken by this PR #5148

GHSA-r4qv-crh6-rjvj was updated after i tell PD template team about the issue and show the solution,

but there is lot more templates which has to be updated with @timeout added to them. I guess you didn't predict this will broke templates in feature...

About conversations, i did not share my template on purpose because my messages is ignored and then someone else copy paste my work and being approved in 5 minutes... If you consider Discord as offline conversation then you must not use Slack and Jira too...

Lets focus on issue now, if anything once come across on my attention with team, i will contact you on person as you are CTO and we are on same mission to simplify the security process . And if you prefer Discord is not right, will call you on phone (bit intrusive)...

[ehsandeep]

all affected time based sql injection templates who are broken by this PR #5148

but there is lot more templates which has to be updated with @timeout added to them. I guess you didn't predict this will broke templates in feature...

@mastercho, can you please provide examples or list specific templates you are talking about? In PR #5148, we fixed a bug that caused @timeout not to work, but it seems you mentioning this fix has caused new issues. Sharing specific examples will help us address the problem more effectively.

And if you prefer Discord is not right, will call you on phone (bit intrusive)...

Regarding offline communication, I meant private conversations. Public discussions (either on Discord or GitHub) are beneficial for context, ensuring everyone is on the same page.

[mastercho]

all affected time based sql injection templates who are broken by this PR #5148

but there is lot more templates which has to be updated with @timeout added to them. I guess you didn't predict this will broke templates in feature...

@mastercho, can you please provide examples or list specific templates you are talking about? In PR #5148, we fixed a bug that caused @timeout not to work, but it seems you mentioning this fix has caused new issues. Sharing specific examples will help us address the problem more effectively.

And if you prefer Discord is not right, will call you on phone (bit intrusive)...

Regarding offline communication, I meant private conversations. Public discussions (either on Discord or GitHub) are beneficial for context, ensuring everyone is on the same page.

Upon your request i went and check on templates which i made with valid targets and have interesting founds

https://github.com/projectdiscovery/nuclei-templates/blob/55aa46ad954013889492a3ed75c818cc80cbe903/http/cves/2023/CVE-2023-30150.yaml#L40

For example this template suffer from same error, Could not execute step: [:RUNTIME] got err while executing but think i found another issue too. adding @timeout would fix Could not execute step: [:RUNTIME] got err while executing error and match vuln site unless if site doesn't have redirect like site.com/en/, if site follows redirect its not matching the final destination page response. That's on every template which i tried. I don't know redirect issue since which version occurred.

My second find is if there is direct request to payload we won't get Could not execute step: [:RUNTIME] got err while executing error, this error appears only on templates who have other request before the time based injection payload.

Example template

https://github.com/projectdiscovery/nuclei-templates/blob/55aa46ad954013889492a3ed75c818cc80cbe903/http/cves/2023/CVE-2023-46347.yaml#L37

Still if have redirect won't match final destination page response...

Lastly...
https://github.com/projectdiscovery/nuclei-templates/blob/55aa46ad954013889492a3ed75c818cc80cbe903/http/cves/2022/CVE-2022-22897.yaml#L5

For some reason not getting Could not execute step: [:RUNTIME] got err while executing error without @timeout but even after i add host-redirects: true still not matching redirect location response, if i run site without redirect would match without error.

I can provide targets for testing whoever from PD contacts me for debug. I think there is some serious issue on DSL because who know how many templates like that is affected right now

[mastercho]

@ehsandeep any update on this, or any more requests from me?

[ehsandeep]

@mastercho all the templates you listed doesn't have additional timeout that I've added in projectdiscovery/nuclei-templates#10095 along with few other improvements to similar templates.

[mastercho]

Yeah we can do this for RAW requests but these with non RAW requests like was GHSA-r4qv-crh6-rjvj before still can cause this issue, why you delete some templates doe? Also seems redirect issue which i mention is more critical than timeout one

[ehsandeep]

@mastercho I did mention about why some templates were removed in the PR description, also can you please create separate issue for redirect issue you mentioning in nuclei template project with example?

[mastercho]

So you will go on update tons of Time Based SQLi templates method instead of fixing in Nuclei?

I don't know why you want redirect issue in templates project when is nuclei based issue but ok

[ehsandeep]

So you will go on update tons of Time Based SQLi templates method instead of fixing in Nuclei?

I don't know why you want redirect issue in templates project when is nuclei based issue but ok

@mastercho for the timeout issue, as I mentioned earlier, custom timeout is not and won't be supported for base format as base format is not suitable for advance use cases, so updating templates is only option and nothing needs to be changed in nuclei.

later you mentioned about new issue related to redirects which is still unclear so I suggest opening new issue with information that can be followed to validate and fix, following multiple issues in single ticket is not sufficient I would suggest closing this issue unless their is anything else related to timeout?

[mastercho]

Yeah i do understand this, what i meant, to overcome and solve this issue need go and update every SQL Time based template who is done in base format requests into raw requests with added @timeout to make it work, it's a lot of work (unless template bot can do it) then we can close this issue...

I'll proceed with redirects issue now on templates project to keep seperate as you said.

[ehsandeep]

Yeah i do understand this, what i meant, to overcome and solve this issue need go and update every SQL Time based template who is done in base format requests into raw requests with added @timeout to make it work, it's a lot of work (unless template bot can do it)

We have already converted all the templates related to SQLI to raw format, so we don't need to convert any, instead create new templates in raw format given base format can not be used.