Auto-Merge Parameters and Paths

[wdahlenburg]

I find that I'm usually working with a lot of urls that have mixed endings. Some end in paths, others in endpoints, and some with URI parameters.

One thing that I think would go a long ways towards sending better requests would be auto merging parameters and paths.

Currently if I send https://server.com/search?s=foobar through the following template I will get a request made for https://server.com/search?s=foobar/services/foo

requests:
  - method: GET
    path:
      - '{{BaseURL}}/services/foo'

    matchers-condition: and
    matchers:
      - type: word
        words:
          - '<title>Foo</title>'
      - type: status
        status:
          - 200

Manually I'd expect the path to be merged and the parameter to be set in the correct location. I'd expect the request to go to https://server.com/search/services/foo?s=foobar instead.

Similarly if I send the same url through a template that defines additional parameters I'd expect the parameters are joined where possible.

Take a variation of https://github.com/projectdiscovery/nuclei-templates/blob/master/fuzzing/linux-lfi-fuzzing.yaml for example

- "{{BaseURL}}?name=..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2fetc/passwd"

With this, I'll see a request for

https://server.com?s=foobar?name=..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2fetc/passwd

Manually I'd want this to be

https://server.com?s=foobar&name=..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2fetc/passwd

For parameters where the param already exists, I think there could be discussion about what the default case should be. Is it fine to replace the predefined parameter? Should both values be submitted? Should the value of the parameter in the template be appended to the url defined value?

A safe default could be to replace it the existing parameter value.

---

What are some thoughts on this?

[ehsandeep]

@wdahlenburg Hey, we need to replace {{BaseURL}} with {{RootURL}}, we will making this changes throughout all the HTTP templates except a few where {{BaseURL}} is actually expected, but for most of them {{RootURL}} is required value which got added later in nuclei engine and now supported in recent versions of nuclei, so we are good to make this change in all templates, for your own template consider using {{RootURL}}, in that way, it will always pick the Root URL, no matter if the input contains paths or whatever.

[ehsandeep]

Ignore this response, this might be valid for other case and not for this specific discussion.

[wdahlenburg]

It looks like {{RootURL}} will clobber the existing path and remove any existing parameters.

With the first example I see a request to

https://server.com/services/foo

The second will send a request for

https://server.com?name=..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2fetc/passwd

---

I think {{RootURL}} makes a lot of sense for vendor devices where the path is static and you are confident there is no proxy in front of the app. Reverse proxies and other routing mechanisms can be used to place a CMS, web app, or really anything else in a non-root folder. I'm not sure if a broad swap is great. I've had fairly good success running non-root folders through nuclei because most bug bounty hunters only test the root folder.

[ehsandeep]

@wdahlenburg this is now fixed in dev with #3010

[wdahlenburg]

Awesome job @tarunKoyalwar