Request generation failure when url contains %u encoded patterns

[denizcevik]

Here is tested environment version details

Nuclei Engine 2.8.9 (latest)
Nuclei Templates 9.3.7 (latest)

Sample template portion:

requests:
  - method: GET
    path:
      - "{{BaseURL}}/%u002e/WEB-INF/web.xml"

When defined test pattern is like above, nuclei is generating request as below.

GET /https://hostname/%25u002e/WEB-INF/web.xml HTTP/1.1
Host: hostname
User-Agent: Go-http-client/1.1

It seems there is a problem when %u encoded patterns are used.

[tarunKoyalwar]

@denizcevik , the url you mentioned {{BaseURL}}/%u002e/WEB-INF/web.xml is not a valid url and even go standard library does not support it

package main

import (
	"fmt"
	"net/url"
)

func main() {
	testurl := "https://scanme.sh/%u002e/WEB-INF/web.xml"

	urlx, err := url.Parse(testurl)
	if err != nil {
		panic(err)
	}
	fmt.Println(urlx.String())
}

$  go run .                                                                                                                                                                                                                                                                                              
panic: parse "https://scanme.sh/%u002e/WEB-INF/web.xml": invalid URL escape "%u0"

goroutine 1 [running]:
main.main()
        /Users/tarun/testing/urllx/main.go:13 +0x8c
exit status 2

Instead of panic we escape it our version of url parser . that is why %u002e -> %25u002e

To Summarize

this is the reason we have unsafe option in nuclei template . which keeps the payload intact even if it is not correct
https://nuclei.projectdiscovery.io/templating-guide/protocols/http/#unsafe-http-requests

[tarunKoyalwar]

@denizcevik refer #3331 (comment) for detailed url encoding edgecases and examples . we will also add some notes about url encoding to nuclei docs