Repository navigation
Refresh header target is missed when it uses URL=, quotes, or contains url=
#1903
Replies: 1 comment
|
the drop happens because parsefreshtag splits the header on the literal lowercase string url=. that is why an uppercase url or a space around the equals sign gives you nothing, and why a second url= inside the target cuts the target short. the fix is to stop splitting on that literal string and instead match the url keyword case-insensitively, only for the first one after the delay. after the semicolon, look for url, allow spaces around the equals sign, and then take everything after that equals sign as the target. that way a target like /login?url=/home stays whole, because you never split on the second url=. you also need to strip matching quotes from the target, so url='/next' becomes /next instead of keeping the quotes. i have not run this inside katana itself, so treat this as the patch direction rather than a tested katana build, but the parsing change itself is small and it covers every case in your table. |
Uh oh!
There was an error while loading. Please reload this page.
Katana version
devat 6f22665 (also v1.8.0).Current behavior
headerRefreshParserextracts the redirect target from theRefreshresponse header throughutils.ParseRefreshTag, which splits the header on the literal lowercase stringurl=. As a result:Refreshheader0; URL=/next/next0;Url=/next/next5; url = /next/next0; url='/next''/next'/next0; url=/login?url=/home/login?/login?url=/homeURL=in uppercase is a common way to write this header. The HTML refresh parsing algorithm matchesurlcase-insensitively, allows whitespace around=, and strips matching quotes. Katana drops these targets from the crawl, or queues a truncated URL when the target has aurl=query parameter.Expected behavior
Katana extracts the target URL in every row of the table above.
Steps to reproduce
Run
go test ./pkg/utils/ -run TestRefreshRepro -v. Output:"0; URL=/next" -> ""and"0; url=/login?url=/home" -> "/login?".Against a live server, a page that responds with
Refresh: 0; URL=/nextdoes not lead to/nextin the crawl output.All reactions