Param Digger Tracker #7694

ArkaprabhaChakraborty · 2023-01-16T14:11:56Z

This is the list of tasks underway for the Param Digger add-on for ZAP

Change output window style for easy access to messages. (paramdigger: Add output table by removing output textpane zap-extensions#4363)
Add confirmed URLs/messages to the site tree. (With icon like spiders, force browse, etc)
Automation Framework Job/Tasks.
Clear the Param Digger panel on session change. - paramdigger: Handle session change & previous PR follow-up zap-extensions#5306
Provide a mechanism to retain original headers, or do so by default and provide a mechanism to minimize/trim them. - 🚧 @kingthorin

thc202 · 2023-04-21T16:53:51Z

There are some exceptions when using the header guesser, e.g.:

[ZAP-ParamGuesser-0-thread-2] ERROR org.zaproxy.addon.paramdigger.HeaderGuesser - java.lang.NullPointerException
java.lang.NullPointerException: null
	at org.zaproxy.addon.paramdigger.HeaderGuesser.checkFirstRequestPoisoning(HeaderGuesser.java:320) ~[?:?]
	at org.zaproxy.addon.paramdigger.HeaderGuesser.forwardTemplate(HeaderGuesser.java:218) ~[?:?]
	at org.zaproxy.addon.paramdigger.HeaderGuesser.forwardingHeaderGuess(HeaderGuesser.java:182) ~[?:?]
	at org.zaproxy.addon.paramdigger.HeaderGuesser.startGuess(HeaderGuesser.java:164) ~[?:?]
	at org.zaproxy.addon.paramdigger.HeaderGuesser.run(HeaderGuesser.java:132) ~[?:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]

ArkaprabhaChakraborty · 2023-04-21T17:17:15Z

That shouldn't have happened tho. Will check again why its happening.

…

On Fri, Apr 21, 2023, 10:24 PM thc202 ***@***.***> wrote: There are some exceptions when using the header guesser, e.g.: [ZAP-ParamGuesser-0-thread-2] ERROR org.zaproxy.addon.paramdigger.HeaderGuesser - java.lang.NullPointerException java.lang.NullPointerException: null at org.zaproxy.addon.paramdigger.HeaderGuesser.checkFirstRequestPoisoning(HeaderGuesser.java:320) ~[?:?] at org.zaproxy.addon.paramdigger.HeaderGuesser.forwardTemplate(HeaderGuesser.java:218) ~[?:?] at org.zaproxy.addon.paramdigger.HeaderGuesser.forwardingHeaderGuess(HeaderGuesser.java:182) ~[?:?] at org.zaproxy.addon.paramdigger.HeaderGuesser.startGuess(HeaderGuesser.java:164) ~[?:?] at org.zaproxy.addon.paramdigger.HeaderGuesser.run(HeaderGuesser.java:132) ~[?:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] — Reply to this email directly, view it on GitHub <#7694 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AN6F3GM7M3ZE3VN5JQYHMQLXCK3SXANCNFSM6AAAAAAT4ZMZEE> . You are receiving this because you were assigned.Message ID: ***@***.***>

thc202 · 2023-04-21T17:32:02Z

You can reproduce with the following server (extender script), e.g.:

var address = "0.0.0.0"
var port = 9000

var HttpResponseHeader = Java.type("org.parosproxy.paros.network.HttpResponseHeader")
var extensionNetwork = control.getExtensionLoader().getExtension("ExtensionNetwork")
var server

function messageHandler(ctx, msg) {
    ctx.overridden()

    msg.setResponseHeader(new HttpResponseHeader("HTTP/1.1 200\r\nConnection: close"))
}

function install(helper) {
    server = extensionNetwork.createHttpServer(messageHandler)
    server.start(address, port)
}

function uninstall(helper) {
    server.stop()
}

ArkaprabhaChakraborty · 2023-04-22T08:28:15Z

You need to specify the method and wordlist right? :)

thc202 · 2023-04-22T08:54:05Z

It's using the defaults, GET and Predefined.

ArkaprabhaChakraborty · 2023-04-23T14:06:06Z

There are some exceptions when using the header guesser, e.g.:

[ZAP-ParamGuesser-0-thread-2] ERROR org.zaproxy.addon.paramdigger.HeaderGuesser - java.lang.NullPointerException
java.lang.NullPointerException: null
	at org.zaproxy.addon.paramdigger.HeaderGuesser.checkFirstRequestPoisoning(HeaderGuesser.java:320) ~[?:?]
	at org.zaproxy.addon.paramdigger.HeaderGuesser.forwardTemplate(HeaderGuesser.java:218) ~[?:?]
	at org.zaproxy.addon.paramdigger.HeaderGuesser.forwardingHeaderGuess(HeaderGuesser.java:182) ~[?:?]
	at org.zaproxy.addon.paramdigger.HeaderGuesser.startGuess(HeaderGuesser.java:164) ~[?:?]
	at org.zaproxy.addon.paramdigger.HeaderGuesser.run(HeaderGuesser.java:132) ~[?:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]

While the patch for this is underway... I noticed a miss in my detection but couldn't pinpoint why it is happening..
Request through Param Digger:

GET https://005a3c92.poison.digi.ninja:2443/basic.php HTTP/1.1
X-Forwarded-Host: 9549327
Host: 005a3c92.poison.digi.ninja:2443

Response with no seeming reflection:

HTTP/1.1 200 OK
Date: Sun, 23 Apr 2023 13:57:56 GMT
Server: Apache
Strict-Transport-Security: max-age=63072000
Vary: Accept-Encoding
X-Content-Type-Options: nosniff
Content-Type: text/html; charset=UTF-8
X-Varnish: 7005808 1525956
Age: 3
Via: 1.1 varnish (Varnish/6.5)
Accept-Ranges: bytes
Content-Length: 1036
Referrer-Policy: no-referrer-when-downgrade

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
	<title>Basic - Cache Poisoning Lab</title>
	<link rel="shortcut icon" type="image/x-icon" href="/favicon.ico" /> 
	<link rel="apple-touch-icon" href="/apple-touch-icon.png" />
	<meta property="og:title" content="Basic Poisoning Lab" />
	<meta property="og:type" content="website" />
	<meta property="og:image" content="http://005a3c92.poison.digi.ninja:2443/cms/social.png" />
</head>
<body>
	<h1>Basic Poisoning Lab</h1>
	<p>A random number is: 7045530</p>
	<p>
		This page loads its open graph image based on the host requested but isn't that clever when working out the host to use.
	</p>
	<p>
		Use this weakness to poison every visitor with a nice bit of Cross-Site Scripting.
	</p>
	<p><a href="/index.php">&laquo; Back to home</a></p>
	<hr />
	<p>
		Lab created by Robin Wood - <a href="https://digi.ninja">DigiNinja</a>
	</p>
</body>
</html>

But when I send the same using the requester tab/ manually. I get a reflection (<meta property="og:image" content="http://9549327/cms/social.png" />):
Response:

HTTP/1.1 200 OK
Date: Sun, 23 Apr 2023 14:02:08 GMT
Server: Apache
Strict-Transport-Security: max-age=63072000
Vary: Accept-Encoding
X-Content-Type-Options: nosniff
Content-Type: text/html; charset=UTF-8
X-Varnish: 1525959
Age: 0
Via: 1.1 varnish (Varnish/6.5)
Accept-Ranges: bytes
Content-Length: 1015
Referrer-Policy: no-referrer-when-downgrade

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
	<title>Basic - Cache Poisoning Lab</title>
	<link rel="shortcut icon" type="image/x-icon" href="/favicon.ico" /> 
	<link rel="apple-touch-icon" href="/apple-touch-icon.png" />
	<meta property="og:title" content="Basic Poisoning Lab" />
	<meta property="og:type" content="website" />
	<meta property="og:image" content="http://9549327/cms/social.png" />
</head>
<body>
	<h1>Basic Poisoning Lab</h1>
	<p>A random number is: 1673380440</p>
	<p>
		This page loads its open graph image based on the host requested but isn't that clever when working out the host to use.
	</p>
	<p>
		Use this weakness to poison every visitor with a nice bit of Cross-Site Scripting.
	</p>
	<p><a href="/index.php">&laquo; Back to home</a></p>
	<hr />
	<p>
		Lab created by Robin Wood - <a href="https://digi.ninja">DigiNinja</a>
	</p>
</body>
</html>

Edit: I forgot to mention I did find a reflection on the same tag with the Host Header successfully with the add-on :). Don't know if it can be exploited tho.
Edit 2: I think I may have found the rookie mistake :).
Lab credits: @digininja

ArkaprabhaChakraborty · 2023-04-23T14:08:28Z

I think I'm missing some configuration for HttpSender() but not sure why :)... Can be a rookie mistake tho :)

kingthorin · 2023-04-23T14:50:18Z

Re: Edit 2, so it did work in the end? Or you think you’ve identified the potential discrepancy?

ArkaprabhaChakraborty · 2023-04-23T15:29:07Z

Working out a fix to the rookie mistake. The cache controller needs a Fastly/Varnish Caching patch :) which I'm incorporating into the fix.

ArkaprabhaChakraborty · 2023-04-23T15:30:58Z

I had a "PURGE" and "FASTLYPURGE" logic which is interfering in the detection :)

ArkaprabhaChakraborty · 2023-04-23T15:34:20Z

More context:
Request:

PURGE https://005a3c92.poison.digi.ninja:2443/basic.php?fcbz=1 HTTP/1.1
Host: 005a3c92.poison.digi.ninja:2443

Response (No X-Varnish Header and no Age Header as well):

HTTP/1.1 200 OK
Date: Sun, 23 Apr 2023 15:04:32 GMT
Server: Apache
Strict-Transport-Security: max-age=63072000
Vary: Accept-Encoding
X-Content-Type-Options: nosniff
Content-Type: text/html; charset=UTF-8
Referrer-Policy: no-referrer-when-downgrade
content-length: 1038

After this, I'm stupidly checking for caching... :)

digininja · 2023-04-23T18:43:06Z

It's been four years since I wrote that lab, if you want any changes so you can test things I can see what I can do but can't promise.

…

On Sun, 23 Apr 2023, 16:34 Arkaprabha Chakraborty, ***@***.***> wrote: More context: Request: PURGE https://005a3c92.poison.digi.ninja:2443/basic.php?fcbz=1 HTTP/1.1 Host: 005a3c92.poison.digi.ninja:2443 Response (No X-Varnish Header and no Age Header as well): HTTP/1.1 200 OK Date: Sun, 23 Apr 2023 15:04:32 GMT Server: Apache Strict-Transport-Security: max-age=63072000 Vary: Accept-Encoding X-Content-Type-Options: nosniff Content-Type: text/html; charset=UTF-8 Referrer-Policy: no-referrer-when-downgrade content-length: 1038 After this, I'm stupidly checking for caching... :) — Reply to this email directly, view it on GitHub <#7694 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA4SWMS75ARY32E6HNJMLDXCVDYPANCNFSM6AAAAAAT4ZMZEE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

ArkaprabhaChakraborty · 2023-04-23T19:08:18Z

Thanks @digininja! But your 4-year-old work isn't that bad tho :).

ArkaprabhaChakraborty · 2023-04-23T19:15:52Z

Fixed... all it needed was some sleepy time to detect the caching :P

double16 · 2024-02-15T14:49:09Z

The requests used by paramdigger are missing a lot of headers from the original request.

Original request:

GET http://manager.htb/search.html HTTP/1.1
host: manager.htb
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Connection: keep-alive
Upgrade-Insecure-Requests: 1

paramdigger request:

GET http://manager.htb/search.html?zap=123 HTTP/1.1
host: manager.htb

On a bug bounty I'm working, the missing headers are triggering cloudfront to deny the requests.

ArkaprabhaChakraborty · 2024-02-15T14:54:41Z

You're missing the headers that are generally present when a request has been generated by a browser. Configuring ZAP to have these headers for every request might help. Thanks Arka

…

On Thu, Feb 15, 2024, 8:19 PM Patrick Double ***@***.***> wrote: The requests used by paramdigger are missing a lot of headers from the original request. Original request: GET http://manager.htb/search.html HTTP/1.1 host: manager.htb User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Connection: keep-alive Upgrade-Insecure-Requests: 1 paramdigger request: GET http://manager.htb/search.html?zap=123 HTTP/1.1 host: manager.htb On a bug bounty I'm working, the missing headers are triggering cloudfront to deny the requests. — Reply to this email directly, view it on GitHub <#7694 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AN6F3GKXROUGY4FASKP53K3YTYN7DAVCNFSM6AAAAAAT4ZMZEGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBWGI2DQOJQGU> . You are receiving this because you were assigned.Message ID: ***@***.***>

double16 · 2024-02-15T15:14:04Z

When using the context menu on a request to attack using paramdigger, it seems reasonable for the user to expect the attack requests will look as much like the original request as reasonable. Configuring these headers before running paramdigger isn't the best experience, especially if they include things like session cookies.

Is there a reason why the original request headers aren't included?

kingthorin · 2024-02-17T21:12:31Z

@ArkaprabhaChakraborty had there been a reason for removing all the request headers?

ArkaprabhaChakraborty · 2024-02-17T21:16:46Z

Yes. One of the reasons is that Header brute force supplies a list of headers from a wordlist. Secondly it helps in detection of caching mechanism for web cache poisoning. That being said, I think I should provide an option to preserve the original request headers. Thanks Arka

…

On Sun, Feb 18, 2024, 2:42 AM Rick M ***@***.***> wrote: @ArkaprabhaChakraborty <https://github.com/ArkaprabhaChakraborty> had there been a reason for removing all the request headers? — Reply to this email directly, view it on GitHub <#7694 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AN6F3GNI3ESMLB633RV5RW3YUEMMXAVCNFSM6AAAAAAT4ZMZEGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJQGMZTSNZVHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

kingthorin · 2024-02-17T21:18:54Z

Thanks, I'll look at adding a button or something.

thc202 added add-on tracker labels Jan 16, 2023

thc202 assigned ArkaprabhaChakraborty Jan 16, 2023

ArkaprabhaChakraborty mentioned this issue Jan 16, 2023

paramdigger: Add output table by removing output textpane zaproxy/zap-extensions#4363

Merged

ArkaprabhaChakraborty mentioned this issue Apr 23, 2023

paramdigger: Add patches for header guesser errors zaproxy/zap-extensions#4561

Merged

kingthorin self-assigned this Feb 18, 2024

kingthorin added the In:paramdigger Items pertaining to the Param Digger add-on. label Feb 18, 2024

kingthorin mentioned this issue Feb 18, 2024

paramdigger: Handle session change & previous PR follow-up zaproxy/zap-extensions#5306

Open

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Param Digger Tracker #7694

Param Digger Tracker #7694

ArkaprabhaChakraborty commented Jan 16, 2023 •

edited by kingthorin

thc202 commented Apr 21, 2023

ArkaprabhaChakraborty commented Apr 21, 2023 via email

thc202 commented Apr 21, 2023

ArkaprabhaChakraborty commented Apr 22, 2023

thc202 commented Apr 22, 2023

ArkaprabhaChakraborty commented Apr 23, 2023 •

edited

ArkaprabhaChakraborty commented Apr 23, 2023

kingthorin commented Apr 23, 2023

ArkaprabhaChakraborty commented Apr 23, 2023 •

edited

ArkaprabhaChakraborty commented Apr 23, 2023

ArkaprabhaChakraborty commented Apr 23, 2023

digininja commented Apr 23, 2023 via email

ArkaprabhaChakraborty commented Apr 23, 2023

ArkaprabhaChakraborty commented Apr 23, 2023

double16 commented Feb 15, 2024

ArkaprabhaChakraborty commented Feb 15, 2024 via email

double16 commented Feb 15, 2024

kingthorin commented Feb 17, 2024

ArkaprabhaChakraborty commented Feb 17, 2024 via email

kingthorin commented Feb 17, 2024

Param Digger Tracker #7694

Param Digger Tracker #7694

Comments

ArkaprabhaChakraborty commented Jan 16, 2023 • edited by kingthorin

thc202 commented Apr 21, 2023

ArkaprabhaChakraborty commented Apr 21, 2023 via email

thc202 commented Apr 21, 2023

ArkaprabhaChakraborty commented Apr 22, 2023

thc202 commented Apr 22, 2023

ArkaprabhaChakraborty commented Apr 23, 2023 • edited

ArkaprabhaChakraborty commented Apr 23, 2023

kingthorin commented Apr 23, 2023

ArkaprabhaChakraborty commented Apr 23, 2023 • edited

ArkaprabhaChakraborty commented Apr 23, 2023

ArkaprabhaChakraborty commented Apr 23, 2023

digininja commented Apr 23, 2023 via email

ArkaprabhaChakraborty commented Apr 23, 2023

ArkaprabhaChakraborty commented Apr 23, 2023

double16 commented Feb 15, 2024

ArkaprabhaChakraborty commented Feb 15, 2024 via email

double16 commented Feb 15, 2024

kingthorin commented Feb 17, 2024

ArkaprabhaChakraborty commented Feb 17, 2024 via email

kingthorin commented Feb 17, 2024

ArkaprabhaChakraborty commented Jan 16, 2023 •

edited by kingthorin

ArkaprabhaChakraborty commented Apr 23, 2023 •

edited

ArkaprabhaChakraborty commented Apr 23, 2023 •

edited