Clarify some nuance around JSON extraction in the docs

[byt3bl33d3r]

Heya,

I struggled with this for a while and I think it would definitely be worth adding to the docs as this is a common situation. I have a template that basically looks like the below and the application returns a JSON response on each request:

id: mylfi

info:
  name: mylfi
  author: me
  severity: high
  description: my big bad lfi
  tags: lfi

variables:
  experiment_name: "{{rand_text_alpha(6)}}"

http:
  - raw:
      - |
        POST /anotherpath/create HTTP/1.1
        Host: {{Hostname}}
        Content-Type: application/json

        {"name": "{{experiment_name}}", "location": "/../../../../../../../../../../../../../../etc/"}

      - |
        POST /anotherpath/create HTTP/1.1
        Host: {{Hostname}}
        Content-Type: application/json

        {"experiment_id": "{{exid}}"}

      - |
        POST /yetanotherpath HTTP/1.1
        Host: {{Hostname}}
        Content-Type: application/json

        {"name": "{{experiment_name}}"}

      - |
        POST /anotherpath HTTP/1.1
        Host: {{Hostname}}
        Content-Type: application/json

        {"name": "{{experiment_name}}", "runid": "{{runid}}"}

      - |
        GET /somepath?name={{experiment_name}}HTTP/1.1
        Host: {{Hostname}}

    extractors:
      - type: json
        part: body
        internal: true
        name: exid
        json:
          - '.key1'

      - type: json
        part: body
        internal: true
        name: runid
        json:
          - '.run.key_2'

    matchers-condition: and
    matchers:
      - type: regex
        regex:
          - "root:.*:0:0:"

      - type: status
        status:
          - 200

Running nuclei in debug mode (e.g. nuclei -debug -stats -svd -u http://127.0.0.1 -t lfi.yaml) showed me that exid and runid we're being correctly parsed but they were also being "cleared out" by the time Nuclei issues the last request. Everything that I read from the docs/blogs told me this should be working and I initially thought it was a bug.

However, the problem was the following: Nuclei attempts to extract the values of the specified json keys on each request and as Nuclei uses JQ under the hood if a JSON response doesn't contain the specified key JQ returns null which is a valid value that Nuclei will assign to exid and runid.

In order to keep the values of exid and runid I had to modify the extractors to have the following JQ expression:

    extractors:
      - type: json
        part: body
        internal: true
        name: exid
        json:
          - '.key1//empty'

      - type: json
        part: body
        internal: true
        name: runid
        json:
          - '.run.key2//empty'

This way if the key doesn't exist in one of the many JSON response, JQ won't return null and Nuclei will keep the previously extracted valid values of the keys.

Hopefully this makes sense.

[MetzinAround]

Thanks for bringing this up! We appreciate you showing off this use case and bringing a solution, too!