add custom field

[ShubhamRasal]

• add custom field with regex to parse
• create default custom field file
• add flag to accept the custom field file

- name: title
  type: regex
  group: 1
  regex:
  - '(?im)<\s*title.*>(.*?)<\s*/\s*title>'

./katana -h flc
Katana is a fast crawler focused on execution in automation
pipelines offering both headless and non-headless crawling.

Usage:
  ./katana [flags]

Flags:
   -flc, -field-config string  path to custom field configuration file

eg

./katana -u https://projectdiscovery.io/#/  -f email -v  -d 2 -sf email

[ehsandeep]

For now, http headers are not accessible; similar to nuclei, part needs to be supported for field config.

part attribute can have 3 values, response which is default and have access to http response headers + body, header, and body as is additional options and can be used for specific needs.

- name: title
  type: regex
  part: body
  group: 1
  regex:
  - '(?im)<\s*title.*>(.*?)<\s*/\s*title>'

@ShubhamRasal let's not make any changes in JSON output, we will track it as a separate issue for changes in the JSON output.

[ShubhamRasal]

• added support for part option

# field-config.yaml
- name: email
  type: regex
  part: response
  regex:
  - ([a-zA-Z0-9._-]+@[a-zA-Z0-9._-]+\.[a-zA-Z0-9_-]+)

- name: server
  type: regex
  part: header
  regex:
  - ECS
  - "Server: cloudflare"

output:

./katana -f server -u https://example.com  -silent 
ECS
-----------------------------------------------------------------------------
./katana -f server -u https://projectdiscovery.io  -silent 
Server: cloudflare
Server: cloudflare
Server: cloudflare
-----------------------------------------------------------------------------
./katana -f email -u https://projectdiscovery.io  -silent 
team@launchdarkly.com
contact@projectdiscovery.io
contact@projectdiscovery.io

[ehsandeep]

@ShubhamRasal

• JOSN output fields update
• part: response should be used (implicitly) if not defined.