add resume option

[dogancanbakir]

This PR adds -resume option to recover from an early exit or ungraceful termination. Closing #374.

[Mzack9999]

Suggesting struct instead of bool in map value field type

[Mzack9999]

I think we might need to rethink this approach a bit. Using the inflight urls as seed root urls lead to start n instances equal to the number of urls and revisiting the same nodes multiple times (I'm using a simplehttpserver pointed to my go folder):

$ echo http://0.0.0.0:46661/  | go run . -resume /home/marco/.config/katana/resume-cj1sss13gitq4gtfs8mg.cfg
[INF] Resuming from save checkpoint

   __        __                
  / /_____ _/ /____ ____  ___ _
 /  '_/ _  / __/ _  / _ \/ _  /
/_/\_\\_,_/\__/\_,_/_//_/\_,_/                                                   

                projectdiscovery.io

[INF] Current katana version v1.0.3-dev (development)
[INF] Started standard crawling for => http://0.0.0.0:46661/bin/nuclei
[INF] Started standard crawling for => http://0.0.0.0:46661/bin/gopkgs
[INF] Started standard crawling for => http://0.0.0.0:46661/bin/go-outline
[INF] Started standard crawling for => http://0.0.0.0:46661/bin/dlv
[INF] Started standard crawling for => http://0.0.0.0:46661/bin/simplehttpserver
[INF] Started standard crawling for => http://0.0.0.0:46661/bin/naabu
[INF] Started standard crawling for => http://0.0.0.0:46661/bin/staticcheck
[INF] Started standard crawling for => http://0.0.0.0:46661/
[INF] Started standard crawling for => http://0.0.0.0:46661/bin/gopls
[INF] Started standard crawling for => http://0.0.0.0:46661/bin/dlv-dap
http://0.0.0.0:46661/src/
http://0.0.0.0:46661/pkg/
http://0.0.0.0:46661/src/
http://0.0.0.0:46661/src/

I think that we might need to track original root url and then just enqueue navigation requests for the same root url instance.

[Mzack9999]

As tracking in-flight URLs turned out to be quite challenging with list based approach, let's simplify this by tracking only the input items processed/in-flight/todo. For example the following input:

http://192.168.5.36:8000/pkg/
http://192.168.5.36:8000/src/
http://192.168.5.36:8000/bin/

With the execution interrupted at half:

$ cat list.txt  | go run . -c 1 -p 1
[INF] Started standard crawling for => http://192.168.5.36:8000/pkg/
...
[INF] Started standard crawling for => http://192.168.5.36:8000/src/
...
^C

So with the following situation:

http://192.168.5.36:8000/pkg/ # completed
http://192.168.5.36:8000/src/ # in-flight
http://192.168.5.36:8000/bin/ # todo

Should start processing from the beginning only those in-flight/todo http://192.168.5.36:8000/src/ and http://192.168.5.36:8000/bin/ )

[Mzack9999]

lgtm!

The elaboration starts from inflight incomplete items:

$ cat list.txt
http://192.168.5.36:8000/pkg/
http://192.168.5.36:8000/src/
http://192.168.5.36:8000/bin/
$ cat list.txt  | go run . -c 1 -p 1
...
[INF] Started standard crawling for => http://192.168.5.36:8000/src/
^C[INF] - Ctrl+C pressed in Terminal
$ cat list.txt  | go run . -c 1 -p 1 -resume /Users/xxx/.config/katana/resume-xxxxxxx.cfg
...
[INF] Started standard crawling for => http://192.168.5.36:8000/src/
...

Note: only fully completed items are skipped