[BUG] Erroneous output to terminal when using --extract-links #114
Comments
|
Hi @Greenwolf, Thanks for the request and the kind words! I'm really glad you're enjoying it and getting some use out of it.
The current logic is as follows when
I'd love to know if you're seeing requests off the primary target domain, as that's definitely not intended. Can you let me know what you've observed and whether or not the description above meets the intent of this feature request? |
|
Hi @epi052, i ran it on domain A, and it seemed to start making requests on domain B. Am i misreading the output? I've checked the proxy logs and actually it doesn't seem to be making the request, but it's messing up the console output with all the non in scope items. Is that intentional? |
|
Just to make sure I understand correctly: When run with Do you ever see any of the off-target domain lines in the 'upper' output area, i.e. not just the progress bar? I'm guessing if they're not in the proxy logs, they're not in that output either. |
|
Yes that is correct. But i actually got 1000's of lines of the off-target domain output listed in the console. The command i used was this: ./feroxbuster -u https://original.domainA.org/ --extract-links --depth 2 --wordlist ./content-discovery/content_discovery_all.txt |
|
Good deal. Definitely sounds like it needs some attention. I'm wrapping up 1.5.0 now and should be able to check this out over the weekend. You've already narrowed down the possible location of the problem significantly, thank you! I'm switching this to a bug for now. |
epi052
added
bug
epi052
changed the title
|
@Greenwolf good morning! I'm trying to replicate what you're seeing. If you're able, could you confirm that some of the domains you saw requested are included below? |
|
probably some more |
|
Here's the update on this one. The wordlist you used from jhaddix contains entries like i showed above. Normally, a word from the wordlist is joined using reqwest::Url::join. When that function is called using a fully formed url as the 'word', it actually overwrites the base url. Example: So, the urls from the wordlist were the reason those requests were being shown. I tested with and without I added logic that issues a warning if a url is found in the wordlist, but it stops processing that word before anything actually happens. |
|
Sounds great, thank you @epi052. Sorry for the late reply, but yes I was seeing: 'http:techblog.dahmus.org'. Thank you for looking at this and for making a great project even better! 😊 |
Is your feature request related to a problem? Please describe.
When using --extract-links, it would be nice to have an option which only grabbed links from the original domain. I'm also not sure if it is starting to dir bust on other domains that are extracted? The output is unclear.
Describe the solution you'd like
A flag to limit the scope of the tool would be great. Also additional clarity in the ReadMe on if it starts busting new domains when using the --extract-links option would be great.
P.S. - Absolutely loving the tool! I think you've got a real edge on gobuster & ffuf with this one 👍. I've been sharing will all my colleagues! You've done some really great work on this!
The text was updated successfully, but these errors were encountered: