[Bug] Headless crawler fails to crawl links from navigation dropdowns

[evanrolfe]

The Katana headless crawler fails to crawl pages which are linked to from navigation dropdowns. I have done a demo in this repo including steps to reproduce:

https://github.com/evanrolfe/katana_bug

Example of such links:

What I expect to happen

I expect to see every page from the example server visited by Katana.

What actually happens

Katana logs the URLs scraped, which includes the /hidden/* paths, but it never actually visits them. I know this because the logs from the example server show that:

go run .
2026/06/05 13:48:02 nested_navigation server listening on http://localhost:8888
2026/06/05 13:48:11 [VISITED] /
2026/06/05 13:48:16 [VISITED] /
2026/06/05 13:48:19 [VISITED] /top/page-a
2026/06/05 13:48:22 [VISITED] /top/page-b
2026/06/05 13:48:25 [VISITED] /top/page-c

Here is the output from Katana:

$ katana -u http://localhost:8888/ -headless

   __        __
  / /_____ _/ /____ ____  ___ _
 /  '_/ _  / __/ _  / _ \/ _  /
/_/\_\\_,_/\__/\_,_/_//_/\_,_/

                projectdiscovery.io

[INF] Current katana version v1.6.1 (latest)
http://localhost:8888/top/page-a
http://localhost:8888/top/page-b
http://localhost:8888/top/page-c
http://localhost:8888/hidden/alpha
http://localhost:8888/hidden/beta
http://localhost:8888/hidden/gamma
http://localhost:8888/hidden/delta
http://localhost:8888/hidden/epsilon
http://localhost:8888/
http://localhost:8888/api/tracker?page=%2F
http://localhost:8888/api/tracker?page=%2Ftop%2Fpage-a
http://localhost:8888/api/tracker?page=%2Ftop%2Fpage-b
http://localhost:8888/api/tracker?page=%2Ftop%2Fpage-c