Add GitHub code-search discovery, tests, and CI for subdomain_searcher - #1
Add GitHub code-search discovery, tests, and CI for subdomain_searcher#1Mankeluvsit wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5f0554dd71
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| text = " ".join( | ||
| [ | ||
| str(item.get("name", "")), | ||
| str(item.get("path", "")), | ||
| str(item.get("html_url", "")), |
There was a problem hiding this comment.
Extract hostnames from matched code snippets
This loop builds text from metadata fields (name, path, html_url, repo name) rather than matched file content, so for typical code-search hits where the hostname appears only inside the file body, HOSTNAME_RE finds nothing and the GitHub source silently returns near-empty results. To make this feature work as intended, parse matched fragments (for example via text_matches) or fetch file contents before running hostname extraction.
Useful? React with 👍 / 👎.
| ) | ||
| for host in HOSTNAME_RE.findall(text): | ||
| candidate = host.lower().strip(".") | ||
| if candidate.endswith(domain): |
There was a problem hiding this comment.
Enforce label boundary when filtering by domain suffix
Using candidate.endswith(domain) accepts unrelated hostnames that merely share trailing characters (for example, badexample.com when scanning example.com), which can pollute results with false positives whenever such strings are present in search output. The filter should require an actual DNS boundary, e.g. exact match or '.' + domain suffix.
Useful? React with 👍 / 👎.
Motivation
Description
discover_from_github(domain, github_token=None)which queries the GitHub code search API and extracts hostnames using existingHOSTNAME_RE.--github-tokenCLI flag for higher API limits.README.mdusage and method list to include GitHub code search and a tokenized example command using--github-token.tests/test_subdomain_searcher.pycoveringnormalize_domain,discover_from_crtsh,discover_from_homepage,discover_bruteforce,discover_from_github, andload_wordlist..github/workflows/subdomain-searcher-tests.ymlto runpyteston pushes and PRs touching the tool or tests.Testing
pytest -qlocally; all tests passed (6 passed).python3 subdomain_searcher.py example.com --jsonto validate runtime behavior and JSON output; the command completed successfully.pyteststeps on push and pull requests to keep tests enforced.Codex Task