Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reworking favicon hash calculation #906

Merged
merged 7 commits into from
Jan 6, 2023
Merged

Conversation

Mzack9999
Copy link
Member

Description

This PR refactor #842 and simplifies it to perform at most one additional request for favicon calculation.

@Mzack9999 Mzack9999 added Type: Enhancement Most issues will probably ask for additions or changes. Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors. labels Dec 18, 2022
@Mzack9999 Mzack9999 self-assigned this Dec 18, 2022
runner/runner.go Fixed Show fixed Hide fixed
Copy link
Member

@ehsandeep ehsandeep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • path handling fix to request favicon resource
echo https://api.hackerone.com | ./httpx -favicon -v

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   /
 / / / / /_/ /_/ /_/ /   |
/_/ /_/\__/\__/ .___/_/|_|
             /_/              v1.2.5

		projectdiscovery.io

Use with caution. You are responsible for your actions.
Developers assume no liability and are not responsible for any misuse or damage.
[WRN] could not calculate favicon hash: could not fetch favicon: Get "assets/images/favicon.ico": unsupported protocol scheme ""
https://api.hackerone.com
  • disabling automatic redirect for favicon option

runner/runner.go Outdated Show resolved Hide resolved
Copy link
Member

@ehsandeep ehsandeep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Mzack9999 since the favicon path may have more possible paths than /favicon.ico, it would be nice to write the path information in the json output for identification.

When the favicon is identified, favicon_path can be included in json output.

echo https://api.hackerone.com | ./httpx -favicon -json | jq .
  "favicon": "595148549",
+  "favicon_path": "/assets/images/favicon.ico",

@sonarcloud
Copy link

sonarcloud bot commented Jan 6, 2023

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

Copy link
Member

@ehsandeep ehsandeep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

echo https://api.hackerone.com | ./httpx -favicon -json -silent | jq .
{
  "timestamp": "2023-01-06T18:23:29.749197+05:30",
  "csp": {
    "domains": [
      "cdn.jsdelivr.net",
      "*.algolia.net",
      "*.algolianet.com",
      "www.google-analytics.com\n",
      "www.google-analytics.com"
    ]
  },
  "hash": {
    "body_md5": "65782d606fe962c372dd0515e25282c9",
    "body_mmh3": "-1965878003",
    "body_sha256": "c23f6854459a5c324efe5007d30a753a566c9639539e6fc72b5eb62157ada729",
    "body_simhash": "9815579009636610750",
    "header_md5": "f49442f489e10e4db61119b22869bafa",
    "header_mmh3": "556908435",
    "header_sha256": "6967ce655312798fe467bfe4a1634044e829aec12bbd18c75aba0f45d660d737",
    "header_simhash": "15597847997707957536"
  },
  "port": "443",
  "url": "https://api.hackerone.com:443",
  "input": "https://api.hackerone.com",
  "title": "HackerOne API",
  "scheme": "https",
  "webserver": "cloudflare",
  "content_type": "text/html",
  "method": "GET",
  "host": "104.16.99.52",
  "path": "/",
  "favicon": "595148549",
  "favicon_path": "/assets/images/favicon.ico",
  "time": "606.735167ms",
  "a": [
    "104.16.99.52",
    "104.16.100.52",
    "2606:4700::6810:6434",
    "2606:4700::6810:6334"
  ],
  "words": 1642,
  "lines": 360,
  "status_code": 200,
  "content_length": 8486,
  "failed": false
}

@ehsandeep ehsandeep merged commit db1bc2e into dev Jan 6, 2023
@ehsandeep ehsandeep deleted the feat-favicon-improvements branch January 6, 2023 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors. Type: Enhancement Most issues will probably ask for additions or changes.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Favicon path appending issue Improvement in favicon detection and hash calculation
2 participants