Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add onboard hash / classic ML classifier for standard error-pages/home-pages #1201

Closed
Mzack9999 opened this issue May 30, 2023 · 0 comments · Fixed by #1245
Closed

Add onboard hash / classic ML classifier for standard error-pages/home-pages #1201

Mzack9999 opened this issue May 30, 2023 · 0 comments · Fixed by #1245
Assignees
@dogancanbakir
Labels
Investigation Something to Investigate Status: Completed Nothing further to be done with this issue. Awaiting to be closed. Type: Enhancement Most issues will probably ask for additions or changes.
Milestone
httpx v1.3.4

Comments

@Mzack9999
Copy link
Member

Please describe your feature request:

httpx should be able to add within the output heuristic guess if the page is an error-page/home-page of the most common web frameworks. This requires some investigation. Some potential approaches are the following:

  • Perceptual hash of error pages with threshold
  • simhash comparison with pre-calculated ones from most common web apps
  • Bayesian classifier
  • SVM

Some analysis might be performed, taking advantage of the screenshot taken if the headless modality is used.
@Mzack9999 Mzack9999 added Type: Enhancement Most issues will probably ask for additions or changes. Investigation Something to Investigate labels May 30, 2023
@dogancanbakir dogancanbakir mentioned this issue Jun 20, 2023
Merged
@dogancanbakir dogancanbakir linked a pull request Jun 20, 2023 that will close this issue
Error page classifier #1245
Merged
@Mzack9999 Mzack9999 added the Status: Completed Nothing further to be done with this issue. Awaiting to be closed. label Jul 19, 2023
@ehsandeep ehsandeep added this to the httpx v1.3.4 milestone Jul 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dogancanbakir dogancanbakir

Labels
Investigation Something to Investigate Status: Completed Nothing further to be done with this issue. Awaiting to be closed. Type: Enhancement Most issues will probably ask for additions or changes.
Projects
None yet
Milestone
httpx v1.3.4
Development

Successfully merging a pull request may close this issue.

Error page classifier dogancanbakir/httpx
3 participants
@ehsandeep @Mzack9999 @dogancanbakir