Skip to content

Technology detection failures #67

Open
@max-ostapenko

Description

@max-ostapenko

During the crawl no technology detected for 2% out of top 10000 pages.
Running detection on my laptop is successful.

SELECT
  page,
  rank
FROM `httparchive.all.pages`
WHERE date = "2024-09-01"
  AND is_root_page
  AND client = 'mobile'
  AND rank <= 10000
  AND ARRAY_LENGTH(technologies)=0
ORDER BY rank, page

Tried to retest it on WPT:

WPT testing lasted really long (10-20min) and nothing detected again. Pages loaded slow too (up to 100s).
Seems like a performance issue.

@pmeenan is it possible to investigate these?

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions