Skip to content

fix(scrapper): manage audio and video requests#58

Merged
vvatelot merged 6 commits intomainfrom
fix/scrapper/read-mp3-url
Feb 28, 2024
Merged

fix(scrapper): manage audio and video requests#58
vvatelot merged 6 commits intomainfrom
fix/scrapper/read-mp3-url

Conversation

@PaulPHPE
Copy link
Copy Markdown
Collaborator

Modify get_requests_from_har_file method to take into account mp3 and audio requests.

Basically manage cases when _transferSize in the HTTP request is equal to -1.

close #30

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 21, 2024

Coverage PR 58

Branch coverage •
FileStmtsMissCoverMissing
components/ecoindex/scraper
   scrap.py955146%45, 48–49, 51, 60, 63, 66–68, 73–75, 77–81, 84–87, 89, 91, 98–101, 108–110, 112–121, 130–131, 134–135, 137, 152–153, 158–161
TOTAL69923067% 

@PaulPHPE PaulPHPE self-assigned this Feb 21, 2024
Comment thread components/ecoindex/scraper/scrap.py Outdated
@vvatelot
Copy link
Copy Markdown
Member

I am reading the #30 issue again... I think I was not completely clear 🙂 : In this case as it is a mp3 file, the analysis should return an error: We only accept to analyse html content. This is the behaviour we had in the old scraper and I did not implement it yet...

So, here I think we have 2 issues:

  • The first one is the one regarding partial responses that your PR covers
  • The second one is about the type of content that we analyse

What do you think ?

@PaulPHPE
Copy link
Copy Markdown
Collaborator Author

Hello @vvatelot !

Ah alright ! I will modify the code to raise an error when the url doesn't point to a text/html content type.

@vvatelot vvatelot merged commit 771ad9b into main Feb 28, 2024
@vvatelot vvatelot deleted the fix/scrapper/read-mp3-url branch February 28, 2024 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Error with mp3

2 participants