Skip to content
This repository has been archived by the owner on Jun 25, 2021. It is now read-only.

Optimalizace Web-Scrapping robota #11

Open
thonzyk opened this issue Apr 15, 2018 · 0 comments
Open

Optimalizace Web-Scrapping robota #11

thonzyk opened this issue Apr 15, 2018 · 0 comments
Assignees
@thonzyk
Labels
Priority: Low Do not hurry with this one Type: Performance Improving code efficiency
Milestone
Web scraping

Comments

@thonzyk
Copy link
Collaborator

thonzyk commented Apr 15, 2018

1) optimalizovat metodu scrap_text()
- zabírá cca 1/2 celkového času
2) paralelizovat načítání HTML od dalšího zpracování
- zřejmě zvlášť "sypat" raw HTML do fronty a paralelně je zpracovávat
- důvodem je většinou rychlejší načtení HTML, než další zpracování
- zpracování nebude muset pokaždé čekat
@thonzyk thonzyk added the Priority: Low Do not hurry with this one label Apr 15, 2018
@thonzyk thonzyk added this to the Web scraping milestone Apr 15, 2018
@thonzyk thonzyk self-assigned this Apr 15, 2018
@thonzyk thonzyk added the Type: Performance Improving code efficiency label Apr 15, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@thonzyk thonzyk

Labels
Priority: Low Do not hurry with this one Type: Performance Improving code efficiency
Projects
None yet
Milestone
Web scraping
Development

No branches or pull requests
1 participant
@thonzyk