[MRG+1] docs: update overview spider code to use toscrape.com and minor changes #2249
Conversation
So, this will replace the spider example code from the overview that scrapes questions from StackOverflow by a spider scraping quotes (much like the one in the tutorial), and upates the text around it to be consistent. There are also minor wording changes plus a small Sphinx/reST syntax fix on the features list at the bottom (it was creating a definition list, causing one line to be bold).
docs/intro/overview.rst
Outdated
| of them as they finish. | ||
| an argument. In the ``parse`` callback we loop through the quote elements | ||
| using a CSS Selector, yield a Python dict with the extracted quote text and author, | ||
| look for a link to the next page and schedules another request using the same |
stummjr
Sep 15, 2016
Member
and schedules another request using the same: schedules should be in 1st person of plural.
and schedules another request using the same: schedules should be in 1st person of plural.
eliasdorneles
Sep 15, 2016
Author
Member
fixed!
fixed!
Current coverage is 83.36% (diff: 100%)
|
|
Nice! |
docs/intro/overview.rst
Outdated
| 'link': response.url, | ||
| } | ||
| next_page = response.css('li.next a::attr("href")').extract_first() | ||
| if next_page: |
kmike
Sep 15, 2016
Member
I think next_page is None could be slightly better because an empty string is a valid relative URL
I think next_page is None could be slightly better because an empty string is a valid relative URL
eliasdorneles
Sep 15, 2016
Author
Member
fixed!
fixed!
redapple
changed the title
|
hey @kmike, I'll merge this to move things forward, feel free to point out if you have any other concern. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
So, this will replace the spider example code from the overview that
scrapes questions from StackOverflow by a spider scraping quotes (much
like the one in the tutorial), and upates the text around it to be
consistent.
There are also minor wording changes plus a small Sphinx/reST syntax fix
on the features list at the bottom (it was creating a definition list,
causing one line to be bold).
Does this look good?
Thanks!