feat: update the scraping variants lesson of the JS2 course to be about JavaScript #1846
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
honzajavorek
added
the
t-academy
|
Preview for this PR was built for commit |
honzajavorek
force-pushed
the
honzajavorek/js2-scraping-variants
branch
from
a083a57 to
0d22cf5
Compare
|
Preview for this PR was built for commit |
|
Thanks! I noticed there was a conflict with master now, so I resolved it. I'll wait for @gullmar to check code before merging. |
|
@cursor review |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
@cursor review |
|
Preview for this PR was built for commit |
gullmar
approved these changes
Sep 3, 2025
sources/academy/webscraping/scraping_basics_javascript2/11_scraping_variants.md
Outdated
Show resolved
Hide resolved
| item = parse_product(product, listing_url) | ||
| product_soup = download(item["url"]) | ||
| vendor = product_soup.select_one(".product-meta__vendor").text.strip() | ||
| const $promises = $(".product-item").map(async (i, element) => { |
There was a problem hiding this comment.
Suggested change
| const $promises = $(".product-item").map(async (i, element) => { | |
| const promises = $(".product-item").toArray().map(async (element) => { |
|
|
||
| return item; | ||
| }); | ||
| const data = await Promise.all($promises.get()); |
There was a problem hiding this comment.
Suggested change
| const data = await Promise.all($promises.get()); | |
| const data = await Promise.all(promises); |
| const $ = await download(listingURL); | ||
|
|
||
| Python dictionaries are mutable, so if we assigned the variant with `item["variant_name"] = ...`, we'd always overwrite the values. Instead of saving an item for each variant, we'd end up with the last variant repeated several times. To avoid this, we create a new dictionary for each variant and merge it with the `item` data before adding it to `data`. If we don't find any variants, we add the `item` as is, leaving the `variant_name` key empty. | ||
| const $promises = $(".product-item").map(async (i, element) => { |
There was a problem hiding this comment.
Suggested change
| const $promises = $(".product-item").map(async (i, element) => { | |
| const promises = $(".product-item").toArray().map(async (element) => { |
| // highlight-end | ||
| }); | ||
| // highlight-start | ||
| const itemLists = await Promise.all($promises.get()); |
There was a problem hiding this comment.
Suggested change
| const itemLists = await Promise.all($promises.get()); | |
| const itemLists = await Promise.all(promises); |
| } | ||
| return [{ variantName: null, ...item }]; | ||
| }); | ||
| const itemLists = await Promise.all($promises.get()); |
There was a problem hiding this comment.
Suggested change
| const itemLists = await Promise.all($promises.get()); | |
| const itemLists = await Promise.all(promises); |
| const listingURL = "https://www.npmjs.com/search?page=0&q=keywords%3Allm&sortBy=dependent_count"; | ||
| const $ = await download(listingURL); | ||
|
|
||
| const $promises = $("section").map(async (i, element) => { |
There was a problem hiding this comment.
Suggested change
| const $promises = $("section").map(async (i, element) => { | |
| const promises = $("section").toArray().map(async (element) => { |
| return { name, url, description, dependents, downloads }; | ||
| }); | ||
|
|
||
| const data = await Promise.all($promises.get()); |
There was a problem hiding this comment.
Suggested change
| const data = await Promise.all($promises.get()); | |
| const data = await Promise.all(promises); |
| const listingURL = "https://edition.cnn.com/sport"; | ||
| const $ = await download(listingURL); | ||
|
|
||
| const $promises = $(".layout__main .card").map(async (i, element) => { |
There was a problem hiding this comment.
Suggested change
| const $promises = $(".layout__main .card").map(async (i, element) => { | |
| const promises = $(".layout__main .card").toArray().map(async (element) => { |
| return { url: articleURL, length: content.length }; | ||
| }); | ||
|
|
||
| const data = await Promise.all($promises.get()); |
There was a problem hiding this comment.
Suggested change
| const data = await Promise.all($promises.get()); | |
| const data = await Promise.all(promises); |
Co-authored-by: gullmar <gullmar@mailbox.org>
|
Preview for this PR was built for commit |
Thank you! 🙇♂️ |
daveomri
pushed a commit
to daveomri/apify-docs
that referenced
this pull request
Sep 3, 2025
…ut JavaScript (apify#1846) Part of apify#1584 ----⚠️ 🐍 This PR contains also a small change to the Python course, to keep the lessons consistent and synced. --------- Co-authored-by: gullmar <gullmar@mailbox.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Part of #1584