Fix Cocktail Party scraper #270
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Generate slug if missing
Add copy, add external classes
Sync only cocktail thumbs
Cocktail Party have changed their website. It now appears to be using something called Elementor. Quite possibly https://elementor.com/. This has caused the old scraper to stop working. This commit fixes the scraper. I've tried it with 192 cocktails from Cocktail Party and, except for a few, it worked fine. The ones that didn't work were using units that recipe-utils doesn't understand, such as "stick" or "half," and thus I assume that they wouldn't've worked with the old scraper as well.
|
Nice, lgtm. Sort is used for sorting of ingredients, in scraping context its usually just autoincrementing from 1 for each ingredient. Thanks! |
|
Ah, I missed the fact that there is a develop branch. Probably should've based my changes on top of it instead of on master, right?
That was basically my understanding of it as well, so that's good.
No worries, thank you! |
zdenek-biberle
added a commit
to zdenek-biberle/bar-assistant
that referenced
this pull request
Apr 16, 2024
This is a bit of a successor to karlomikus#270. This commit adds four improvements to the Cocktail Party scraper: First, some Cocktail Party recipes use units that the recipe-utils parser doesn't understand. For example, the [Manhattan Bianco](https://cocktailpartyapp.com/drinks/manhattan-bianco/) uses a "piece." Such ingredients would've simply been presented to the user without the unit and the user had to fill that in themselves. Now the code will fall back to whatever the parser didn't parse, which is a fairly good default for Cocktail Party. Next, the Cocktail Party website uses "parts" for lots of ingredients, but they actually mean fluid ounces (i.e. the same recipes in their mobile app show up with fluid ounces instead of parts). Thus the scraper now maps parts to fluid ounces. Next, the scraper now reads the links in the "post info" part of the page as tags. The links usually provide categories or names of the cocktail's creator, so this works out nicely. Finally, I've fixed an oversight introduced in karlomikus#270. The code for parsing the cocktail's description goes through all the paragraphs and then joins them up to form proper Markdown paragraphs. However, those paragraphs were then squashed together within the toArray() function in the clean up process. That's obviously undesirable. So now the paragraphs are cleaned up before they're joined together, which produces nice Markdown with multiple paragraphs.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cocktail Party have changed their website. It now appears to be using something called Elementor. Quite possibly https://elementor.com/. This has caused the old scraper to stop working.
This commit fixes the scraper. I've tried it with 192 cocktails from Cocktail Party and, except for a few, it worked fine. The ones that didn't work were using units that recipe-utils doesn't understand, such as "stick" or "half," and thus I assume that they wouldn't've worked with the old scraper as well.
I've been stumped a bit by the change in 2c4c6a4#diff-21fdca419f69f37e6f27b80fc3a35eae8faa9e1920dd2762e8bba5a8d081a891 - I'm not exactly sure what the plan is with the
sortfield, but it seems to me that it has broken scraping. Thus I've changed how theingredientsfield is build inAbstractSiteExtractor- it now includes thesortfield as well. Please let me know if that's okay or not.