-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provide a ScrapingClient that doesn't need API access #5
Draft
pR0Ps
wants to merge
23
commits into
master
Choose a base branch
from
feature/standalone-scraping
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
b2f0204
to
923a1c3
Compare
923a1c3
to
13737f7
Compare
9a82176
to
d8ed33a
Compare
Also adds the ability to list activities using web scraping instead of the API. The activities are returned as `ScrapedActivity` objects that are mostly compatible with the normal `Activity` objects that are returned by the list activities function that uses the API.
This should be done by the library consumer if it's needed
It's not going to be perfect, but the idea is that for the most basic of cases it should be a pretty close replacement. The goal is to keep the amount of work to support both API and scraping-based clients to a minimum. To support this, the WebClient now uses delegation instead of inheritance to add scraper-based functionality. This enables the `ScrapingClient` class to use the same function names without automatically overriding the `stravalib.Client` functions when used through the `WebClient` class.
The default used to be to just download the JSON blob. It was changed to request the GPX format instead since this is a more standardized format for an activity.
Now accepts (but ignores) parameters that the `stravalib` version accepts
- Make pagination actually work (forgot to increment page number) - Handle stopping based on the `before` param - Properly handle workout types
Replaces `get_bike_components`
BeautifulSoup v4.9.0 changed how `.text` works for `<script>` tags (ie. not at all), breaking parsing. See https://bazaar.launchpad.net/~leonardr/beautifulsoup/bs4/revision/564
d8ed33a
to
3685464
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Also adds the ability to list activities using web scraping instead of the API. The activities are returned as
ScrapedActivity
objects that are mostly compatible with the normalActivity
objects that are returned by the list activities function that uses the API.Fixes #4
NOTE:
stravalib
moving to Pydantic for its models is going to break a LOT of this. Will need some work.