-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] Unable to do any kind of request #47
Comments
A few questions:
As a side note, I'm working on a new version of the library that should hopefully |
Getting the same error with |
@jasonxubw It's unlikely you're getting the exact same error. Please copy the traceback. @jasonxubw @reddere if both of you could set a data dump file in the API constructor and send the dumped data here, that would be very helpful. I also need to know what region you both are in and what operating system you're both using. |
Region: U.S. My code:
Full stack trace:
|
Italy, EU
|
Sure @Russell-Newton , just sent it to your email as I cannot upload json files here. Region is Italy and Windows 10 is the OS |
@jasonxubw your issue seems to be related to a change TikTok made with how challenges are loaded. I think this should be fixed with the v0.2 changes I've been working on. @reddere your issue seems to be caused by TikTok sending you to a login page. I'm not sure what might be causing this, I can't seem to recreate it, but it may be fixed by the v0.2 changes. It could be worth trying out the new stuff: pip install -U git+https://github.com/Russell-Newton/TikTokPy.git@v0.2-overhaul I removed some features, but what you two are doing shouldn't be affected. |
Thanks @Russell-Newton, hope you could find more information with dump file I sent you on the email. Also, I tried updating with the CLI command you sent and an ImportError pops up:
Probably it's because tiktokpy is expecting a precise version of pydantic, as of now pydantic's version is 2.0b3 |
@reddere it actually looks like tiktokapipy didn't update properly because I didn't set the versioning correctly in this branch. Try the install command from above again and make sure that tiktokapipy is set to version 0.2.0a1 afterwards |
Updated and this works, thanks @Russell-Newton! There are some bad news though, all the links provided are accessible except for the ones related to the actual video, which are the links given by "play_addr" and "download_addr", when I make a request on my browser it returns "Access Denied". Also, I tried using tiktokpy to fetch videos from a public user profile with this code
This does not work as it returns the same error given by first issue, what can we do to fix it? I will send your email the dump file for this one |
This has been an issue with video downloading in the past. It's a tricky issue for me to debug because I can't recreate it consistently. See #35 and #44. As for the other issue, this seems to be once again related to TikTok sending you to a login page. I'm not sure why you're the first person who's posted about this issue, but I'll see what I can do to address it. One thing you could try is setting |
Also fingers crossed the changes I just pushed should work. You can try reinstalling the library, and it should be set to version 0.2.0a2 now. The changes attempt to bypass the login page by going immediately to the redirect link specified by the login page. I've no idea if it will actually work, but it's worth a shot. |
I updated the library to version 0.2.0a2 and tried both running with and without Well, at least I gave it a shot. Is there anything I can do to help you figure this out even further? I can screenshare on Discord too (Discord username: @robb0312) @Russell-Newton |
@reddere when you go to TikTok in incognito mode on a normal browser, does it try to automatically take you to a login page when you view an account's page? |
Yep @Russell-Newton ! Tried requesting both with a tiktok video URL and a tiktok profile URL, both redirect me straight to login page. |
That's really interesting. Some followup questions then @reddere:
|
Sure @Russell-Newton
Nope. Tested on both video URL and profile URL. Tried refreshing more times too, still on login page.
Sure, but I cannot upload the htm file here, I just sent the file to your mail.
Sure, here it is: https://www.tiktok.com/login?redirect_url=https%3A%2F%2Fwww.tiktok.com%2F%40silvio.berlusconi&lang=en&enter_method=mandatory The link is generated by the code pasted at #issuecomment-1613360584 |
Maybe helps prevent forced logins
@reddere I just pushed a new change as v0.2.0a3, this time with the goal of attempting to stop the login page redirect. This time, there's also a TikTokAPIWarning that informs you that it got sent to a login page. Because I can't seem to recreate this issue on my end, you'll have to try again, and this time it'll be a bit easier to tell what's going on with that warning. |
@Russell-Newton Cool, I updated to v0.2.0a3, still not able to get the data. If this can help, if you set a VPN to Italy, you should be able to encounter this problem too. On my end, when I set my VPN to Atlanta (US), it does not redirect me to the login page. It just shows me the video. So yeah, even if this seems very odd, you can recreate this by setting a VPN on Italy or to any other country that has this issue. |
I think I know exactly what the issue is. TikTok has had to instate changes and is instating changes in the EU to comply with it's data privacy regulations. Therese regulations are much more strict than their US counterparts. I guess to comply with the mandatory opt-in standard set by the EU and to achieve pseudonymisation, TikTok currently requires all EU users to log in to use their platform. With this in mind, there isn't anything I can do. The only solution is to use a VPN or proxy (proxy information can be passed to the API, see the constructor documentation). At the moment then, some features of TikTokPy won't work 100% within the EU without a VPN or proxy. |
This issue should be resolved in the official version 0.2.0 release. The newly discovered issue of EU library users being unable to access TikTok user information is something that I won't be able to fix unless I can adapt the user collection to also work in line with #21, which is a WIP. I'm closing this issue as resolved. |
Sure. thanks. |
Describe the bug
I am unable to do any kind of request. This error pops up:
tiktokapipy.TikTokAPIError: Data scraping unable to complete in 30.0s (retries: 0)
To Reproduce
Here is the code you can run:
Expected behavior
print out the scraped
Video
objectVersion Information
pydantic==1.10.9
playwright==1.35.0
playwright==1.35.0
tiktokapipy==0.1.13.post1
Please include what versions of pydantic, playwright, and tiktokapipy you have installed (can be found with
pip freeze
).Additional context
Here is the full traceback:
Traceback (most recent call last):
File "C:\Users\path\venv\lib\site-packages\tiktokapipy\api.py", line 430, in _scrape_data
data = self._extract_and_dump_data(content, extras_json, data_model)
File "C:\Users\path\venv\lib\site-packages\tiktokapipy\api.py", line 469, in _extract_and_dump_data
parsed = data_model.parse_raw(data)
File "pydantic\main.py", line 549, in pydantic.main.BaseModel.parse_raw
File "C:\Users\path\venv\lib\site-packages\tiktokapipy\models\raw_data.py", line 130, in parse_obj
return super().parse_obj(obj)
File "pydantic\main.py", line 526, in pydantic.main.BaseModel.parse_obj
File "pydantic\main.py", line 341, in pydantic.main.BaseModel.init
pydantic.error_wrappers.ValidationError: 1 validation error for VideoResponse
VideoPage
field required (type=value_error.missing)
Traceback (most recent call last):
File "c:\Users\path\main.py", line 11, in
video = api.video('https://www.tiktok.com/@ahormozi/video/7242371997324315946')
File "C:\Users\path\venv\lib\site-packages\tiktokapipy\api.py", line 348, in video
response, api_extras = self._scrape_data(
File "C:\Users\path\venv\lib\site-packages\tiktokapipy\api.py", line 445, in _scrape_data
raise TikTokAPIError(
tiktokapipy.TikTokAPIError: Data scraping unable to complete in 30.0s (retries: 0)
The text was updated successfully, but these errors were encountered: