`TikTok blocks this request displaying a Captcha` even with `custom_verifyFp` and `proxy` #787

kostyakoz · 2022-01-03T10:01:53Z

Everything was fine until last few days.

from TikTokApi import TikTokApi
import json
import logging
tiktok_api = TikTokApi.get_instance(request_delay=1)

tiktoks = tiktok_api.by_username('japanontiktok', count=1, custom_verifyFp="...", proxy='http://...', use_test_endpoints=True)

for tiktok in tiktoks[:1]:
    print(json.dumps(tiktok))

Error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/TikTokApi/tiktok.py", line 1333, in get_user
    j_raw = self.__extract_tag_contents(r.text)
  File "/usr/local/lib/python3.8/dist-packages/TikTokApi/tiktok.py", line 1553, in __extract_tag_contents
    nonce = html.split(nonce_start)[1].split(nonce_end)[0]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tests.py", line 220, in <module>
    tiktoks = tiktok_api.by_username('japanontiktok', count=1, custom_verifyFp="...", proxy='http://...', use_test_endpoints=True)
  File "/usr/local/lib/python3.8/dist-packages/TikTokApi/tiktok.py", line 700, in by_username
    data = self.get_user_object(username, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/TikTokApi/tiktok.py", line 1301, in get_user_object
    return self.get_user(username, **kwargs)["userInfo"]["user"]
  File "/usr/local/lib/python3.8/dist-packages/TikTokApi/tiktok.py", line 1339, in get_user
    raise TikTokCaptchaError()
TikTokApi.exceptions.TikTokCaptchaError: TikTok blocks this request displaying a Captcha 
Tip: Consider using a proxy or a custom_verifyFp as method parameters

The most interesting thing there is that I can partly see obtained data from TikTok because its print HTML page in console before the error. You can see full response here: https://gist.githubusercontent.com/kostyakoz/1d5a7339217d646f1556d9fd63bcf7a8/raw/b18f4ae5e869325bb746c48fae102c8786b43fdb/error.py

My question is why obtained data not returned and I have only error?

botcoder-sys · 2022-01-03T12:36:05Z

I am facing the same problem as well. It was working fine with custom_verifyFp earlier. But, I think a week ago, it starts breaking. Today, I run the by.username() in loop until it succeeded with custom_verifyFp and it worked once after each 20-30 tries. Sometimes changing proxy also helps but I think there is some issue with API. I have the exact same error as @kostyakoz Showed above.

angelopouloschristos · 2022-01-03T14:31:31Z

there is the same issue but in the "closed" section because someone closed it without a solution
We are still waiting for a solution a lot of poeple have the same issue and usualy it returns a HTML response that correspond to the use profil page and it does not download anything. The captcha is not the problem here. They made something with their API

joseantgv · 2022-01-03T15:33:38Z

Everything was fine until last few days.

from TikTokApi import TikTokApi
import json
import logging
tiktok_api = TikTokApi.get_instance(request_delay=1)

tiktoks = tiktok_api.by_username('japanontiktok', count=1, custom_verifyFp="...", proxy='http://...', use_test_endpoints=True)

for tiktok in tiktoks[:1]:
    print(json.dumps(tiktok))

Error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/TikTokApi/tiktok.py", line 1333, in get_user
    j_raw = self.__extract_tag_contents(r.text)
  File "/usr/local/lib/python3.8/dist-packages/TikTokApi/tiktok.py", line 1553, in __extract_tag_contents
    nonce = html.split(nonce_start)[1].split(nonce_end)[0]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tests.py", line 220, in <module>
    tiktoks = tiktok_api.by_username('japanontiktok', count=1, custom_verifyFp="...", proxy='http://...', use_test_endpoints=True)
  File "/usr/local/lib/python3.8/dist-packages/TikTokApi/tiktok.py", line 700, in by_username
    data = self.get_user_object(username, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/TikTokApi/tiktok.py", line 1301, in get_user_object
    return self.get_user(username, **kwargs)["userInfo"]["user"]
  File "/usr/local/lib/python3.8/dist-packages/TikTokApi/tiktok.py", line 1339, in get_user
    raise TikTokCaptchaError()
TikTokApi.exceptions.TikTokCaptchaError: TikTok blocks this request displaying a Captcha 
Tip: Consider using a proxy or a custom_verifyFp as method parameters

The most interesting thing there is that I can partly see obtained data from TikTok because its print HTML page in console before the error. You can see full response here: https://gist.githubusercontent.com/kostyakoz/1d5a7339217d646f1556d9fd63bcf7a8/raw/b18f4ae5e869325bb746c48fae102c8786b43fdb/error.py

My question is why obtained data not returned and I have only error?

If you save the response as HTML and open it in a browser, you will see that there's information about the profile. I think that it's a parse problem, maybe TikTok has changed the structure.

If you search for SIGI_STATE you will see a JSON object which already contains all the information.

It's used this function to get the data:

def __extract_tag_contents(self, html):
    nonce_start = '<head nonce="'
    nonce_end = '">'
    nonce = html.split(nonce_start)[1].split(nonce_end)[0]
    j_raw = html.split(
        '<script id="__NEXT_DATA__" type="application/json" nonce="%s" crossorigin="anonymous">'
        % nonce
    )[1].split("</script>")[0]
    return j_raw

and can't find <script id="__NEXT_DATA__" type="application/json" nonce="%s" crossorigin="anonymous">' in the response.

I think that this could be the clue: https://git.raptorpond.com/henine/yt-dlp/commit/11aa91a12f95821500fa064402a3e2c046b072fb

Daan-Grashoff · 2022-01-04T19:54:05Z

I've made a quick dirty fix for the meantime, works a lot better since they moved from next to sigi.

https://github.com/Daan-Grashoff/TikTok-Api

You can simply install it by using pip git+https://github.com/Daan-Grashoff/TikTok-Api.

Thanks @joseantgv for pointing out where it goes wrong!

ju · 2022-01-06T11:28:14Z

I've made a quick dirty fix for the meantime, works a lot better since they moved from next to sigi.

https://github.com/Daan-Grashoff/TikTok-Api

You can simply install it by using pip git+https://github.com/Daan-Grashoff/TikTok-Api.

Thanks @joseantgv for pointing out where it goes wrong!

Thanks for the fix! Works flawlessly, but would it be possible to return the Likes, Followers & Following count for a user?

ghost · 2022-01-06T14:31:37Z

Thanks, but it isn't working for me. I only get 'UserModule' as error output when I run
user = api.by_username(username=username).

botcoder-sys · 2022-01-06T16:15:57Z

Thanks, but it isn't working for me. I only get 'UserModule' as error output when I run user = api.by_username(username=username).

Have you tried remove the API and then installing this one. It should work

ghost · 2022-01-06T16:26:41Z

Yes, I ran python -m pip uninstall TikTokApi, changed into the new Api folder and ran python setup.py install

akorchyn · 2022-01-08T12:51:18Z

I've made a quick dirty fix for the meantime, works a lot better since they moved from next to sigi.

https://github.com/Daan-Grashoff/TikTok-Api

You can simply install it by using pip git+https://github.com/Daan-Grashoff/TikTok-Api.

Thanks @joseantgv for pointing out where it goes wrong!

Thank you. It works.

ghost · 2022-01-08T13:30:06Z

Thank you. It works.

How did you install it? I'm still getting the error.

akorchyn · 2022-01-08T14:21:17Z

I just replaced TktokApi with
git+https://github.com/Daan-Grashoff/TikTok-Api
in my requirements.txt file. Afterward, I rebuild my docker image and restarted instance.

ghost · 2022-01-08T15:15:04Z

Ok. I think that @Daan-Grashoff didn't fix the method by_username. The get_user method partly works.

akorchyn · 2022-01-08T15:26:37Z

I'm using it. It works. Do you use proxy ? The method is not working after some time. TikTok bans ip after some scraping.

ghost · 2022-01-08T16:02:28Z

No, I tried it on my home ip. I just found out that the error appears, when the username doesn't exist anymore. For now it is working again. Thx.

akorchyn · 2022-01-08T16:03:38Z

Great to hear

sedanijay · 2022-01-15T13:45:42Z

@joseantgv
https://github.com/Daan-Grashoff/TikTok-Api this is working but giving an error for count 1000
or in case of
api = TikTokApi.get_instance()
tiktoks = api.by_username("ford", count=1000)

error:

playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded.
=========================== logs ===========================
navigating to "https://www.tiktok.com/@redbull", waiting until "load"
============================================================
Note: use DEBUG=pw:api environment variable to capture Playwright logs.

@Daan-Grashoff

Initial changes for v5 of the project. Thanks @Daan-Grashoff for some of the helpers.extract_tag_contents code from a comment on #787 Co-Authored-By: Daan Grashoff <9222025+Daan-Grashoff@users.noreply.github.com>

Doritos464 · 2022-02-08T21:26:23Z

hello, I still have the same issue and tried almost everything and nothing works.
Any suggestions?

from TikTokApi import TikTokApi
fp = "<myfp>"
api = TikTokApi.get_instance(use_test_endpoints=True)
users = api.by_username(username="davidspam464", custom_verifyFp=fp)

The error is:
TikTokApi.exceptions.TikTokCaptchaError: TikTok blocks this request displaying a Captcha Tip: Consider using a proxy or a custom_verifyFp as method parameters

@Daan-Grashoff

* Initial changes for v5 of the project. * Thanks @Daan-Grashoff for some of the helpers.extract_tag_contents code from a comment on #787 * High Level Modeling of Classes * update tests & auto call object * Fix package-test * simplify info_full * Logging Improvements * Logging changes from #804 by @zokalo * Improve docs * Fix some mypy typing issues * Move old search functions to trending & fix process_kwargs * Private self variables Co-authored-by: davidteather <davidteather@users.noreply.github.com> Co-authored-by: Daan Grashoff <9222025+Daan-Grashoff@users.noreply.github.com> Co-authored-by: Dmitriy <16061619+zokalo@users.noreply.github.com>

davidteather · 2022-02-14T02:46:49Z

Please try using V5.0.0 this may solve your problem

zgadev · 2022-02-15T14:31:06Z

Daan-Grashoff's fix was working perfect until today.. 5.0 is getting "Timeout 30000ms exceeded."

zakrian07 · 2023-12-20T11:00:22Z

async def fetch_tiktok_data(video_url: str) -> dict:
"""Fetch TikTok video data."""
async with TikTokApi() as api:

    await api.create_sessions(proxy=[proxy_config],ms_tokens=[ms_token], num_sessions=1, sleep_after=3)

    video = api.video(proxy=proxy,url=video_url)
    data = await video.info()
    return data

if name == "main":
app.run(debug=True) do we need proxy while create session or while fetch video info ? please updat me best practices when i use proxy config it not working for create session

kostyakoz added the bug Something isn't working label Jan 3, 2022

walterjesus88 mentioned this issue Jan 20, 2022

[BUG] - does not work "by_username()" #798

Closed

davidteather closed this as completed Feb 14, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`TikTok blocks this request displaying a Captcha` even with `custom_verifyFp` and `proxy` #787

`TikTok blocks this request displaying a Captcha` even with `custom_verifyFp` and `proxy` #787

kostyakoz commented Jan 3, 2022 •

edited

Loading

botcoder-sys commented Jan 3, 2022

angelopouloschristos commented Jan 3, 2022

joseantgv commented Jan 3, 2022 •

edited

Loading

Daan-Grashoff commented Jan 4, 2022

ju commented Jan 6, 2022

ghost commented Jan 6, 2022 •

edited by ghost

Loading

botcoder-sys commented Jan 6, 2022

ghost commented Jan 6, 2022

akorchyn commented Jan 8, 2022

ghost commented Jan 8, 2022 •

edited by ghost

Loading

akorchyn commented Jan 8, 2022

ghost commented Jan 8, 2022

akorchyn commented Jan 8, 2022

ghost commented Jan 8, 2022

akorchyn commented Jan 8, 2022

sedanijay commented Jan 15, 2022 •

edited

Loading

Doritos464 commented Feb 8, 2022

davidteather commented Feb 14, 2022

zgadev commented Feb 15, 2022

zakrian07 commented Dec 20, 2023

TikTok blocks this request displaying a Captcha even with custom_verifyFp and proxy #787

TikTok blocks this request displaying a Captcha even with custom_verifyFp and proxy #787

Comments

kostyakoz commented Jan 3, 2022 • edited Loading

botcoder-sys commented Jan 3, 2022

angelopouloschristos commented Jan 3, 2022

joseantgv commented Jan 3, 2022 • edited Loading

Daan-Grashoff commented Jan 4, 2022

ju commented Jan 6, 2022

ghost commented Jan 6, 2022 • edited by ghost Loading

botcoder-sys commented Jan 6, 2022

ghost commented Jan 6, 2022

akorchyn commented Jan 8, 2022

ghost commented Jan 8, 2022 • edited by ghost Loading

akorchyn commented Jan 8, 2022

ghost commented Jan 8, 2022

akorchyn commented Jan 8, 2022

ghost commented Jan 8, 2022

akorchyn commented Jan 8, 2022

sedanijay commented Jan 15, 2022 • edited Loading

Doritos464 commented Feb 8, 2022

davidteather commented Feb 14, 2022

zgadev commented Feb 15, 2022

zakrian07 commented Dec 20, 2023

`TikTok blocks this request displaying a Captcha` even with `custom_verifyFp` and `proxy` #787

`TikTok blocks this request displaying a Captcha` even with `custom_verifyFp` and `proxy` #787

kostyakoz commented Jan 3, 2022 •

edited

Loading

joseantgv commented Jan 3, 2022 •

edited

Loading

ghost commented Jan 6, 2022 •

edited by ghost

Loading

ghost commented Jan 8, 2022 •

edited by ghost

Loading

sedanijay commented Jan 15, 2022 •

edited

Loading