-
-
Notifications
You must be signed in to change notification settings - Fork 960
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Getting more video ids than 30 fails #1119
Comments
hey, try to look through this code |
same here, more than 35 videos fail |
I've had mixed luck with trying to scrape large amounts at once. I'll post a modified version of your code adding the changes I've made that seem to work best, though I see a strong positive correlation between failure rate and post count (especially for profiles with >1000 posts). Codefrom TikTokApi import TikTokApi
import asyncio
import os
ms_token = os.environ.get(
"ms_token", None
)
async def user_example(username):
async with TikTokApi() as api:
await api.create_sessions(headless=False, ms_tokens=[ms_token], num_sessions=1, sleep_after=3)
user = api.user(username)
user_data = await user.info()
post_count = user_data["userInfo"]["stats"].get("videoCount")
async for video in user.videos(count=post_count):
print(video)
video = str(video) + "\n"
with open('test1.json', 'a') as file:
file.write(video)
if __name__ == "__main__":
asyncio.run(user_example("truong_se")) BreakdownFirst, I added a Hope this is helpful! |
My goal is to get all the user's video ids, I used my own method, surprisingly it works perfectly, it doesn't miss a single video. This is my code:
|
When I set the count to 35 or higher, the same error occurs. I've encountered a limitation where I cannot retrieve information for more than 35 videos at once. However, I found a workaround using the This argument specifies the starting point for the count. By updating I hope this solution is helpful to you! |
Hi @koonn , I tried to use
Have you tried using it? Then can I see your code please? |
here is my code.
I actually use this solution for |
@koonn
Btw thanks for your solution. Have a nice day! |
Which version of TikTokApi are you using? Installing v6.2.2 broke user.videos() for me. Downgrading to 6.2.0 with |
I personally downgraded and I could fetch more than 30+ videos per account without issues! EDIT: wording |
cursor: 0 ms_token = os.environ.get("ms_token", None) # Set your own ms_token async def get_user_videos(user_name, num_data=1):
def run_async_function(async_func, *args, **kwargs): Running the async functionif name == "main": |
Do this The recent version makes you use an older version of chromium and you would need to pass it in as a parameter to the "create_session" method. For me personally, some accounts were just simply not working or i got a return code of "shadow banned". Using version 6.2.0 did not give me any errors on the accounts that were supposedly "shadow banned". EDIT: wording |
Why we are passing cursor 31 etc
Cursor is something long integer string
…On Wed, Jun 5, 2024 at 6:05 PM Gereks123 ***@***.***> wrote:
cursor: 0 TikTokApi.video(id='7364804047850884398')
TikTokApi.video(id='7374084684180901163')
TikTokApi.video(id='7370409071545044267')
TikTokApi.video(id='7369999192237935914')
TikTokApi.video(id='7369630891267738922')
TikTokApi.video(id='7367034743424142634')
TikTokApi.video(id='7366660218496830762')
TikTokApi.video(id='7361095320073211178')
TikTokApi.video(id='7360015406045711662')
TikTokApi.video(id='7356640627938954539')
TikTokApi.video(id='7353301578083863839')
TikTokApi.video(id='7343713119836949791')
TikTokApi.video(id='7341070239158799646')
TikTokApi.video(id='7340345650963434782')
TikTokApi.video(id='7336647877168614686')
TikTokApi.video(id='7333029200418426143')
TikTokApi.video(id='7312884877811109166')
TikTokApi.video(id='7309179805361179950')
TikTokApi.video(id='7307330647427665195')
TikTokApi.video(id='7298050264441818410')
TikTokApi.video(id='7296168512408603947')
TikTokApi.video(id='7292088806361107755')
TikTokApi.video(id='7291000957708602666')
TikTokApi.video(id='7289861757915417899')
TikTokApi.video(id='7289861757915417899')
TikTokApi.video(id='7289125744494759214')
TikTokApi.video(id='7285778438940560686')
TikTokApi.video(id='7280583185115647263')
TikTokApi.video(id='7277986355668438314')
TikTokApi.video(id='7270193233441885486')
TikTokApi.video(id='7244634709748174126')
TikTokApi.video(id='7219372552878116139') cursor: 30 cursor: 60 cursor: 90
cursor: 120 help me import os import asyncio from TikTokApi import TikTokApi
ms_token = os.environ.get("ms_token", None) # Set your own ms_token
async def get_user_videos(user_name, num_data=1): result = [] cursor = 0
async with TikTokApi() as api:
await api.create_sessions(ms_tokens=[ms_token], num_sessions=1, sleep_after=3, headless=False)
user = api.user(username=user_name)
while cursor <= num_data:
print(f"cursor: {cursor}")
async for video in user.videos(count=30, cursor=cursor):
print(video)
video_data = video.as_dict
result.append(video_data)
cursor += 30
return result
def run_async_function(async_func, *args, **kwargs): loop =
asyncio.get_event_loop() if loop.is_running(): return
asyncio.ensure_future(async_func(*args, **kwargs)) else: return
loop.run_until_complete(async_func(*args, **kwargs))
Running the async function
if *name* == "*main*": user_name = "mrbeast" videos =
run_async_function(get_user_videos, user_name, 19000) print(videos) its not
giving pagination videos even cursor is wrong
Do this pip install TikTokApi==6.2.0 --force-reinstall. This seems to be
the most stable solution so far.
The recent version makes you use an older verision of chromium and you
would need to pass it in as a parameter to the "create_session" method. For
me personally, some accounts were just simply not working or i got a return
code of "shadow banned".
Using bersion 6.2.0 did not give me any errors on the accounts that were
supposedly "shadow banned".
—
Reply to this email directly, view it on GitHub
<#1119 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABPI5XPFEUNM7SRK53P65S3ZF4EJ5AVCNFSM6AAAAABDXVHIOWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBZHAZDSOJTGQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
I can confirm pagination is not working in versions 6.3.0 or 6.2.0 for api.hashtag. Looking at |
my 2 cents: the api is kinda badly designed. if you went with the async iterator route then the iterator should hold state (i.e. the cursor) and not expose it to the user. only have the batch size as a parameter. |
Agreed, having |
Hi everyone,
I want to get all tiktok video id from a user, this is code:
in this line, I try edit count=30 to count=1000 but it doesn't work:
output:
2024-02-24 09:39:55,961 - TikTokApi.tiktok - ERROR - Got an unexpected status code: {'log_pb': {'impr_id': '202402240240321EB88FFC04941C07B40E'}, 'statusCode': 10201, 'statusMsg': '', 'status_code': 10201, 'status_msg': ''}
Is there any solution to fix this situation? Thanks.
The text was updated successfully, but these errors were encountered: