-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
when time_list == 'N/A\r', it would break #10
Comments
Hi eweca1992, Thanks for pointing out this issue! I tried to set "time_list = 'NAr'" in "def time_str2float", and got "ValueError: invalid literal for int() with base 10" (I would like to suggest you check the name of the error you have met, here is "ValueError"). Thus, I modified this function by adding "try" and "except" operations as below:
You could replace "ValueError" in "except ValueError" with the name of error you have met. I hope this could help you to fix your issue. :P Regards, |
Thank you for your reply. I would try it when my computer is free. By the way, is it possible to auto re-connect weibo after request error? It seems that a "try...except" can do the work. |
Hi, Yes! First of all, the code will try at most 3 times to crawl the data of one Weibo post (3 "try-except" code in "def crawl_***") if "urllib.request.urlretrieve" function failed to download the data. If the code still cannot crawl the data after 3 times attempts, the data will not be saved. (I have never met this problem so far.) Second, if you get the error: "ConnectionError 104: ('Connection aborted.')" (the most common one which is caused by the restriction of Sina Weibo. It is different with the first problem), the code will stop! In this case, you could follow the information described in the 8-th procedure of User Settings of Usage in README and manually re-start the code to continue to crawl data. Currently, the crawler cannot automatically do this. I am sorry. XD Regards, |
Thank you for your patient reply. |
Sometimes, the value of time_list could be ['N/A\r'] (def time_str2float). It may mean the video missed. Then, it would break.
My temporary solution is that set 'time_num = 0', if the 'N/A\r' appears. However, I do not know if it would cause unknown flaws.
The text was updated successfully, but these errors were encountered: