Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New strange behaviour in tags or tags list mode #143

Closed
TheDoc69 opened this Issue Aug 4, 2016 · 8 comments

Comments

Projects
None yet
3 participants
@TheDoc69
Copy link

TheDoc69 commented Aug 4, 2016

Hi !
I mainly use PixivUtil in tags list mode (mode=7), and everything was working well recently at least until August 1st. Today I used exactly the same tags (tags list, partial mode) and this time some tags are working but other are not.
Pixivutil return no error but just says "No more images"
Nothing interestesting is reported in the log.
An exemple of this is the tag ( けもの AND R-18 ) which is working (strange because I normally don't put parentheses) but けもの AND R-18 or ( ケモ AND R-18 ) or even ケモ do not work at all in mode 3 or 7.
A strange thing is that the link generated by pixivutil for the web page (ex: http://www.pixiv.net/search.php?s_mode=s_tag&p=1&word=%E3%82%B1%E3%83%A2&order=date_d) works perfectly when inserted in a web browser like firefox.

Here is the log for example :

2016-08-04 16:33:45,825 - PixivUtil20160726 - INFO - ###############################################################
2016-08-04 16:33:45,825 - PixivUtil20160726 - INFO - Starting...
2016-08-04 16:33:45,842 - PixivUtil20160726 - INFO - Setting log level to: DEBUG
2016-08-04 16:33:45,842 - PixivUtil20160726 - INFO - No default cookie jar available, creating...
2016-08-04 16:33:45,842 - PixivUtil20160726 - INFO - using default DB Path: E:\MSOCache\Pixiv\pixivutil\db.sqlite
2016-08-04 16:33:45,842 - PixivUtil20160726 - INFO - Only process member where day last updated >= 300
2016-08-04 16:33:45,842 - PixivUtil20160726 - INFO - Only process image where day last updated >= 300
2016-08-04 16:33:45,842 - PixivUtil20160726 - INFO - Using Username: ########
2016-08-04 16:33:45,842 - PixivUtil20160726 - INFO - Limit up to: 100 page(s). (set via commandline)
2016-08-04 16:33:45,842 - PixivUtil20160726 - INFO - Trying to log with saved cookie
2016-08-04 16:33:48,368 - PixivUtil20160726 - INFO - Login successfull.
2016-08-04 16:33:48,368 - PixivUtil20160726 - INFO - Logged in using cookie
2016-08-04 16:33:48,368 - PixivUtil20160726 - INFO - Taglist mode.
2016-08-04 16:33:48,368 - PixivUtil20160726 - DEBUG - Sanitized Filename: ケモ
2016-08-04 16:33:48,368 - PixivUtil20160726 - INFO - Searching for: (ケモ) %E3%82%B1%E3%83%A2
2016-08-04 16:33:48,368 - PixivUtil20160726 - INFO - Looping... for http://www.pixiv.net/search.php?s_mode=s_tag&p=1&word=%E3%82%B1%E3%83%A2&order=date_d

And the active window :

PixivDownloader2 version 20160726
https://github.com/Nandaka/PixivUtil2/releases
Donate at https://bit.ly/PixivUtilDonation
Reading E:\MSOCache\Pixiv\pixivutil\config.ini ...
done.
using default DB Path: E:\MSOCache\Pixiv\pixivutil\db.sqlite
Creating database... done.
Only process member where day last updated >= 300
Only process image where day last updated >= 300
Using Username: #######
Limit up to: 100 page(s). (set via commandline)
Trying to log with saved cookie
Login successfull.
Reading: pixivutil/kem2.TXT
Reading E:\MSOCache\Pixiv\pixivutil\config.ini ...
done.
Save to each directory using query tags.
Searching for: ???? %E3%82%B1%E3%83%A2
Using Partial Match (search.php)
Looping... for http://www.pixiv.net/search.php?s_mode=s_tag&p=1&word=%E3%82%B1%E3%83%A2&order=date_d
No more images
done

I don't know the origin of this, just that it is quite new (3 days maximum). I tried to use an older version (20160711), renew the cookie, change the language on pixiv, but nothing had any effect. Maybe they made a change on Pixiv search page or a new type of file which make Pixivutil believe that there is no image to download ?

@TheDoc69 TheDoc69 changed the title Strange behaviour in tags or tags list mode New strange behaviour in tags or tags list mode Aug 4, 2016

@TheDoc69

This comment has been minimized.

Copy link
Author

TheDoc69 commented Aug 4, 2016

As a temporary fix, I found that starting the download at page 2 and not at page 1 works. Pixiv probably changed something on their website.

@Nandaka

This comment has been minimized.

Copy link
Owner

Nandaka commented Aug 6, 2016

Weird, I tried to key in from console for option 3 and 7, and it is working without issue (tested with 尻神様)

@TheDoc69

This comment has been minimized.

Copy link
Author

TheDoc69 commented Aug 6, 2016

Yes, it very weird. A lot of tags work without issue, but in some special cases, no images are detected when the search start at page 1.
尻神様 works perfectly for me, but ケモ doesn't (if I start at page 1 in mode 3 or 7, command line or batch).
In most cases, the problem tags are tags with a lot of results (ex: 調教 ,ショタ, or ケモ do not work ) and it's even worse for completely standard tags like 女 or 女の子 return "No more images". Start at page 2 and everything works again.
It's not that critical since starting at page 2 can remedy the problem (at the cost of not downloading the first 20 pics) for the moment, but it's very strange and started very recently (begining of august).

@fredgido

This comment has been minimized.

Copy link

fredgido commented Aug 7, 2016

I confirm the problem. Using page 2 works normally.

@Nandaka

This comment has been minimized.

Copy link
Owner

Nandaka commented Aug 7, 2016

I think there is a bug on the pixiv website itself.
When I tried to look for '女', it only return 4 images in the 1st pages.
image

only the 2nd page show the actual result.
image

EDIT: weird, reloading the page now show the full list from page 1
image

not sure what happened

@Nandaka

This comment has been minimized.

@TheDoc69

This comment has been minimized.

Copy link
Author

TheDoc69 commented Aug 7, 2016

Thanks a lot for the quick update. It seems to work.

I had an error on the first try but I didn't manage to reproduce it (based on the html dump, it seems that pixivutil was not "logged" on pixiv, but the log said "Login successfull").

By the way what was the cause on the problem with the tags ?

P.S. : Just in case, here is the error log of the error I had on the first try :
2016-08-07 19:02:04,765 - PixivUtil20160807-beta1 - ERROR - Error at process_tags_list(): (<type 'exceptions.AttributeError'>, AttributeError("'NoneType' object has no attribute 'findAll'",), <traceback object at 0x030D5788>)
Traceback (most recent call last):
File "PixivUtil2.py", line 926, in process_tags_list
File "PixivUtil2.py", line 780, in process_tags
File "PixivModel.pyc", line 858, in parseTags
AttributeError: 'NoneType' object has no attribute 'findAll'

But as i said i didn't manage to reproduce it so it was probably just an issue on pixiv side.

@TheDoc69

This comment has been minimized.

Copy link
Author

TheDoc69 commented Aug 13, 2016

Hi !
Just to let you know that after 1 week of use with the new 20160807-beta1 I had no problem, : everything is fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.