New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

illustrator stored as self(0) in file #236

Closed
Kilo19 opened this Issue Jun 12, 2017 · 18 comments

Comments

Projects
None yet
7 participants
@Kilo19

Kilo19 commented Jun 12, 2017

I'm using 20170524-beta3. I downloaded works by pixiv member 3723954 (https://www.pixiv.net/member.php?id=3723954) today. The download is fine, but the file is stored in directories like "L:\pic\self(0)\self(0)-❤あや❤(38349407)東方,東方project,落書き,射命丸文.jpg".

@Nandaka

This comment has been minimized.

Owner

Nandaka commented Jun 13, 2017

Weird, I tried and it stores properly to PixivUtil\testapp\あまの@お仕事募集中 (3723954)\38349407_p0 - ❤あや❤.jpg. Tested with filename format %artist% (%member_id%)\%urlFilename% - %title% in config.ini

@PixivAddict

This comment has been minimized.

PixivAddict commented Jun 19, 2017

also with same issue, seems like it randomly happens? I have a folder self (0) with tons of images, I just never noticed it was placing files there. I saw this in my window:

Already downloaded: 31817836
Last Page
Done.

done.
Processing Member Id: 1197484
Reading C:\pixivutils\pixivutil(4b)\config.ini ...
done.
Page 1
Member Url: http://www.pixiv.net/member_illust.php?id=1197484&p=1
Member Name : self
Member Avatar: no_profile
Member Token : self
#1
Processing Image Id: 63375313
Title: 蕾咪集合
Tags : 東方, レミリア·スカーレット, めーレミ, 十六夜咲夜, 東方Project500users入り

Date : 2017-06-14 03:15:00
Mode : manga
Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=0
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p0.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=1
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p1.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=2
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p2.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=3
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p3.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=4
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p4.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=5
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p5.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=6
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p6.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=7
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p7.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=8
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p8.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=9
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p9.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=10
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p10.jp
g
Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=11
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p11.jp
g
Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=12
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p12.jp
g
Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=13
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p13.jp
g
Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=14
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p14.jp
g
Page Count : 15
Image URL : https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p0
.jpg
Filename : C:\PixivDownloaded\self (0)\63375313_p0.jpg
Checking local filename... Getting remote filesize...
Using Referer: http://www.pixiv.net/member_illust.php?mode=medium&illust_id=6337
5313
Remote filesize = 250.00 KiB (256627 Bytes)
Start downloading... Using Referer: http://www.pixiv.net/member_illust.php?mode=
medium&illust_id=63375313
256627 of 256627 Bytes Completed in 0.761s (329.32 KiB/s)
Image verified.

Image URL : https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p1
.jpg
Filename : C:\PixivDownloaded\self (0)\63375313_p1.jpg
Checking local filename... Getting remote filesize...
Using Referer: http://www.pixiv.net/member_illust.php?mode=medium&illust_id=6337
5313
Remote filesize = 616.00 KiB (631777 Bytes)
Start downloading... Using Referer: http://www.pixiv.net/member_illust.php?mode=
medium&illust_id=63375313
631777 of 631777 Bytes Completed in 0.863s (714.91 KiB/s)
Image verified.

Image URL : https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p2
.jpg
Filename : C:\PixivDownloaded\self (0)\63375313_p2.jpg
Checking local filename... Getting remote filesize...
Using Referer: http://www.pixiv.net/member_illust.php?mode=medium&illust_id=6337
5313

@Nandaka Nandaka added the Bug label Jun 19, 2017

@Nandaka Nandaka closed this in 8d5d6e6 Jun 19, 2017

@Nandaka

This comment has been minimized.

Owner

Nandaka commented Jun 19, 2017

need sample input

@Nandaka Nandaka reopened this Jun 19, 2017

@Nandaka

This comment has been minimized.

Owner

Nandaka commented Jun 19, 2017

Can you try https://github.com/Nandaka/PixivUtil2/releases/tag/20170619-beta4

It will generate an html page if the member id is not able to be parsed. Please upload it here if its happen.

@PixivAddict

This comment has been minimized.

@zoram999

This comment has been minimized.

zoram999 commented Jun 26, 2017

Hi, I can confirm this bug happens at random, I just close the program, relaunch, and the next time it downloads correctly. Happened with both 20170411 and 20170524-beta3. I'll try the newest one and observe what happens.

@Nandaka

This comment has been minimized.

Owner

Nandaka commented Jun 26, 2017

one of it is due to deleted image.
other one is the correct file, looks like they update the layout

==> will update the parser.

Nandaka added a commit that referenced this issue Jun 26, 2017

@Nandaka

This comment has been minimized.

@WhiteSp1rit

This comment has been minimized.

WhiteSp1rit commented Aug 27, 2017

Just had the exact same issue. I'll check if the new version then.

Is there any way to fix my current situation - in this "self" folder I have a ton of images now, and they use the same names as on Pixiv so author's name isn't indicated. Even if I relaunch PixivUtil2 this won't help since all these images will be marked as downloaded already. How do I delete only these ones from the database, so that the program could re-download them into correct folders?

@Nandaka

This comment has been minimized.

Owner

Nandaka commented Aug 27, 2017

You can use d. Manage database then 10. Delete image by image_id.

Or delete the wrong images and temporarily change the config.ini values for:

overwrite = True
alwayscheckfilesize = True
@NHOrus

This comment has been minimized.

Contributor

NHOrus commented Aug 27, 2017

Alternatively, purge folder, d. Manage database then c. Cleanup database then redownload.

@WhiteSp1rit

This comment has been minimized.

WhiteSp1rit commented Aug 27, 2017

Delete image by image_id won't work, there is like a thousand images there, haven't launched an update in a while.

overwrite = True
alwayscheckfilesize = True

I assume this will also re-download all the images I deleted prior to that? Which I definitely do not want to happen. Also it will overwrite every image which I modified, by cropping or whatever.

Is there any way to delete the entries from DB by date?
Because all of them were made within a known period of time, which can be specified.

Alternatively, purge folder, d. Manage database then c. Cleanup database then redownload.

That is not a solution at all. I have thousands of images. And many images were deleted, which do not need to be downloaded. Suggesting deleting entire database when I need to delete like less than 1% of it sounds insane.

@NHOrus

This comment has been minimized.

Contributor

NHOrus commented Aug 27, 2017

Well, you can find external SQLite DB browser then, and use it to remove all images with directory self(0), but I haven't tried that and got no experience here.

@WhiteSp1rit

This comment has been minimized.

WhiteSp1rit commented Aug 27, 2017

Yeah, that is the last resort and will probably what I'll have to do then.

Re-reading the readme, I don't see any commands for deleting entries by dates.

  • Delete member by member_id
  • Delete image by image_id
  • Delete member and image (cascade deletion)

This is all of what is listed.

@Nandaka

This comment has been minimized.

Owner

Nandaka commented Aug 27, 2017

@WhiteSp1rit It will not do the actual download if the local filesize is the same as the server filesize, but it still issue a HEAD request to the server to get the file size.

The other option is using DB editor to delete by dates. You can use SQLiteBrowser (http://sqlitebrowser.org/)

@WhiteSp1rit

This comment has been minimized.

WhiteSp1rit commented Aug 27, 2017

Yeah just opened the DB with this, and here they are, entries with a path like:

Nandaka's PixivUtil2\pixiv_images\[0] self\64553359_p0.png

Thankfully since the issue is fairly simple I won't have to re-learn SQL ;)
Filtering by "[0] self" showed all invalid entries.

Looking at the dates earlist one was "2017-07-06 02:46:44", so this problem first appeared more than a month ago.

@JustAQuinn

This comment has been minimized.

JustAQuinn commented Sep 5, 2017

So the procedure is to use SQLiteBrowser, find all '[0] self' entries, delete them all, and hit Cleanup database and redownload?

@WhiteSp1rit

This comment has been minimized.

WhiteSp1rit commented Sep 5, 2017

"[0] self" is how my folder was named under the naming scheme I used. Yours will probably be different.
For example, if you use just member id for folder names, your folder might be just "0" or something.
I don't even know where you got the "Cleanup database" command or what does it do, you use "Write changes"

@Nandaka Nandaka closed this Sep 20, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment