Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

illustrator stored as self(0) in file #236

Closed
Kilo19 opened this issue Jun 12, 2017 · 18 comments
Closed

illustrator stored as self(0) in file #236

Kilo19 opened this issue Jun 12, 2017 · 18 comments
Labels
Bug

Comments

@Kilo19
Copy link

@Kilo19 Kilo19 commented Jun 12, 2017

I'm using 20170524-beta3. I downloaded works by pixiv member 3723954 (https://www.pixiv.net/member.php?id=3723954) today. The download is fine, but the file is stored in directories like "L:\pic\self(0)\self(0)-あや(38349407)東方,東方project,落書き,射命丸文.jpg".

@Nandaka
Copy link
Owner

@Nandaka Nandaka commented Jun 13, 2017

Weird, I tried and it stores properly to PixivUtil\testapp\あまの@お仕事募集中 (3723954)\38349407_p0 - ❤あや❤.jpg. Tested with filename format %artist% (%member_id%)\%urlFilename% - %title% in config.ini

@PixivAddict
Copy link

@PixivAddict PixivAddict commented Jun 19, 2017

also with same issue, seems like it randomly happens? I have a folder self (0) with tons of images, I just never noticed it was placing files there. I saw this in my window:

Already downloaded: 31817836
Last Page
Done.

done.
Processing Member Id: 1197484
Reading C:\pixivutils\pixivutil(4b)\config.ini ...
done.
Page 1
Member Url: http://www.pixiv.net/member_illust.php?id=1197484&p=1
Member Name : self
Member Avatar: no_profile
Member Token : self
#1
Processing Image Id: 63375313
Title: 蕾咪集合
Tags : 東方, レミリア·スカーレット, めーレミ, 十六夜咲夜, 東方Project500users入り

Date : 2017-06-14 03:15:00
Mode : manga
Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=0
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p0.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=1
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p1.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=2
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p2.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=3
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p3.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=4
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p4.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=5
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p5.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=6
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p6.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=7
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p7.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=8
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p8.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=9
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p9.jpg

Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=10
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p10.jp
g
Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=11
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p11.jp
g
Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=12
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p12.jp
g
Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=13
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p13.jp
g
Fetching big image page: http://www.pixiv.net/member_illust.php?mode=manga_big&i
llust_id=63375313&page=14
Found: https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p14.jp
g
Page Count : 15
Image URL : https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p0
.jpg
Filename : C:\PixivDownloaded\self (0)\63375313_p0.jpg
Checking local filename... Getting remote filesize...
Using Referer: http://www.pixiv.net/member_illust.php?mode=medium&illust_id=6337
5313
Remote filesize = 250.00 KiB (256627 Bytes)
Start downloading... Using Referer: http://www.pixiv.net/member_illust.php?mode=
medium&illust_id=63375313
256627 of 256627 Bytes Completed in 0.761s (329.32 KiB/s)
Image verified.

Image URL : https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p1
.jpg
Filename : C:\PixivDownloaded\self (0)\63375313_p1.jpg
Checking local filename... Getting remote filesize...
Using Referer: http://www.pixiv.net/member_illust.php?mode=medium&illust_id=6337
5313
Remote filesize = 616.00 KiB (631777 Bytes)
Start downloading... Using Referer: http://www.pixiv.net/member_illust.php?mode=
medium&illust_id=63375313
631777 of 631777 Bytes Completed in 0.863s (714.91 KiB/s)
Image verified.

Image URL : https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p2
.jpg
Filename : C:\PixivDownloaded\self (0)\63375313_p2.jpg
Checking local filename... Getting remote filesize...
Using Referer: http://www.pixiv.net/member_illust.php?mode=medium&illust_id=6337
5313

@Nandaka Nandaka added the Bug label Jun 19, 2017
@Nandaka Nandaka closed this in 8d5d6e6 Jun 19, 2017
@Nandaka
Copy link
Owner

@Nandaka Nandaka commented Jun 19, 2017

need sample input

@Nandaka Nandaka reopened this Jun 19, 2017
@Nandaka
Copy link
Owner

@Nandaka Nandaka commented Jun 19, 2017

Can you try https://github.com/Nandaka/PixivUtil2/releases/tag/20170619-beta4

It will generate an html page if the member id is not able to be parsed. Please upload it here if its happen.

@zoram999
Copy link

@zoram999 zoram999 commented Jun 26, 2017

Hi, I can confirm this bug happens at random, I just close the program, relaunch, and the next time it downloads correctly. Happened with both 20170411 and 20170524-beta3. I'll try the newest one and observe what happens.

@Nandaka
Copy link
Owner

@Nandaka Nandaka commented Jun 26, 2017

one of it is due to deleted image.
other one is the correct file, looks like they update the layout

==> will update the parser.

Nandaka added a commit that referenced this issue Jun 26, 2017
@WhiteSp1rit
Copy link

@WhiteSp1rit WhiteSp1rit commented Aug 27, 2017

Just had the exact same issue. I'll check if the new version then.

Is there any way to fix my current situation - in this "self" folder I have a ton of images now, and they use the same names as on Pixiv so author's name isn't indicated. Even if I relaunch PixivUtil2 this won't help since all these images will be marked as downloaded already. How do I delete only these ones from the database, so that the program could re-download them into correct folders?

@Nandaka
Copy link
Owner

@Nandaka Nandaka commented Aug 27, 2017

You can use d. Manage database then 10. Delete image by image_id.

Or delete the wrong images and temporarily change the config.ini values for:

overwrite = True
alwayscheckfilesize = True
@NHOrus
Copy link
Contributor

@NHOrus NHOrus commented Aug 27, 2017

Alternatively, purge folder, d. Manage database then c. Cleanup database then redownload.

@WhiteSp1rit
Copy link

@WhiteSp1rit WhiteSp1rit commented Aug 27, 2017

Delete image by image_id won't work, there is like a thousand images there, haven't launched an update in a while.

overwrite = True
alwayscheckfilesize = True

I assume this will also re-download all the images I deleted prior to that? Which I definitely do not want to happen. Also it will overwrite every image which I modified, by cropping or whatever.

Is there any way to delete the entries from DB by date?
Because all of them were made within a known period of time, which can be specified.

Alternatively, purge folder, d. Manage database then c. Cleanup database then redownload.

That is not a solution at all. I have thousands of images. And many images were deleted, which do not need to be downloaded. Suggesting deleting entire database when I need to delete like less than 1% of it sounds insane.

@NHOrus
Copy link
Contributor

@NHOrus NHOrus commented Aug 27, 2017

Well, you can find external SQLite DB browser then, and use it to remove all images with directory self(0), but I haven't tried that and got no experience here.

@WhiteSp1rit
Copy link

@WhiteSp1rit WhiteSp1rit commented Aug 27, 2017

Yeah, that is the last resort and will probably what I'll have to do then.

Re-reading the readme, I don't see any commands for deleting entries by dates.

  • Delete member by member_id
  • Delete image by image_id
  • Delete member and image (cascade deletion)

This is all of what is listed.

@Nandaka
Copy link
Owner

@Nandaka Nandaka commented Aug 27, 2017

@WhiteSp1rit It will not do the actual download if the local filesize is the same as the server filesize, but it still issue a HEAD request to the server to get the file size.

The other option is using DB editor to delete by dates. You can use SQLiteBrowser (http://sqlitebrowser.org/)

@WhiteSp1rit
Copy link

@WhiteSp1rit WhiteSp1rit commented Aug 27, 2017

Yeah just opened the DB with this, and here they are, entries with a path like:

Nandaka's PixivUtil2\pixiv_images\[0] self\64553359_p0.png

Thankfully since the issue is fairly simple I won't have to re-learn SQL ;)
Filtering by "[0] self" showed all invalid entries.

Looking at the dates earlist one was "2017-07-06 02:46:44", so this problem first appeared more than a month ago.

@JustAQuinn
Copy link

@JustAQuinn JustAQuinn commented Sep 5, 2017

So the procedure is to use SQLiteBrowser, find all '[0] self' entries, delete them all, and hit Cleanup database and redownload?

@WhiteSp1rit
Copy link

@WhiteSp1rit WhiteSp1rit commented Sep 5, 2017

"[0] self" is how my folder was named under the naming scheme I used. Yours will probably be different.
For example, if you use just member id for folder names, your folder might be just "0" or something.
I don't even know where you got the "Cleanup database" command or what does it do, you use "Write changes"

@Nandaka Nandaka closed this Sep 20, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants
You can’t perform that action at this time.