Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reopen. client.util.downloadIllust does not wait #32

Closed
dgudim opened this issue Sep 26, 2022 · 1 comment
Closed

reopen. client.util.downloadIllust does not wait #32

dgudim opened this issue Sep 26, 2022 · 1 comment
Labels
invalid This doesn't seem right

Comments

@dgudim
Copy link

dgudim commented Sep 26, 2022

I am reopening this issue because the previous one was closed but the issue is still there.

Here is what i came up with to 'solve' the issue

const img_dir = "./downloaded";
await client.util.downloadIllust(url, img_dir, "original");

// should be downloaded by now
let exit = false;
let prev_hash = "";
let images: string[] = [];
while(true) {
    let images_stats = fs.readdirSync(img_dir);
    let curr_hash = "";
    images = [];
    for (const image_stats of images_stats) {
        const file = img_dir + "/" + image_stats;
        images.push(file);
        curr_hash += await getFileHash(file);
    }
    if(exit) {
        break;
    }
    if (prev_hash != curr_hash || !curr_hash.length) {
        debug(`hash mismatch, waiting... (${prev_hash} || ${curr_hash})`);
    }
    exit = prev_hash == curr_hash && curr_hash.length > 0;
    prev_hash = curr_hash;
    await sleep(1000);
}

// by this point the images are downloaded, but the solution is mega-crap

here is the output

hash mismatch, waiting... (c397c6d991c6ddc033cc089b33a412355 || af1349b9f5f9a1a6a0404dea36dcc9499)
hash mismatch, waiting... (af1349b9f5f9a1a6a0404dea36dcc9497 || efb763f51eb0780232d81f86cdc86135c)
hash mismatch, waiting... (a2344fc328fe26a3ff5f241c10ba91071 || 9183ef0552ac039c012bec3885eb85d79)
hash mismatch, waiting... (2b90f3912a26806e03a96f0a107870f16 || 015537fcb4eee269792ed88ad1d7bdb84)
hash mismatch, waiting... (b90f3912a26806e03a96f0a107870f167 || 12a26806e03a96f0a107870f167258f9a)

as you can see, the hash changes which means that the files are still downloading, please don't close this issue blaming me, this is clearly a problem in the library

@Moebits Moebits added the invalid This doesn't seem right label Sep 26, 2022
@Moebits
Copy link
Owner

Moebits commented Sep 26, 2022

Let me check if the hashes match 5 seconds after the method resolves.

await pixiv.util.downloadIllust(illusts[0], "./downloads", "original")
const files = fs.readdirSync(path.join(__dirname, "../downloads")).map((p) =>{
    return path.join(__dirname, "../downloads", p)
})
const hash = crypto.createHash("md5").update(fs.readFileSync(files[0])).digest("hex")
console.log(hash)
setTimeout(() => {
    const hash2 = crypto.createHash("md5").update(fs.readFileSync(files[0])).digest("hex")
    console.log(hash)
    console.log(hash === hash2)
}, 5000)

be4ce738163323ed07f20308252128ea
be4ce738163323ed07f20308252128ea
true

Yep, they are the same, so it's a problem with your code not mine.

@Moebits Moebits closed this as completed Sep 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
invalid This doesn't seem right
Projects
None yet
Development

No branches or pull requests

2 participants