Downloaders not working (seemingly because of) Temp folder not working

[Oxian1]

Hydrus version

v595

Qt major version

Qt 6

Operating system

Linux (specify distro and version in comments)

Install method

Extract

Install and OS comments

install comment: I specified extract because I had to extract from a zip file, but I also have to open hydrus through the console so i'm not sure if "running from source" might be more accurate or not

OS comment: Linux mint, specifically cinnamon 22

Bug description and reproduction

I was running a gallery Downloader when I got the following error message

upon futher investigation I found that the directory highlighted in red, and its equivalents given the nature of tmp files, always exist.
but they are equally always completely empty, even though the error message pops up unambiguously after the file finishes downloading.

also, the error message says "details should be shown" but I could not find any additional error info anywhere. no pop-up, and nothing on the logs or console that wasn't already in the above screenshot **

**edit: I have since retried the failed files with a few relevant seeming report modes turned on and pasted the results to the log section

I have tried:
restarting my computer - no change
restarting hydrus - no change
re-trying the troublesome files - no change
skipping the troublesome files - different tmp and destination directories, but ultimately the same error with every subsequent file
downloading the troublesome files one at a time with a url downloader - no change
removing the query entirely and starting from scratch - no change
using modifiers in the console to put the temp folder on a different drive - the error message changed to reflect the new directory, but ultimately the same error
using modifiers in the console to not use temp folders at all and store temporary files in memory instead - no change I stg it just ignored me and made non-functional temp folders anyway

the biggest difference I saw was when I completely clean installed hydrus with a fresh DB and you wanna know what changed? I got the error sooner than usual.

I'm completely at a loss, really hoping there's something obvious i've missed
thanks a ton for any help yall can provide

Log output

v595, 2024-10-27 11:48:11: Running db job: read url_statuses
v595, 2024-10-27 11:48:11: File path request: (b"\xd6\xe2A\xb8\xc1\xba\xa1\x05\x86\xe1%\xe6\x9b\x14!\xd7w\x81\xf0\\'\xc7\xb8Zj$\x18\xd5b\x9e\x13-", 1)
v595, 2024-10-27 11:48:11: Running db job: read media_result
v595, 2024-10-27 11:48:12: Running db job: read tag_siblings_lookup
v595, 2024-10-27 11:48:12: Running db job: read url_statuses
v595, 2024-10-27 11:48:12: File path request: (b"\xd6\xe2A\xb8\xc1\xba\xa1\x05\x86\xe1%\xe6\x9b\x14!\xd7w\x81\xf0\\'\xc7\xb8Zj$\x18\xd5b\x9e\x13-", 1)
v595, 2024-10-27 11:48:12: Running db job: read media_result
v595, 2024-10-27 11:48:14: File import job created:
Source: https://e621.net/posts/4830081
Raw import path: /tmp/hydrus7vcrpd2n/hydrusd5img1sn.
v595, 2024-10-27 11:48:14: File import job starting work.
v595, 2024-10-27 11:48:14: File import job hash: d6e241b8c1baa10586e125e69b1421d77781f05c27c7b85a6a2418d5629e132d
v595, 2024-10-27 11:48:14: Running db job: read hash_status
v595, 2024-10-27 11:48:14: File path request: (b"\xd6\xe2A\xb8\xc1\xba\xa1\x05\x86\xe1%\xe6\x9b\x14!\xd7w\x81\xf0\\'\xc7\xb8Zj$\x18\xd5b\x9e\x13-", 1)
v595, 2024-10-27 11:48:14: File import job pre-import status: unknown, The client believed this file was already in the db, but it was truly missing! Import will go ahead, in an attempt to fix the situation.
v595, 2024-10-27 11:48:14: File import job mime: jpeg
v595, 2024-10-27 11:48:14: File import job file info: (393237, 1, 2500, 3450, None, None, False, None)
v595, 2024-10-27 11:48:14: File import job generating thumbnail
v595, 2024-10-27 11:48:14: File import job generating perceptual_hashes
v595, 2024-10-27 11:48:14: File import job generated 1 perceptual_hashes: ['c03f317ccac0372f']
v595, 2024-10-27 11:48:14: File import job generating other hashes
v595, 2024-10-27 11:48:14: File import job testing if good to import for file import options
v595, 2024-10-27 11:48:14: Adding file to client file structure: from /tmp/hydrus7vcrpd2n/hydrusd5img1sn to /media/oxian/Seagate/----/Hydrus Network/db/client_files/fd6/d6e241b8c1baa10586e125e69b1421d77781f05c27c7b85a6a2418d5629e132d.jpg
v595, 2024-10-27 11:48:14: Copying the file from "/tmp/hydrus7vcrpd2n/hydrusd5img1sn" to "/media/oxian/Seagate/----/Hydrus Network/db/client_files/fd6/d6e241b8c1baa10586e125e69b1421d77781f05c27c7b85a6a2418d5629e132d.jpg" failed! Details should be shown and other import queues should be paused. You should shut the client down now and fix this!
v595, 2024-10-27 11:48:14: A critical drive error has occurred. All importers--subscriptions, import folders, and paged file import queues--have been paused. Once the issue is clear, restart the client and resume your imports under the file and network menus!

[hydrusnetwork]

Hey, I am sorry for the trouble here. That /tmp/hydrustoehdunot stuff is supposed to be the normal temp folder that hydrus takes when it boots up and uses for all sorts of normal temp file stuff. Every imported file goes through there. It seems like your OS is not happy with hydrus using it, I presume for a permissions reason. I haven't seen anything quite like your problem before, but I've seen similar when guys have a temp dir that is mounted on a ramdisk that has some unusual properties. I vaguely remember a guy who has this problem when his anti-virus was unhappy about a dodgy pdf coming in, but I may be remembering it wrong.

Let's see if we can short-circuit this whole problem simply. You can tell hydrus to use a different folder than your OS default /tmp for its temp dir using the --temp_dir launch switch. Please check it out here:

https://hydrusnetwork.github.io/hydrus/launch_arguments.html#--temp_dir_temp_dir

So please configure your launch shortcut or script to do the appropriate variant of client --temp_dir="/some/other/path", and then double-check that that was set correctly under help->about, and let me know how it went.

[hydrusnetwork]

Looking again at your log, I am not sure if it is the temp dir that is the problem. The client would have processed the file ok at that point, and the actual error is when it is copying into your normal file structure. I think the 'details should be shown' is supposed to be appearing in the 'file log' of the respective importer, it would be in the notes section, but I have updated tomorrow's v596 release to just spam this exact error to the log, so if moving the temp dir doesn't help you here, please try it again in v596 when it is convenient and we'll see what it puts in the log.

[bbappserver]

@hydrusnetwork @Oxian1
This error occurs when the drive is busy copying from tmp to destination. It is the same bug I was experiencing with my NAS. As the destination is seagate, I assume this means that the drive took longer than the configured system drive timeout, and that's really all we can say. This probably means that the drive is not designed to keep up with a flood of writes to the point that the copy fails, or that drive failure is imminent and the drive failed after trying desperately to write a sector.

The new copy-retry logic should alleviate this if it is not a hardware failure.

[hydrusnetwork]

Yes, this might be it, thanks! Please try updating your client @Oxian1 , and we'll see if the error is relieved or goes away entirely.

[Oxian1]

oh wow, tbh I only just realised you had emailed me directly, thankyou for taking the time out of your day(s) ^^ the issue seemed to be fixed ever since I uninstalled hydrus and re-installed using a different method, unfortunately it's been too long for me to recall what installation method I had switched to (or from), frankly I'm not even on linux anymore either way, I really appreciate you reaching out ^^ if only I had noticed sooner so that I could have been more helpful in Identifying the root issue best wishes & many thanks oxian

…

On Sun, 15 Dec 2024 at 06:02, Hydrus Network Developer < ***@***.***> wrote: Yes, this might be it, thanks! Please try updating your client @Oxian1 <https://github.com/Oxian1> , and we'll see if the error is relieved or goes away entirely. — Reply to this email directly, view it on GitHub <#1620 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A4LWL76DNFFRAOKLYZNLUIL2FR6EHAVCNFSM6AAAAABQU5VFVWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBTGMYTGMZRGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[hydrusnetwork]

Great, I am glad you are working again. Let me know if you have any more trouble!