Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to choose workdir for temporary storage #83

Open
raybooysen opened this issue Jan 12, 2024 · 4 comments
Open

Ability to choose workdir for temporary storage #83

raybooysen opened this issue Jan 12, 2024 · 4 comments

Comments

@raybooysen
Copy link

The Archive Warrior running writes gigabytes of data before uploading. I would love a way to specify via env variable the location of the temporary storage to a file system or storage device that I'd prefer

@TheTechRobo
Copy link

Assuming you're using docker, you can use its -v parameter to bridge the current workdir to wherever you want on the host system.

@raybooysen
Copy link
Author

I've been trying with some variants. /home/warrior, /home/warrior/data, /home/warrior/projects. All cause various amounts of errors.

Unsure which I should be using

@viniciushsantana
Copy link

I've been considering this issue as well. It seems that the working directories /home/warrior/projects and /home/warrior/data are being utilized in a manner that prevents exposing them as volumes, due to the absence of certain necessary files.

For example, within /home/warrior/data, the binaries wget-at and wget-at-gnutls should be present:

&& ln -fs /usr/local/bin/wget-lua /home/warrior/data/wget-at
COPY --from=atdr.meo.ws/archiveteam/grab-base:gnutls /usr/local/bin/wget-lua /home/warrior/data/wget-at-gnutls

I've discovered a potential workaround for this issue by altering Docker's data-root setting. However, this approach is not ideal and could lead to other complications.

Properly managing working directories and enabling the exposure of volumes would also facilitate the use of tmpfs mounts. This is particularly beneficial for users with ample RAM available who wish to conserve some SSD IOPS.

@raybooysen
Copy link
Author

This was my primary usecase. My warriors run for long periods on machines with spare RAM, a tmpfs is a good use case here so avoid the SSD completely for temporal data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants