Archiving

[inabon]

Hi there I am trying to understand archiving.
I have unraid i did the colon separated list
i also mapped the drives correctly /mnt/disks/sdk and on machinaris /sdk
I see all mounts in machinaris perfectly. I use ram drive so all good there.

but when what do i do with
dst variable at plotting configuation?

if i put all dirs there it doesnt like it
i do

dst
- /sdk
/sdm
/sdr

when i hit send it goes peptobismol pink on me saying there is an error.
If i comment out dst thinking the unraid machinaris colon list will take over it doesnt.

can you give me a small step by step easy config? I am going to be doing separte disks so the list will get very long. ahh thats another thing I am not doing raid or anything just a disk fill it up leave it and harvest if it dies it dies i get new one and replot. Is there a way to not have like a page or config yaml that has 100s of /dirivemounts ?

[guydavis]

Greetings. You have a few questions there. Let me repeat to ensure I understand. I think you have many drives on your Unraid server, all attached as Unassigned Devices. They are all mounted at: /mnt/disks on the Unraid server. You don't want to create a Volume Mount in the Docker configuration of Machinaris container for each drive.

Why not Volume mount the entire /mnt/disks path of Unraid to the /mnt path in the Machinaris container? You will still need to set each folder holding plots with in the comma-separated Environment Variable of plots_dir however so the various blockchains will know where to look. Please note, I haven't tried this myself, but believe others have made it work. I have only a few drives so just add Volume Mount for each.

Secondly, you seem to be asking how to plot in a way that automatically fills up your plot folders, taking into account free space on each. Please see "Local Archiving" on the Plotman wiki page. Since you will have your in-container paths at /mnt, try this:

archiving:
        target: local_rsync
        env:
                 site_root: /mnt
                 site_path_filter: plots

When you launch the Plotman Archiving from the Plotting | Workers page, the archive loop will look for completed plots in dst path(s), or just tmp if you are skipping dst (comment out), and then copy them to drives at /mnt/plots* based on available space.

By the way, I would recommend you try the upcoming Machinaris v0.8.5 release, currently in the machinaris:test image, which explicitly prints out the folders that will be used to make this configuration simpler.

[inabon]

many thanks and thank you for taking the time and calories to type all this up. I will give it a shot and report back. you answered all my questions. btw I am borifrmr it was a pleassure doing betatesting and fixing bugs :) looking forward to a second chance. Just have kind of a busy week in my regular job haha cheers mate and again many thanks.

[inabon]

quickie question I think i am almost there

            site_path_filter: plots

in my case i am using the name that unraid gives
sdk
sdm
sda

if i put the filter sd it should look for sdk etc right?

sounds like a lot of troube but this way i can tell on a plug in where the drive is in case it goes to hell i can pull it replace and plot. much easier than having to rebuild raid i man this is chia it is not a bank mainframe :) I am doing junkyard kind of build so if a drive goes oh well i can replace it fast. I got tired of unraid going crazy with parity rebuilding bla bla bla.

[inabon]

Ok time to report back pretty happy with results. Let´s begin.

"Why not Volume mount the entire /mnt/disks path of Unraid to the /mnt path in the Machinaris container?
That works perfect. I just made sure that I do not have any other drive in there which is no problem as i can make anything not chia a pool."

Mind you I did not see this anywhere in your wikki. Unraid likes any drive being used by docker to be setup as readwriteslave that is done at advanced form. well you know that but would be good to let people know. The difference I saw because the regular way works to it was a bit faster for plotter disks for example with the slave setting.

" archiving:
target: local_rsync
env:
site_root: /mnt
site_path_filter: plots"

That worked too what i am not too sure is the site_path_filter I have it there according to my way of naming things but I do not understand how it works.

so to sum things up. No need to map each drive but you do have to use the plots_dir: which can get a little stupid once i have 50 drives. Cant we come up with a flat file or script something that reads the mounted dir

df -h
Filesystem Size Used Avail Use% Mounted on
/dev/loop2 20G 4.0G 16G 21% /
tmpfs 64M 0 64M 0% /dev
tmpfs 63G 0 63G 0% /sys/fs/cgroup
shm 64M 60K 64M 1% /dev/shm
tmpfs 1.0M 0 1.0M 0% /farm
/dev/sdk1 5.5T 141G 5.4T 3% /farm/sdk
/dev/sdm1 5.5T 39G 5.5T 1% /farm/sdm
/dev/sdo1 3.7T 26G 3.7T 1% /farm/sdo
/dev/sdf1 821G 145M 819G 1% /plotting
tmpfs 110G 0 110G 0% /ramdisk
/dev/sdr1 466G 3.3G 463G 1% /archiver
/dev/loop2 20G 4.0G 16G 21% /id_rsa
shfs 1014G 35G 978G 4% /root/.chia
tmpfs 63G 0 63G 0% /proc/acpi
tmpfs 63G 0 63G 0% /sys/firmware
/dev/sds1 932G 703G 227G 76% /farm/WD-WJHGGULXCN

see how it lists all of the ones inside farm. that happens with our without plot dir variable and colon separated list. I can probably code a little script to do so but I do not want to touchy your stuff. hehe.

anyways chats later long night for me

cheers

[guydavis]

Ok time to report back pretty happy with results. Let´s begin.

" archiving:
target: local_rsync
env:
site_root: /mnt
site_path_filter: plots"

That worked too what i am not too sure is the site_path_filter I have it there according to my way of naming things but I do not understand how it works.

Great, glad you solved your concern. As per the Machinaris wiki, site_path_filter: plots allows one to exclude a drive (like /mnt/plotting from being an archive target like others at the same level such as /mnt/plots1, /mnt/plots2, /mnt/plots3 etc. This applies to both local (same machine) and remote (another machine) plot archiving.

[inabon]

Ok so if i have Sdk Sdo Sdj As plot drives Plot filter should be?Sd Or Drive_name_i donot want firstletters?

[inabon]

Ok there is something weird going on here.

your instructions say that the path filter will choose a similar pattern.
so i use this
env:
site_root: /farm/
site_path_filter: sd

and get this on the directories
Archiving Directories (optional)
/farm/backup (243 GB free)
/farm/sdk (5630 GB free)
/farm/sdm (5739 GB free)
/farm/sdo (3970 GB free)

now if i do
env:
site_root: /farm/
site_path_filter: ba

Archiving Directories (optional)
/farm/backup (243 GB free)

why would sd choose backup if there is no S or D in the name.

[inabon]

Hahaha i found out why. it uses the output of df -h
df -h
Filesystem Size Used Avail Use% Mounted on
/dev/loop2 20G 4.0G 16G 21% /
tmpfs 64M 0 64M 0% /dev
tmpfs 63G 0 63G 0% /sys/fs/cgroup
shm 64M 60K 64M 1% /dev/shm
/dev/sdb1 466G 3.3G 463G 1% /archiver
tmpfs 1.0M 0 1.0M 0% /farm
/dev/sdl1 5.5T 344G 5.2T 7% /farm/sdk
/dev/sdn1 5.5T 242G 5.3T 5% /farm/sdm
/dev/sdp1 3.7T 26G 3.7T 1% /farm/sdo
/dev/sdq1 932G 703G 227G 76% /farm/backup
/dev/loop2 20G 4.0G 16G 21% /id_rsa
/dev/sdi1 821G 3.8M 819G 1% /plotting
rootfs 63G 1.9G 61G 3% /ramdisk
shfs 1014G 34G 976G 4% /root/.chia
tmpfs 63G 0 63G 0% /proc/acpi
tmpfs 63G 0 63G 0% /sys/firmware

so sd is in all of them i tried sdl and got /farm/sdk you would have to ommit the fylesystem form the query that would make dir path perfect.

[inabon]

Ok I was able to work around the issue. I just had to get creative with mount names :) cheers