OverlayFS

[StuartIanNaylor]

I like the use of OverlayFS as large directory structures that would not be really possible to copy on the initial sync especially at boot become valid.

Like most things I am an OverlayFS noob and confused by the 2nd overlay directive copied from examples.
In some examples it is none and if you do a DF it seems to set the filesystem type description and a umount of the OverlayFS mount gives an output that the /opt/overlay mount doesn't exist.

I am struggling to find any direct documentation for that directive of as without it will look for a fstab entry.

mount --verbose --types overlay -o lowerdir=$BIND_DIR,upperdir=/opt/zram$RAM_DEV/upper,workdir=/opt/zram$RAM_DEV/workdir overlay $ZRAM_DIR

Also even though it will not error writing out to lower whilst mounted is supposedly a no-no.
Aufs is spin-lock crazy mad from what I can deduce from why some distro's and kernel say no and even though it would be an instant fix to writing out to lower I am slightly reluctant to make the switch.

Also wondering in ztab if log & dir entries should have a concat device option where a singular zram device is used with the sum of disksize and mem-limit entries if 1 and not if 0.

Anyway currently thinking about these 'issues/options' but if you have some input please do.

[amir73il]

Your umount script it doing something wrong.
opt/$ZRAM_DEV is opt/overlay which is something that doesn't exist.
And the first umount $ZRAM_DIR already unmounts overlay.
If you provide the real $ZRAM_DEV instead of overlay source string in mount command you may get what you wanted.

Apart from that I did not fully understand the requirements to make changes to lower directory.
And change like that should be done when overlayfs is not mounted. I do not know of an exiting offline merge tool, but it will be good to have one.

[StuartIanNaylor]

Hey your right doh! I hadn't really looked its the df as they have now no longer directly mounted to a zram device.
Apols and thanks.
Hows its found that!!! Because its in my code! :)
I am still trying to think of a merge without unmount as the unmount in one way or another is going to effect apps that run on them, some apps fail even if a log is unavail and would need individual restarts.
Guess just going to have to enforce a reboot or employ the devil of AUFS spin-locks.
I will sort the unmount script but the lack of an online merge is a pain for this application.
Its need is actually for very rare occurrences of new installs that may create directories and a system crash so that certain apps may just fail on a reboot if not there.
You could just write out on user input or a cron job a synctodisk without need of umount or stop.
But I guess I don't have to write to the lower and can write out to any persistent and just cp -arvfu to the lower before its mounted on boot.
In fact the latter and many thanks as just needed a break to have a think about it, would be great if you could do a merge with overlayFS though, even if it was slow to propagate through the mount.

[amir73il]

Right just be aware that cp may not sync deletes and renames in upper fs correctly to lower. You should test this. It will certainly not sync renamed directory correctly if redirect_dir option is enabled.
For these things a proper offline merge tool is needed.

[StuartIanNaylor]

https://github.com/kmxz/overlayfs-tools?

Apols Amir but if on stop I cp -arvfu the mounted mergedir to lower then umount the unionfs that should be OK? Its just continuing to use as the copy will just screw the OverlayFS resultant merge.
The copy merged to lower then unload is OK?
I will have a read of your stuff at https://github.com/amir73il/overlayfs/wiki/Overlayfs-snapshots and stop bugging you :) many thanks and really appreciate the input.
As if the copy on stop is OK then the write outs without stop could be snapshots I guess.

[amir73il]

overlayfs-tools seems like a nice project. never tried. it does not seem to support redirect_dir,
but it does support opaque dirs and whiteouts, so better than cp.
You can always add support for redirect_dir...
You wont find my answers above in my wiki. You may bug me that's fine ;-)

[StuartIanNaylor]

https://github.com/amir73il/overlayfs/wiki/Overlayfs-TODO#features
Offline/online layers merge tool? But no link.
What would you suggest?

[amir73il]

I would suggest trying out https://github.com/kmxz/overlayfs-tools. It supports the default overlayfs features and you can extend it to extra features if you need to.

[StuartIanNaylor]

Ok some reading needed as my head is spinning slightly.
As Mr Schwarzenegger would say "I probably will"

[StuartIanNaylor]

I did a bit of tidying and included overlayfs-tools which is great.

Just wondering can I enable redirect_dir via sys/module/overlay/parameters/* rather than edit /boot/cmdline.txt?

I am all ears about redirect_dir also wondering if snapshots could be used for what was the now deleted write method to call syncToDisk.
Currently its just on stop and my log in /usr/local/share/zram-config/log/zram-config.log looks fine and so does the resultant merge.

zram-config stop 2019-04-08-13:37:49
ztab remove log /zram2 /var/log /log.bind
/zram2
Warning: Stopping rsyslog.service, but it can still be activated by:
  syslog.socket
umount: /var/log (overlay) unmounted
overlay --lowerdir=/opt/zram/log.bind --upperdir=/opt/zram/zram2/upper
The script overlay-tools-Qkcisq.sh is created. Run the script to do the actual work please. Remember to run it when the OverlayFS is not mounted.
+ rm -rf /opt/zram/log.bind/auth.log
+ mv -T /opt/zram/zram2/upper/auth.log /opt/zram/log.bind/auth.log
+ rm -rf /opt/zram/log.bind/kern.log
+ mv -T /opt/zram/zram2/upper/kern.log /opt/zram/log.bind/kern.log
+ rm -rf /opt/zram/log.bind/daemon.log
+ mv -T /opt/zram/zram2/upper/daemon.log /opt/zram/log.bind/daemon.log
+ rm -rf /opt/zram/log.bind/wtmp
+ mv -T /opt/zram/zram2/upper/wtmp /opt/zram/log.bind/wtmp
+ rm -rf /opt/zram/log.bind/messages
+ mv -T /opt/zram/zram2/upper/messages /opt/zram/log.bind/messages
+ rm -rf /opt/zram/log.bind/syslog
+ mv -T /opt/zram/zram2/upper/syslog /opt/zram/log.bind/syslog
+ rmdir /opt/zram/zram2/upper
removed 'overlay-tools-Qkcisq.sh'
umount: /opt/zram/zram2/ unmounted
removed directory '/opt/zram/zram2'
umount: /opt/zram/log.bind/ unmounted
removed directory '/opt/zram/log.bind'
Warning: Stopping rsyslog.service, but it can still be activated by:
  syslog.socket
/dev/zram2 removed
ztab remove dir /zram1 /var/backups /backups.bind
/zram1
umount: /var/backups (overlay) unmounted
overlay --lowerdir=/opt/zram/backups.bind --upperdir=/opt/zram/zram1/upper
The script overlay-tools-iPXQBs.sh is created. Run the script to do the actual work please. Remember to run it when the OverlayFS is not mounted.
+ mv -T /opt/zram/zram1/upper/test /opt/zram/backups.bind/test
+ rm -rf /opt/zram/backups.bind/fart
+ mv -T /opt/zram/zram1/upper/fart /opt/zram/backups.bind/fart
+ rmdir /opt/zram/zram1/upper
removed 'overlay-tools-iPXQBs.sh'
umount: /opt/zram/zram1/ unmounted
removed directory '/opt/zram/zram1'
umount: /opt/zram/backups.bind/ unmounted
removed directory '/opt/zram/backups.bind'
/dev/zram1 removed
ztab remove swap /zram0 zram-config0 
/dev/zram0 removed
removed '/usr/local/share/zram-config/zram-device-list.rev'

Do you think snapshots will get included in the kernel? To be honest the `write' syncToDisk 'snapshot' function is not hugely important and from starting with log2ram always been slightly bemused as to why?
A snapshot or syncToDisk will always have a volatile gap to the previous that its probably better anyway to stop apps/services -> merge down to lower and restart.
But would be interested as some seem to think in this application it has value.

[amir73il]

You can change redirect_dir default via sys/module/overlay/parameters/
and you can enable it per overlay mount with mount option, see Documentation/filesystems/overlayfs.txt
for details.

overlayfs-tools does not seem to support redirect_dir. You might want to contact the author to ask about plans to support redirect_dir. I do not know the author and never used the tools myself.

I am not sure how overlayfs snapshots are related to syncToDisk functionality.
The feature is not rotating upper layer to lower layer if that is what you think.
Could be a nice feature, but I did not implement it.
Upstreaming plans are still quite far away at this point.

[StuartIanNaylor]

mount --verbose --types overlay -o redirect_dir=on,lowerdir=${ZDIR}${BIND_DIR},upperdir=${ZDIR}/zram${RAM_DEV}/upper,workdir=${ZDIR}/zram${RAM_DEV}/workdir overlay ${TARGET_DIR}

Is now a mount option.
Asked kmxz in my noob manner about about rename/redirect_dir support.
Still slightly bemused at what would seem an almost total lack of any supporting tools for OverlayFS?
Are there any plans for maybe a Linux-utils addition for OverlayFS now it is a kernel module?

Yeah I was thinking maybe could mount as upper merge down then mount updated lower with zram upper on next boot (with snapshots rotate down that is).

[flyingchipmunk]

Thanks for this project btw. I'm also interested in finding an atomic copy_down and will keep tabs on the collective progress here!

I noticed you turned off redirect_dir in your latest commits, curious why? Did you run into specific problems with directory renames in lower?

[StuartIanNaylor]

Purely the because the only tool for offline merge was written before that addition to OverlayFS.
To be honest unsure of the full implications but its to match the tools we have and that its choice if someone wishes to add it even though I think it is now default.

amir and the FS kernel file system guys know about the tool shortage as opposed to Aufs, but hoping for some suggestions but nothing seemed to be offered.

[mstormi]

has anyone considered using rsync ?

[yutayu]

has anyone considered using rsync ?

I agree with using rsync to sync.
I imagine like profile-sync-daemon.

[StuartIanNaylor]

Thanks for this project btw. I'm also interested in finding an atomic copy_down and will keep tabs on the collective progress here!

I noticed you turned off redirect_dir in your latest commits, curious why? Did you run into specific problems with directory renames in lower?

Think it became a default or I have made a mistake, with it being default I decicided to leave out so implementers could decide.

https://github.com/amir73il is the man to ask about all this as active in the dev group.
I have been meaning to get back in touch and say hey overlayfs guys when you going to give us more tools like aufs?!

[amir73il]

@StuartIanNaylor you should address your question to containers developers, because they have the most to gain from overlayfs tools. I have been involved with implementing many overlayfs features in the kernel for the past couple of years (redirect_dir,index,nfs_export,xino). All of these features bring benefits to end users, but AFAIK, they are all still disabled by default in containers (e.g. docker) because lack of tools support (e.g. migration/import/export).
I cannot force anyone to use the new overlayfs features nor to write offline tools support for them.
I will post a message to the kernel developers list to ask how we can improve the situation.

[StuartIanNaylor]

Thanks amir as its so frustrating as OverlayFS is brilliant but it does lack tools.
Really appreciate your input, many thanks.