Change find to ls -r in 30[0]_create_extlinux.sh

[lrirwin]

• rear version (/usr/sbin/rear -V): 1.18 forward

• Brief description of the issue:
  When "backup" folders exist, using "find" takes a long time.
  Suggest this change at line 164 in output/USB/Linux-i386/30[0]_create_extlinux.sh:

- for file in $(cd $BUILD_DIR/outputfs; find rear/*/* -name syslinux.cfg); do
+ for file in $(cd $BUILD_DIR/outputfs; ls -r rear/*/*/syslinux.cfg); do

This is much quicker when the backup folders contain thousands of files.
It also accomplishes part of the TODO mentioned in the script.
Thanks for all you do!

[jsmeix]

In general regarding using 'ls' versus 'find'
when its argument contains a wildcard '*' see
https://raw.githubusercontent.com/rear/rear/master/usr/share/rear/prep/NETFS/default/070_set_backup_archive.sh
(excerpt)

# Here things like 'find /path/to/dir -name '*.tar.gz' | sort' are used because
# one cannot use bash globbing via commands like 'ls /path/to/dir/*.tar.gz'
# because /usr/sbin/rear sets the nullglob bash option which leads to plain 'ls'
# when '/path/to/dir/*.tar.gz' matches nothing (i.e. when no backup file exists)
# so that then plain 'ls' would result nonsense.

I.e. before calling

ls -r rear/*/*/syslinux.cfg

one would need to ensure that "rear/*/*/syslinux.cfg"
does never evaluate to nothing - i.e. an additional test
would be needed before calling "ls -r rear/*/*/syslinux.cfg".

Comparison how much time is actually saved
(on my test system):
with "find"

real    0m0.007s
user    0m0.006s
sys     0m0.001s

with "ls -r"

real    0m0.018s
user    0m0.015s
sys     0m0.002s

Surprise!
With "ls -r" ist is actually much slower.

But even if it was faster with "ls -r"
the whole stuff would be about saving
at most a few thousandths of a second
which is not worth any effort.

[jsmeix]

@lrirwin
can you explain what you mean with
"backup folders contain thousands of files"?

[lrirwin]

Yes -- each night these run: rear -v savelayout rear -vdD mkbackup That creates folders like: $BUILD_DIR/outputfs/rear/$HOSTNAME/20170110.2215/backup The entire linux system is backed up into that folder. I've succeeded in being able to incorporate --link-dest into ReaR's rsync method, so there may be as many as 15 folders on any given media. Each of them is a complete system backup, so there are thousands of entries in each backup folder. ls -r runs a lot faster than using find and it gives them in reverse order as well... BTW - this is the --link-dest solution I came up with in /usr/share/rear/backup/NETFS/default/50_make_backup.sh at line 90 where the rsync option is: === (rsync) # make sure that the target is a directory mkdir -p $v "$backuparchive" >&2 LINKDEST=`ls -d $BUILD_DIR/outputfs/rear/$HOSTNAME/*/backup/bin 2>/dev/null | tail -n 1 | cut -f1-8 -d"/"` case $LINKDEST in "") LINKDESTOPT="";; *) LINKDESTOPT="--link-dest=$LINKDEST";; esac Log $BACKUP_PROG $v "${BACKUP_RSYNC_OPTIONS[@]}" --one-file-system --delete \ --exclude-from=$TMP_DIR/backup-exclude.txt --delete-excluded ${LINKDESTOPT} \ $(cat $TMP_DIR/backup-include.txt) "$backuparchive" $BACKUP_PROG $v "${BACKUP_RSYNC_OPTIONS[@]}" --one-file-system --delete \ --exclude-from=$TMP_DIR/backup-exclude.txt --delete-excluded ${LINKDESTOPT} \ $(cat $TMP_DIR/backup-include.txt) "$backuparchive" ;; === Saves a ton of space on backup media...

…

On 01/11/2017 06:01 AM, Johannes Meixner wrote: @lrirwin <https://github.com/lrirwin> can you explain what you mean with "backup folders contain thousands of files"? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1161 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABZTrFk9CV7El8dl9DUvStHAPHxcrkRQks5rRLafgaJpZM4LfuZ5>.

[jsmeix]

By guessing from your comments it seems
you use rysnc as backup method and that results
very many individual files while I use 'tar' so that
I get only a few tar archives so that for me the
whole difference is about a few millisecods.

I really helps to provide the information as requested in
https://github.com/rear/rear/blob/master/.github/ISSUE_TEMPLATE.md
to aviod wasting time with guesswork.

Furthermore preferably provide your proposed changes
as GitHub pull requests which also makes it easier for us
to see what you actually like to change.

Finally see
https://github.com/rear/rear/wiki/Coding-Style
so that over the time the ReaR code can get better step by step.