Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FS#579 - Rename failure when file both in rom and overlay in mvebu / wrt3200acm, due to kernel 4.9 or ubi? #5608

Closed
openwrt-bot opened this issue Mar 1, 2017 · 5 comments
Labels

Comments

@openwrt-bot
Copy link

openwrt-bot commented Mar 1, 2017

hnyman:

LEDE master r3623 mvebu WRT3200ACM

Router fails to rename files that are both in /rom and in /overlay, i.e. they were part of the original flash and have been modified later or were included in sysupgrade.

Rename fails.
Deleting the same file succeeds.
Copying is also ok.

(Deleting the file and copying a new one into its place does not enable rename, which still fails.)

I noticed this in mvebu / WRT3200ACM, but the bug may also concern all mvebu devices, all kernel 4.9 users and/or all ubi filesystem users.

I did originally build my settings with 17.01 with 4.4 and/or master with k4.4 and noticed no problems at that stage, so this may have surfaced at the bump to 4.9, or at the last bump to 4.9.13. (But I have only used mvebu for a few days, and haven't yet verified the 4.9.10 or lede-17.01 behaviour)

I originally noticed this when debugging the new opkg, but then realised that this is a wider file-system level problem. I verified the problem by flashing a pure buildbot snapshot image without any additional settings.

Rename fails
* in console

root@LEDE:~# mv /etc/dropbear/dropbear_rsa_host_key /etc/dropbear/dropbear_rsa_host_key-x
mv: can't rename '/etc/dropbear/dropbear_rsa_host_key': Invalid argument

* in script:

root@LEDE:/etc/config# /etc/init.d/luci_statistics restart
mv: can't rename '/etc/collectd.conf': Invalid argument

* "rename" function call in a C program (strace from opkg):

rename("/etc/config/bcp38", "/etc/config/bcp38-opkg") = -1 EINVAL (Invalid argument)

No such problem in ar71xx device with the same r3623 build but with kernel 4.4 (and without NAND & ubi).

Steps to reproduce:

  • Try to rename a file that is both in firmware and in overlay. E.g. a config file related to a package included in the build

I used /etc/dropbear/dropbear_rsa_host_key in the next example, as that is a file included in the clean snapshot firmware.

Reboot (SNAPSHOT, r3623-339de82)

root@LEDE:~# mv /etc/dropbear/dropbear_rsa_host_key /etc/dropbear/dropbear_rsa_host_key-x
mv: can't rename '/etc/dropbear/dropbear_rsa_host_key': Invalid argument

root@LEDE:~# ls -l /rom/etc/dropbear/dropbear_rsa_host_key
-rw-r--r-- 1 root root 0 Mar 1 06:10 /rom/etc/dropbear/dropbear_rsa_host_key

root@LEDE:~# ls -l /overlay/upper/etc/dropbear/dropbear_rsa_host_key
-rw------- 1 root root 805 Mar 1 06:10 /overlay/upper/etc/dropbear/dropbear_rsa_host_key

root@LEDE:# rm /etc/dropbear/dropbear_rsa_host_key
root@LEDE:
# ls -l /overlay/upper/etc/dropbear/dropbear_rsa_host_key
c--------- 1 root root 0, 0 Mar 1 14:34 /overlay/upper/etc/dropbear/dropbear_rsa_host_key

root@LEDE:# cp /etc/dropbear/dropbear_rsa_host_key-x /etc/dropbear/dropbear_rsa_host_key
root@LEDE:
# mv /etc/dropbear/dropbear_rsa_host_key /etc/dropbear/dropbear_rsa_host_key-x
mv: can't rename '/etc/dropbear/dropbear_rsa_host_key': Invalid argument

Rename also fails for newly created files in /overlay. E.g. a file does not yet exist in /overlay, but is created there due to editing:

root@LEDE:/etc# ls /overlay/upper/etc/
board.json dropbear firewall.user switch_boot_partition.sh urandom.seed
config ethers fw_env.config uci-defaults
root@LEDE:/etc# nano /etc/opkg.conf
root@LEDE:/etc# mv /etc/opkg.conf /etc/opkg.conf2
mv: can't rename '/etc/opkg.conf': Invalid argument

@openwrt-bot
Copy link
Author

openwrt-bot commented Mar 1, 2017

hnyman:

Renaming works ok in 17.01.0

I flashed the official 17.01.0 build without settings, and tested renaming. Works ok.

So, this is something new. Likely related to the 4.9 bump.

Reboot (17.01.0, r3205-59508e3)

root@LEDE:/etc# ls -l /overlay/upper/etc/dropbear/
-rw------- 1 root root 804 Feb 20 17:13 dropbear_rsa_host_key
root@LEDE:/etc# ls -l /rom/etc/dropbear/dropbear_rsa_host_key
-rw-r--r-- 1 root root 0 Feb 20 17:13 /rom/etc/dropbear/dropbear_rsa_host_key

root@LEDE:/etc# mv /etc/dropbear/dropbear_rsa_host_key /etc/dropbear/dropbear_rsa_host_key-x
root@LEDE:/etc#

@openwrt-bot
Copy link
Author

openwrt-bot commented Mar 1, 2017

hnyman:

r3509-2374549916 kernel 4.4 : rename works ok.
r3510-2bf9ea6a31 kernel 4.9 : fails to rename

I compiled the last commit where mvebu is with kernel 4.4 and rename works normally. The next commit bumping mvebu to 4.9 makes rename to fail. So this bug came with kernel 4.9.

I have no idea if this is restricted to mvebu or if this concerns all 4.9 targets (using ubifs?).

Supposedly kernel 4.9 supports ubifs rename "built-in", so the patches for that (052-...) were dropped from LEDE's generic patches-4.9

Linux kernel's ubifs rename support commits have been committed on 2016-10-02:
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/log/fs/ubifs?h=linux-4.9.y

@openwrt-bot
Copy link
Author

openwrt-bot commented Mar 2, 2017

hnyman:

I think that I have identified the culprit commit causing the rename bug for ubifs with overlayfs, but I am not sure about the fix. Deleting the offending two lines?

I compared the respective files in 4.4 and 4.9. I took the files that the kernel 4.4 patches 052-... handled, meaning fs/ubifs/ubifs.h, journal.c and dir.c.

  • 4.4 patched journal.c equals 4.9 journal.c
  • 4.4 patched ubifs.h has only trivial differences to 4.9
  • dir.c has otherwise only trivial changes (e.g. mutex lock changes), but there is one significant change in 4.9, where "return -EINVAL" has been inserted.

https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/fs/ubifs/dir.c?h=linux-4.9.y#n1091

struct timespec time; unsigned int uninitialized_var(saved_nlink);
if (flags & ~RENAME_NOREPLACE)
	return -EINVAL;

/*

The change is here:
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/fs/ubifs/dir.c?h=linux-4.9.y&id=f03b8ad8d38634d13e802165cc15917481b47835

Full commit:
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?h=linux-4.9.y&id=f03b8ad8d38634d13e802165cc15917481b47835

fs: support RENAME_NOREPLACE for local filesystems

This is trivial to do:

  • add flags argument to foo_rename()
  • check if flags doesn't have any other than RENAME_NOREPLACE
  • assign foo_rename() to .rename2 instead of .rename

Filesystems converted:

affs, bfs, exofs, ext2, hfs, hfsplus, jffs2, jfs, logfs, minix, msdos,
nilfs2, omfs, reiserfs, sysvfs, ubifs, udf, ufs, vfat.

(overlayfs is not mentioned in the list.)

@openwrt-bot
Copy link
Author

openwrt-bot commented Mar 2, 2017

hnyman:

Fix proposal here. Any comments from ubifs / overlayfs specialists?

I patched my system with this small change that reverts things more or less to the state they were for ubifs with 4.4:

target/linux/generic/patches-4.9/552-ubifs-overlayfs-fix-rename.patch

--- a/fs/ubifs/dir.c +++ b/fs/ubifs/dir.c @@ -1088,8 +1088,10 @@ struct timespec time; unsigned int uninitialized_var(saved_nlink);
  • /*
    if (flags & ~RENAME_NOREPLACE)
    return -EINVAL;

  • */

    /*

    • Budget request settings: deletion direntry, new direntry, removing

At the first glance it looks ok:
Renaming a problematic file works again, and both the deletion record and the newly named file are visible in /overlay. And /etc looks normal:

root@LEDE:~# mv /etc/dropbear/dropbear_rsa_host_key /etc/dropbear/dropbear_rsa_host_key-x root@LEDE:~# ls -l /rom/etc/dropbear/ -rw-r--r-- 1 root root 0 Mar 2 10:09 dropbear_rsa_host_key root@LEDE:~# ls -l /overlay/upper/etc/dropbear/ -rwxr--r-- 1 root root 959 Feb 18 18:09 authorized_keys c--------- 1 root root 0, 0 Mar 2 10:16 dropbear_rsa_host_key -rw------- 1 root root 804 Feb 20 17:13 dropbear_rsa_host_key-x root@LEDE:~# ls -l /etc/dropbear/ -rwxr--r-- 1 root root 959 Feb 18 18:09 authorized_keys -rw------- 1 root root 804 Feb 20 17:13 dropbear_rsa_host_key-x

@openwrt-bot
Copy link
Author

openwrt-bot commented Mar 6, 2017

nbd:

Fixed in r3651-697ff33771

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant