Can not see any file in the directory via smb when the directory is a mount point, but the files indeed exist. #436

liu-fuming · 2023-07-05T17:08:54Z

I share a directory "myshare". In that directory, I create a directory "mount" which is used as a mount point used to mount "/dev/sdb1". last week,it ran good.But now,I can not see any file in the "mount" via smb. But the files is indeed in the "mount" directory when I "ls" the "mount" directory . I use archlinux, the kernel is updated from 6.3 to 6.4.1.

namjaejeon · 2023-07-05T23:18:03Z

Let me check it. Please use cifsd-team/ksmbd github till fixing this.

bacon-cheeseburger · 2023-07-07T15:00:02Z

I discovered the same problem after going from kernel 6.3.8 to 6.4.2. Loading back into 6.3.8 got it working again. Is there already a suspected bad commit?

falsovsky · 2023-07-08T12:28:55Z

For me its appearing empty while sharing a btrfs partition, a xfs one is working, not sure it its related.

bacon-cheeseburger · 2023-07-08T13:55:40Z

For me its appearing empty while sharing a btrfs partition, a xfs one is working, not sure it its related.

Here it's /dev/sdb1, which is an external hdd ext4 connected by usb.

falsovsky · 2023-07-08T14:26:10Z

Mine is also an external usb disk.

costel78 · 2023-07-08T15:07:31Z

The issue appeared on kernel 6.3.10. 6.3.9 is fine. I did a git bisect and commit ef6cc1c465786b3e4e735d1ab8b616b3e94402df seems to be the culprit.

ef6cc1c465786b3e4e735d1ab8b616b3e94402df is the first bad commit
commit ef6cc1c465786b3e4e735d1ab8b616b3e94402df
Author: Namjae Jeon linkinjeon@kernel.org
Date: Fri Apr 21 16:09:01 2023 +0900

ksmbd: fix racy issue from using ->d_parent and ->d_name

[ Upstream commit 74d7970febf7e9005375aeda0df821d2edffc9f7 ]

Al pointed out that ksmbd has racy issue from using ->d_parent and ->d_name
in ksmbd_vfs_unlink and smb2_vfs_rename(). and use new lock_rename_child()
to lock stable parent while underlying rename racy.
Introduce vfs_path_parent_lookup helper to avoid out of share access and
export vfs functions like the following ones to use
vfs_path_parent_lookup().
 - rename __lookup_hash() to lookup_one_qstr_excl().
 - export lookup_one_qstr_excl().
 - export getname_kernel() and putname().

vfs_path_parent_lookup() is used for parent lookup of destination file
using absolute pathname given from FILE_RENAME_INFORMATION request.

Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Signed-off-by: Steve French <stfrench@microsoft.com>
Stable-dep-of: 40b268d384a2 ("ksmbd: add mnt_want_write to ksmbd vfs functions")
Signed-off-by: Sasha Levin <sashal@kernel.org>

namjaejeon · 2023-07-08T23:53:40Z

Changed the path lookup part to fix various racy issues. You can not lookup cross mount points such as usb disks in share. Same with NFS. In case of NFS, the usb disk should be registered as a share in the share directory and try mount using client. Question is that.... Is this issue important to you?

namjaejeon · 2023-07-09T00:10:23Z

e.g. Add share sections for each usb disks. and mount each shares like NFS.

[files]
        comment = contents share
        path = /mnt/
        read only = no

[sda1]
        comment = contents share
        path = /mnt/sda1
        read only = no

I know this is also a problem with ksmbd, but it's easier to fix than the cross mount lookup as before. As before, we are checking if cross mount lookup is possible. It will take some more time.

falsovsky · 2023-07-09T00:13:23Z

$ mount|grep external                                              
/dev/sdc1 on /mnt/external type btrfs (rw,relatime,compress=zstd:9,space_cache=v2,subvolid=5,subvol=/)

Mine is:

[external]
   comment = External
   path = /mnt/external
   read only = no

The external share appears on Windows 10, but its empty, but in reality it isn't empty.

It used to work, but stopped in recent kernels.

namjaejeon · 2023-07-09T00:27:16Z

@falsovsky Yes, I can fix it easily. But I am wondering,

I share a directory "myshare". In that directory, I create a directory "mount" which is used as a mount point used to mount "/dev/sdb1".

if anyone really needed a cross mount lookup as mentioned above.

bacon-cheeseburger · 2023-07-09T01:09:18Z

This problem is not only for external usb. I found the same problem on a rack server on /dev/sg1, which is an internal 18TB 3.5" hdd ext4. One dir works fine while another appears empty on that same drive.

I create a single root share by bindmount'ing dirs from different hdd's to the shared dir. This way my share only contains dirs I want and doesn't force me to share the entire contents of a hdd mount. I also never have to touch a samba config. It's worked great til this problem.

It would seem if commit ef6cc1c465786b3e4e735d1ab8b616b3e94402df tries to fix a race condition problem, but breaks otherwise working systems in the process, then that commit is problematic. The best solution would be one that resolves the race condition and does not break otherwise working setups.

namjaejeon · 2023-07-09T01:54:08Z

@bacon-cheeseburger You're right. It's not just an external problem. It's all a problem if you have a cross mount. Let me check it. give the time to me.

namjaejeon · 2023-07-09T14:53:36Z

Can you check if this patch fix your issue ? 5f3286b

bacon-cheeseburger · 2023-07-09T15:24:58Z

Can you check if this patch fix your issue ? 5f3286b

Which kernel should this be tested against? It fails to apply against 6.4.2.

namjaejeon · 2023-07-10T08:27:54Z

@bacon-cheeseburger Okay, Please try this patch.

For linux kernel mainline 6.4 or 6.5-rc1
namjaejeon/smb3-kernel@7d8eb8f
For github ksmbd out of tree source.
f6922e0

Thanks.

costel78 · 2023-07-10T10:47:26Z

Hello,

I tested patch 7d8eb8f against kernel 6.4.2. It solves the cross mount issue - folders are no longer empty on smb share, but, unfortunately, it seems it introduce another error:

iul 10 13:34:22 mini.cweb.ro sshd[877]: Accepted publickey for root from 192.168.78.254 port 37554 ssh2: [...]
iul 10 13:34:22 mini.cweb.ro sshd[877]: pam_unix(sshd:session): session opened for user root(uid=0) by root(uid=0)
iul 10 13:34:22 mini.cweb.ro systemd-logind[349]: New session 5 of user root.
iul 10 13:34:22 mini.cweb.ro systemd[1]: Started session-5.scope.
iul 10 13:34:57 mini.cweb.ro kernel: CIFS: No dialect specified on mount. Default has changed to a more secure dialect, SMB2.1 or later (e.g. SMB3.1.1), from CIFS (SMB1). To use the less secure SMB1 dialect to access old servers which do not support SMB3.1.1 (or even SMB3 or SMB2.1) specify vers=1.0 on mount.
iul 10 13:34:57 mini.cweb.ro kernel: CIFS: Attempting to mount //mini/nas
iul 10 13:35:19 mini.cweb.ro kernel: BUG: kernel NULL pointer dereference, address: 0000000000000028
iul 10 13:35:19 mini.cweb.ro kernel: #PF: supervisor read access in kernel mode
iul 10 13:35:19 mini.cweb.ro kernel: #PF: error_code(0x0000) - not-present page
iul 10 13:35:19 mini.cweb.ro kernel: PGD 0 P4D 0
iul 10 13:35:19 mini.cweb.ro kernel: Oops: 0000 [#1] PREEMPT SMP NOPTI
iul 10 13:35:19 mini.cweb.ro kernel: CPU: 1 PID: 868 Comm: kworker/1:6 Not tainted 6.4.2-gentoo-mini #1
iul 10 13:35:19 mini.cweb.ro kernel: Hardware name: ASRock H670M-ITX/ax/H670M-ITX/ax, BIOS 14.03 06/27/2023
iul 10 13:35:19 mini.cweb.ro kernel: Workqueue: ksmbd-io handle_ksmbd_work
iul 10 13:35:19 mini.cweb.ro kernel: RIP: 0010:generic_fillattr+0x11/0xf0
iul 10 13:35:19 mini.cweb.ro kernel: Code: 00 00 00 c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5 41 56 41 55 49 89 fd 41 54 49 89 d4 53 <48> 8b 46 28 48 89 f3 8b 56 04 48 8b b0 80 04 00 00 e8 e9 e0 03 00
iul 10 13:35:19 mini.cweb.ro kernel: RSP: 0018:ffffa463083bfbd0 EFLAGS: 00010246
iul 10 13:35:19 mini.cweb.ro kernel: RAX: 0000000000000000 RBX: ffffa463083bfc98 RCX: ffffa463083bfc98
iul 10 13:35:19 mini.cweb.ro kernel: RDX: ffffa463083bfcb0 RSI: 0000000000000000 RDI: ffffffffa5347c50
iul 10 13:35:19 mini.cweb.ro kernel: RBP: ffffa463083bfbf0 R08: ffff8cc7d124e6c1 R09: ffffffffa4ece685
iul 10 13:35:19 mini.cweb.ro kernel: R10: ffffffffa4f0f1ec R11: ffffa463083bfdd8 R12: ffffa463083bfcb0
iul 10 13:35:19 mini.cweb.ro kernel: R13: ffffffffa5347c50 R14: ffff8cc83600fe00 R15: 0000000000000001
iul 10 13:35:19 mini.cweb.ro kernel: FS: 0000000000000000(0000) GS:ffff8ccb3f700000(0000) knlGS:0000000000000000
iul 10 13:35:19 mini.cweb.ro kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
iul 10 13:35:19 mini.cweb.ro kernel: CR2: 0000000000000028 CR3: 0000000177674000 CR4: 0000000000750ee0
iul 10 13:35:19 mini.cweb.ro kernel: PKRU: 55555554
iul 10 13:35:19 mini.cweb.ro kernel: Call Trace:
iul 10 13:35:19 mini.cweb.ro kernel:
iul 10 13:35:19 mini.cweb.ro kernel: ? show_regs+0x60/0x70
iul 10 13:35:19 mini.cweb.ro kernel: ? __die+0x1f/0x70
iul 10 13:35:19 mini.cweb.ro kernel: ? page_fault_oops+0x15b/0x430
iul 10 13:35:19 mini.cweb.ro kernel: ? ext4_xattr_get+0x90/0x240
iul 10 13:35:19 mini.cweb.ro kernel: ? exc_page_fault+0x332/0x610
iul 10 13:35:19 mini.cweb.ro kernel: ? ext4_xattr_user_get+0x2b/0x40
iul 10 13:35:19 mini.cweb.ro kernel: ? asm_exc_page_fault+0x27/0x30
iul 10 13:35:19 mini.cweb.ro kernel: ? generic_fillattr+0x11/0xf0
iul 10 13:35:19 mini.cweb.ro kernel: ksmbd_vfs_fill_dentry_attrs+0x39/0xe0
iul 10 13:35:19 mini.cweb.ro kernel: ? kfree+0x62/0xb0
iul 10 13:35:19 mini.cweb.ro kernel: ? smb2_populate_readdir_entry+0xb0/0x710
iul 10 13:35:19 mini.cweb.ro kernel: ksmbd_populate_dot_dotdot_entries+0xdb/0x1a0
iul 10 13:35:19 mini.cweb.ro kernel: ? __pfx_smb2_populate_readdir_entry+0x10/0x10
iul 10 13:35:19 mini.cweb.ro kernel: ? smb_strndup_from_utf16+0x133/0x250
iul 10 13:35:19 mini.cweb.ro kernel: ? smb_strndup_from_utf16+0x1ce/0x250
iul 10 13:35:19 mini.cweb.ro kernel: smb2_query_dir+0x2ac/0x5a0
iul 10 13:35:19 mini.cweb.ro kernel: handle_ksmbd_work+0x163/0x480
iul 10 13:35:19 mini.cweb.ro kernel: process_one_work+0x1b8/0x360
iul 10 13:35:19 mini.cweb.ro kernel: worker_thread+0x48/0x3d0
iul 10 13:35:19 mini.cweb.ro kernel: ? __pfx_worker_thread+0x10/0x10
iul 10 13:35:19 mini.cweb.ro kernel: kthread+0xea/0x120
iul 10 13:35:19 mini.cweb.ro kernel: ? __pfx_kthread+0x10/0x10
iul 10 13:35:19 mini.cweb.ro kernel: ret_from_fork+0x29/0x50
iul 10 13:35:19 mini.cweb.ro kernel:
iul 10 13:35:19 mini.cweb.ro kernel: Modules linked in:
iul 10 13:35:19 mini.cweb.ro kernel: CR2: 0000000000000028
iul 10 13:35:19 mini.cweb.ro kernel: ---[ end trace 0000000000000000 ]---
iul 10 13:35:19 mini.cweb.ro kernel: RIP: 0010:generic_fillattr+0x11/0xf0
iul 10 13:35:19 mini.cweb.ro kernel: Code: 00 00 00 c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5 41 56 41 55 49 89 fd 41 54 49 89 d4 53 <48> 8b 46 28 48 89 f3 8b 56 04 48 8b b0 80 04 00 00 e8 e9 e0 03 00
iul 10 13:35:19 mini.cweb.ro kernel: RSP: 0018:ffffa463083bfbd0 EFLAGS: 00010246
iul 10 13:35:19 mini.cweb.ro kernel: RAX: 0000000000000000 RBX: ffffa463083bfc98 RCX: ffffa463083bfc98
iul 10 13:35:19 mini.cweb.ro kernel: RDX: ffffa463083bfcb0 RSI: 0000000000000000 RDI: ffffffffa5347c50
iul 10 13:35:19 mini.cweb.ro kernel: RBP: ffffa463083bfbf0 R08: ffff8cc7d124e6c1 R09: ffffffffa4ece685
iul 10 13:35:19 mini.cweb.ro kernel: R10: ffffffffa4f0f1ec R11: ffffa463083bfdd8 R12: ffffa463083bfcb0
iul 10 13:35:19 mini.cweb.ro kernel: R13: ffffffffa5347c50 R14: ffff8cc83600fe00 R15: 0000000000000001
iul 10 13:35:19 mini.cweb.ro kernel: FS: 0000000000000000(0000) GS:ffff8ccb3f700000(0000) knlGS:0000000000000000
iul 10 13:35:19 mini.cweb.ro kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
iul 10 13:35:19 mini.cweb.ro kernel: CR2: 0000000000000028 CR3: 0000000177674000 CR4: 0000000000750ee0
iul 10 13:35:19 mini.cweb.ro kernel: PKRU: 55555554
iul 10 13:35:19 mini.cweb.ro kernel: note: kworker/1:6[868] exited with irqs disabled
iul 10 13:35:30 mini.cweb.ro systemd-logind[349]: Power key pressed short.

My setup:
/mnt/nas is shared and it contain
filme - > /dev/md0, ext4
info -> /dev/nvme1n1p1, ntfs3
linux -> /dev/nvme0n1p3, ext4

Error appear sporadically, running, for example, ls -alsR /mnt/usb (there smb share was mounted for testing), or simply browsing the directory structure with mc.
The system freezes, and after forced reboot, on every mounted partition fsck found small issues.

Thank you!

namjaejeon · 2023-07-10T11:52:38Z

@costel78 Thanks for your check!

Could you please check this change fix your kernel oops ?

linkinjeon@linkinjeon-System-Product-Name:~/linux/namjae/smb3-kernel$ git diff
diff --git a/fs/smb/server/vfs.c b/fs/smb/server/vfs.c
index a73420c75958..52e1d312aa67 100644
--- a/fs/smb/server/vfs.c
+++ b/fs/smb/server/vfs.c
@@ -116,7 +116,6 @@ static int ksmbd_vfs_path_lookup_locked(struct ksmbd_share_config *share_conf,
                goto err_out;
        }
 
-       path_put(parent_path);
        putname(filename);
 
        return 0;

costel78 · 2023-07-10T15:26:06Z

@namjaejeon
It working perfectly, at least in my case, folders content is as it should be and no more kernel panics.

Thank you very much for your work!

bacon-cheeseburger · 2023-07-11T03:35:57Z

The fix works here as well. I didn't get an error with the unmodified patch but have since applied the secondary one on top and still working good so far.

namjaejeon · 2023-07-11T03:50:27Z

@costel78 @bacon-cheeseburger Thanks for your check. I have combined secondary patch into first patch.
namjaejeon/smb3-kernel@0cd99ca

I will apply this patch to the mainline.
Thanks again.

bacon-cheeseburger · 2023-07-11T04:02:46Z

Thank you for quickly resolving this issue!

mcirsta · 2023-07-11T21:15:24Z

Good to know that the issue was fixed. It also affected me as I'm using Arch and upgraded to the latest 6.4 kernel. I have an external USB drive that I'm sharing using ksmb and all was fine till 6.4

Zheka17 · 2023-07-24T16:56:31Z

The issue is still there with the latest 6.4 kernel on Debian.
...Just realized that ksmbd-tools have not been updated in debian repos yet..

namjaejeon · 2023-07-24T23:43:08Z

This patch is not merged into mainline yet. There is a process to apply it. We need the more time.

database64128 · 2023-07-25T02:15:46Z

Am I alone in thinking that the offending commit should be reverted right away? All of my ksmbd deployments involve non-root mounts (e.g. /dev/sda mounted as /mnt/hdd_18t and shared as hdd_18t). So the current state is completely unusable for me, and I had to revert all of my systems back to 6.3.9. This is the first time I had to revert back to older kernel versions on Arch Linux.

mcirsta · 2023-07-28T19:37:30Z

Am I alone in thinking that the offending commit should be reverted right away? All of my ksmbd deployments involve non-root mounts (e.g. /dev/sda mounted as /mnt/hdd_18t and shared as hdd_18t). So the current state is completely unusable for me, and I had to revert all of my systems back to 6.3.9. This is the first time I had to revert back to older kernel versions on Arch Linux.

I fully agree, as an Arch user it really created problems for me as I was relying on this to share my files from my USB attached drive.

namjaejeon · 2023-07-30T00:38:10Z

This fixes is applied to the mainline. It will propagate to the stable kernel versions next week. You should install the latest ksmbd-tools 3.4.9 version for this issue.

mcirsta · 2023-07-30T09:49:41Z

This fixes is applied to the mainline. It will propagate to the stable kernel versions next week. You should install the latest ksmbd-tools 3.4.9 version for this issue.

Thanks. This is good to know. In the meantime I switched to kernel-lts in Arch and with this kernel (6.1x) things are back to normal.

namjaejeon · 2023-08-03T12:01:28Z

Hi, the patch for this was applied to 6.4.8 stable kernel version and linux-6.5-rc version. Please close THE ISSUE if it is clear.

falsovsky · 2023-08-03T14:22:43Z

Hi @namjaejeon I just compiled 6.4.8, but my problem still persists.
I have an external USB HDD in BTRFS, I'm sharing its root folder, but the share appears empty when I mount it.

namjaejeon · 2023-08-03T20:12:51Z

@falsovsky Can you try it again after upgrading the latest ksmbd-tools 3.4.9 version?

falsovsky · 2023-08-03T20:41:05Z

@namjaejeon Working! Thanks. My problem is fxed.

namjaejeon · 2023-08-05T04:48:58Z

@falsovsky Thanks for your check!

Closed this issue. Please reopen it if problem is still there.
Thanks!

namjaejeon closed this as completed Aug 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can not see any file in the directory via smb when the directory is a mount point, but the files indeed exist. #436

Can not see any file in the directory via smb when the directory is a mount point, but the files indeed exist. #436

liu-fuming commented Jul 5, 2023

namjaejeon commented Jul 5, 2023

bacon-cheeseburger commented Jul 7, 2023

falsovsky commented Jul 8, 2023

bacon-cheeseburger commented Jul 8, 2023

falsovsky commented Jul 8, 2023

costel78 commented Jul 8, 2023

namjaejeon commented Jul 8, 2023

namjaejeon commented Jul 9, 2023

falsovsky commented Jul 9, 2023 •

edited

namjaejeon commented Jul 9, 2023

bacon-cheeseburger commented Jul 9, 2023 •

edited

namjaejeon commented Jul 9, 2023 •

edited

namjaejeon commented Jul 9, 2023

bacon-cheeseburger commented Jul 9, 2023

namjaejeon commented Jul 10, 2023 •

edited

costel78 commented Jul 10, 2023

namjaejeon commented Jul 10, 2023

costel78 commented Jul 10, 2023

bacon-cheeseburger commented Jul 11, 2023

namjaejeon commented Jul 11, 2023

bacon-cheeseburger commented Jul 11, 2023

mcirsta commented Jul 11, 2023

Zheka17 commented Jul 24, 2023 •

edited

namjaejeon commented Jul 24, 2023

database64128 commented Jul 25, 2023

mcirsta commented Jul 28, 2023

namjaejeon commented Jul 30, 2023

mcirsta commented Jul 30, 2023

namjaejeon commented Aug 3, 2023

falsovsky commented Aug 3, 2023

namjaejeon commented Aug 3, 2023

falsovsky commented Aug 3, 2023

namjaejeon commented Aug 5, 2023

Can not see any file in the directory via smb when the directory is a mount point, but the files indeed exist. #436

Can not see any file in the directory via smb when the directory is a mount point, but the files indeed exist. #436

Comments

liu-fuming commented Jul 5, 2023

namjaejeon commented Jul 5, 2023

bacon-cheeseburger commented Jul 7, 2023

falsovsky commented Jul 8, 2023

bacon-cheeseburger commented Jul 8, 2023

falsovsky commented Jul 8, 2023

costel78 commented Jul 8, 2023

namjaejeon commented Jul 8, 2023

namjaejeon commented Jul 9, 2023

falsovsky commented Jul 9, 2023 • edited

namjaejeon commented Jul 9, 2023

bacon-cheeseburger commented Jul 9, 2023 • edited

namjaejeon commented Jul 9, 2023 • edited

namjaejeon commented Jul 9, 2023

bacon-cheeseburger commented Jul 9, 2023

namjaejeon commented Jul 10, 2023 • edited

costel78 commented Jul 10, 2023

namjaejeon commented Jul 10, 2023

costel78 commented Jul 10, 2023

bacon-cheeseburger commented Jul 11, 2023

namjaejeon commented Jul 11, 2023

bacon-cheeseburger commented Jul 11, 2023

mcirsta commented Jul 11, 2023

Zheka17 commented Jul 24, 2023 • edited

namjaejeon commented Jul 24, 2023

database64128 commented Jul 25, 2023

mcirsta commented Jul 28, 2023

namjaejeon commented Jul 30, 2023

mcirsta commented Jul 30, 2023

namjaejeon commented Aug 3, 2023

falsovsky commented Aug 3, 2023

namjaejeon commented Aug 3, 2023

falsovsky commented Aug 3, 2023

namjaejeon commented Aug 5, 2023

falsovsky commented Jul 9, 2023 •

edited

bacon-cheeseburger commented Jul 9, 2023 •

edited

namjaejeon commented Jul 9, 2023 •

edited

namjaejeon commented Jul 10, 2023 •

edited

Zheka17 commented Jul 24, 2023 •

edited