[Solved] Direct restore (smbclient with SMB3) broken with smbclient <= 4.7.3

[jensd0e]

Trying a 'direct restore' to a Windows host, using the (default) configuration for smb restore:

$Conf{SmbClientRestoreCmd} = '$smbClientPath \\$host\$shareName $I_option -U $userName -E -d 1 -c tarmode\ full -mSMB3 -Tx -';

The operation stops after trying to create a directory that already exists.

tar:316 tarmode is now full, system, hidden, noreset, quiet > tar:1596 Can't mkdir test: NT_STATUS_OBJECT_NAME_COLLISION

Tested on three setups:
1.) Server: Debian 8.10, BackupPC 4.1.5, smbclient 4.2.14, tar 1.27.1, Hosts: Windows Server 2012 R2, Windows 10 1709
2.) Server: Debian 9.2, BackupPC 4.1.5, smbclient 4.5.12, tar 1.29, Hosts: Windows Server 2012 R2
3.) VM Server: Debian 9.2, BackupPC 4.1.5, smbclient 4.5.12, tar 1.29, Hosts: Windows Server 2016

What I've done and checked so far:

• As BackupPC user, I can connect to the windows host in an interactive smbclient-session.
• While in smbclient-session: I'm able to create directories and put files on the host.
• While in smbclient-session: Trying to create a directory, that already exists, I get the same error as above, what seems legit, since the directory exists.
• Swapped my smb.conf with the vanilla smb.conf
• Extracting files with BackupPC_tarCreate locally does work.
• Piping BackupPC_tarCreate-command to smbclient-command manually and change path options: if the new path does not exists, it is successfully created, but then instantly stops with the error as seen above (since BackupPC tries to create the path as soon as it tries to put the actual file in the new directory).
• Checked permissions for backuppc-smb-user on windows domain/filesystem.
  Although this was not likely, since smbclient-sessions do work.
• Checked another environment: BackupPC 4.1.x on Debian 9.2 and Windows Server 2012 R2 as host on a different subnet - same error occurs. This system is only a couple of weeks old and does not serve any other purpose. A third system, installed today in a VM, also didn't work.

Proposed solution:
Let BackupPC continue to work, if creation of a directory fails just because it already exists. Alternatively do not try to create a directory, if it already exists.

[craigbarratt]

First off, this issue is specific to smbclient, not BackupPC. In looking at the code in smbclient (source3/client/clitar.c#1639), it should check if the directory exists first before trying to create it (see the call to cli_chkpath() 4 lines earlier). I'm not sure why it fails the directory check, but it must have failed for it to attempt the cli_mkdir() that then fails.

In looking at the interactive commands in client.c, if you do a "cd PATH" it calls cli_chkpath(). Does that command print an error? Similarly, the mkdir PATH command first calls cli_chkpath(). It only tries to create the path if that fails. The fact that it prints an error for you means that for some reason cli_chkpath() doesn't return success on that path.

I am guessing there is something wrong with that directory.

[jensd0e]

Hello Craig,
thanks for taking a look at my problem.

I can open a (SMB3) session to my windows host. In this session I can change and create directories as I like.

backuppc@linuxbox:/$ /usr/bin/smbclient \\server\Sharename -U backupoperator% -mSMB3
Domain=[INNOVATION] OS=[] Server=[]
smb: > mkdir newdir
smb: > cd newdir
smb: \newdir>

Also testes to put some files: No Problem.

I did test the direct restore operation on many shares in my environment and even asked a colleague with a different setup to test a direct restore in his environment. So if there is a problem with my directory, it seems to be something that is quite usual in windows environments.

Right now, I'm not sure, if this combination of BackupPC 4.1.x/PoolV4/smbclient 4.5.x/Windows Server does work for anybody. At least no one replied to my call a few days ago. My use-case might also just be an edge case, nobody else is using.

The last time I needed to recover files was in 2013 with BackupPC 3.x - no problem then. So I can't say, when [my system|BackupPC] stopped to work.

One thing I noticed a few minutes ago, upping debug level:

Domain=[INNOVATION] OS=[] Server=[]
XferErr session setup ok
XferErr signed SMB2 message
XferErr signed SMB2 message
XferErr signed SMB2 message
XferErr signed SMB2 message
XferErr signed SMB2 message
XferErr signed SMB2 message
XferErr tconx ok
tar:316 tarmode is now full, system, hidden, noreset, quiet
tar:1045 +++ ./some_dir/
XferErr signed SMB2 message
tar:1596 Can't mkdir some_dir: NT_STATUS_OBJECT_NAME_COLLISION
tar:631 tar_process done, err = 1
readOutput: sysread returns 0 and got EOF (exit ok = , )
XferErr Non-zero exit status from smbclient
tarCreate: Unable to write to output file (Connection reset by peer)
restore failed: BackupPC_tarCreate failed

Why do I get 'SMB2 messages'? The command should use SMB3. But this might just mean nothing.

Any help on this is really appreciated.

PS: Sorry for closing and reopening this. Must have hit the wrong button.

[jensd0e]

@craigbarratt You are right: the issue is a smbclient issue. Today (after Christmas Holidays) I setup an Windows 7 Client with SMB1 enabled (and not domain joined). As soon as I gave the SMB3 option, backup failed. So it's got to be something with smbclient. Where's the place to bring this to someones attention?

Is this the right place to add my findings? Or should I have written to the list? Sorry for closing & reopening. I tend to hit the wrong button... :-(

Logs from two restore operations. The first one has SMB3 option as only difference. I also highlighted a few rows that seem suspicious. Why is OS/Server field empty?

Running: /usr/bin/smbclient \\win7\ShareSMB1 -U backupoperator -E -d 1 -c tarmode\ full -mSMB3 -Tx -
Running: /usr/local/BackupPC/bin/BackupPC_tarCreate -h server -n 1709 -s Server -t -r /software -p /software/ /software/ResetWUEng.bak
Xfer PIDs are now 16965,16966
Domain=[WIN7PC] OS=[] Server=[]
tar:316 tarmode is now full, system, hidden, noreset, quiet
tar:1596 Can't mkdir software: NT_STATUS_OBJECT_NAME_COLLISION
readOutput: sysread returns 0 and got EOF (exit ok = , )
XferErr Non-zero exit status from smbclient
tarCreate: Unable to write to output file (Connection reset by peer)
restore failed: BackupPC_tarCreate failed

Running: /usr/bin/smbclient \\win7\ShareSMB1 -U backupoperator -E -d 1 -c tarmode\ full -Tx -
Running: /usr/local/BackupPC/bin/BackupPC_tarCreate -h server -n 1709 -s Server -t -r /software -p /software/ /software/ResetWUEng.bak
Xfer PIDs are now 17055,17056
Domain=[WIN7PC] OS=[Windows 7 Enterprise 7601 Service Pack 1] Server=[Windows 7 Enterprise 6.1]
tar:316 tarmode is now full, system, hidden, noreset, quiet
tarCreate: Done: 12 files, 956428 bytes, 1 dirs, 0 specials, 0 errors
readOutput: sysread returns 0 and got EOF (exit ok = 1, )

[jensd0e]

Final note on this issue: I switched my VM from Debian 9.2 to testing (buster/sid).
With smbclient 4.7.3 restore is working again with SMB3. Now, I just have to wait for an updated smbclient. :-)
[If this is the issue fixed: https://bugzilla.samba.org/show_bug.cgi?id=13093, it should work with smbclient >= 4.7.1. (https://www.samba.org/samba/history/samba-4.7.1.html) @craigbarratt Please comment, if I'm wrong or close this issue.]

[craigbarratt]

Thanks for tracking this down! Closing the issue.