Parsing a .json configuration for WebDAV does not work

[moellney]

• I have searched open and closed issues for duplicates.

---

Environment info

• Duplicati version: 2.0.4.12
• Operating system: Windows
• Backend: WebDAV

Description

I tried to configure a backup by using a .json config file that I saved before.
The re-configuration from this .json file failed.

Steps to reproduce

1. Create a backup configuration for WebDAV for the "1&1 online Speicher"
2. Save the configuration to a .json file
3. Remove the backup configuration (but not the backup files) (simulation of accessing the backup from a new computer)
4. Create a new backup from this .json file

• Actual result:
  The relevant line in the .json is probably:
  "TargetURL": "webdavs://sd2dav.1und1.de/Duplicati_Test1?auth-username=xxxxxxxxxxx@online.de"
  The setting that were parsed from this line for the WebDAV backend were mixed up:
  Server and Port: online.de /
  Path on Server:
  Username: sd2dav.1und1.de/Duplicati_Test1?auth-username=xxxxxxxxxxx

• Expected result:
  Server and Port: sd2dav.1und1.de
  Path on Server: Duplicati_Test1
  Username: xxxxxxxxxxx@online.de

Screenshots

Debug log

[warwickmm]

I am unable to reproduce this. I don't have access to such an account, but I don't think it matters for this issue.

Here's a screenshot of my test configuration:

Here is the relevant line in the exported JSON:

"TargetURL": "webdav://sd2dav.1und1.de:123/Duplicati_Test1?auth-username=username%40online.de&auth-password=password"

Here's a screenshot of the imported configuration:

Can you post a screenshot of the configuration, as well as the TargetURL line (with authentication censored) from the exported JSON?

[moellney]

Thank your for looking into the topic.

The Target-URL was written in my original post, but I repeat it here:
"TargetURL": "webdavs://sd2dav.1und1.de/Duplicati_Test1?auth-username=ka1234-567@online.de"

However, there is a difference to your example:
my Target-URL uses "webdavs" - I guess the 's' at the end comes from the usage of SSL?
And a second "specialty" might be the '-' in the username in front of @online.de that was not noted in my original post....

The result in the imported field was also noted in the original post.
But anyway the screenshot:

Maybe this gives some more details.

[warwickmm]

I tried to replicate your configuration:

Exporting this yields the following TargetURL:

"TargetURL": "webdavs://sd2dav.1und1.de/Duplicati_Test1?auth-username=ka1234-567%40online.de"

I was able to import this successfully.

However, if I modify the TargetURL by replacing the %40 with @ (to match yours)

TargetURL": "webdavs://sd2dav.1und1.de/Duplicati_Test1?auth-username=ka1234-567@online.de"

I see similar behavior:

Do you recall what version you used when you exported the JSON? I cannot replicate this using version
2.0.4.12. It's possible that a bug in the export functionality has been fixed since then.

[ts678]

Do you recall what version you used when you exported the JSON?

At least in 2.0.4.5, the JSON records it:

{
"CreatedByVersion": "2.0.4.5",

[warwickmm]

Closing this as we cannot reproduce this. My guess is that the JSON was created with a version that didn't properly escape certain characters, and that issue has since been resolved.

[duplicatibot]

This issue has been mentioned on Duplicati. There might be relevant details there:

https://forum.duplicati.com/t/importing-config-file-mixes-up-fields/10489/2

[ts678]

Importing config file mixes up fields confirms the varying export behavior (so varying import bug) that I found when I looked at this:

Yes, if “export passwords” is selected, @ is replaced with %40 in the JSON file and import works as expected.

is issue workaround, however I suspect the reason @warwickmm could not reproduce this is due to either not exporting passwords, or not setting a password to export (which is, oddly, sort of what the screenshots asked for). So problem being seen seems two-part.

On import part, an @ sign is mishandled (even in manual edit test above). On export part, it is typically possible to export an @ sign.

I propose this issue be opened.

[warwickmm]

Thanks @ts678. I can reproduce this now. It seems the key step is to not export passwords for a configuration that has passwords. So far, my debugging the problem with import is pointing to the issue lying in the javascript code. The C# code appears to be deserialzing the JSON properly, and the target URL appears ok. The problem might lie in how the various components are extracted from the target URL.

Also, we should probably fix export so that all paths use the same encoding (in this case, %40 instead of @).

[warwickmm]

My suspicion is this regex

duplicati/Duplicati/Server/webroot/ngax/scripts/services/AppUtils.js

Line 459 in 145ef49

var URL_REGEXP = /([^:]+)\:\/\/(?:(?:([^\:]+)(?:\:?:([^@]*))?\@))?(?:([^\/\?\:]*)(?:\:(\d+))?)(?:\/([^\?]*))?(?:\?(.+))?/;

and the function that follows it.

[warwickmm]

It seems that when passwords are not exported, Backup.SanitizeTargetUrl is called. When passwords are exported, this method is not called.

[warwickmm]

Revision c41f2c6 from pull request #3467 appears to be the cause of @ appearing when we don't export passwords. While avoiding regex is much cleaner, the reliance on Uri.QueryParameters (which contains decoded strings) probably breaks some assumptions that our import code in javascript is relying on.

[warwickmm]

Pull request #4293 should fix the problem where export sometimes uses @ instead of %40, which our import code cannot handle. To fully address this issue, we should fix the import code to handle both situations.