Newly created folders with wrong name

[matteocontrini]

Hi,
I'm experiencing an issue when uploading to a custom path.
For example, uploading to:

/existing/path/newfolder/filename.ext

sometimes creates new folders like:

/existing 1/path/newfolder/filename.ext
/existing/path 1/newfolder/filename.ext
/existing/path/newfolder 1/filename.ext

This happens with both normal and multipart uploads.
It happens only one time out of 10 (aproximately), but when it does, it creates new folder I have to delete manually.

[fkalis]

When you use the -f option, the script tries to find the specified folder. If it does not find the folder, it creates a new one using the "rename" strategy. That's the reason why the "1" occurs in the path.

I'm not sure why the script can't find the folder, if it already exists. Do your folder names include any special characters?

[matteocontrini]

It includes a space, but nothing special, only ASCII characters. It happens randomly...

[fkalis]

I have updated the script and merged the feature branch into the master. Now the script does not create a new folder with wrong name, but fails instead.

For now you can check the exit code of the script to build a temporary workaround.

I have tried to reproduce the problem, but for me it never tried to create an already existing folder.

Can you post the anonymized output of a failing execution of

./onedrive-upload -d -f "existing/path/newfolder" "filename.ext"

Do you have a bad internet connection?

[matteocontrini]

Reproduced the issue after 5-6 times I ran the script.

Searching for '2015_12_03' in '8FDC0AB78D585D02!264208'
Creating folder '2015_12_03' in '8FDC0AB78D585D02!264208'
An error has occurred while creating folder '2015_12_03' in '8FDC0AB78D585D02!264208' (An item with the same name already exists under the parent)

The 2015_12_03 directory is actually existing.

The phase "Searching for..." takes a lot of time (10-12 seconds for every "search"). The one in the partial output above took even more than that so it probably timed out at a system level (which should be about 20 seconds).

No idea how that's possible since the script is running on a DigitalOcean droplet, with an average speed of 600 DL/25 UL speed (to OneDrive as well). I can't test ping since api.onedrive.com has ICMP disabled.

[matteocontrini]

I've seen the second commit now, I'll try with that one as well.

[fkalis]

Do you have many subfolders in the folder represented by '8FDC0AB78D585D02!264208'?

[matteocontrini]

Sorry, I was wrong. I have probably more than 100 folders in that one.
(edited)

[fkalis]

That might be the reason. The API returns only 200 items when fetching the list of children and provides a follow-up url, which is not yet evaluated by the script.

Do you have more than 200 folders in the mentioned folder?

[matteocontrini]

Checked now, 90 directories...
Is the response slower with more children?

[fkalis]

Maybe. At least it is larger ;-)

But I have found a filter option. Maybe I can use that to ask the server for the specified folder name.

I'll keep you updated.

[fkalis]

Since commit a136b14 the script only downloads the necessary meta data from the server. Hopefully this speeds the initial folder traversal up when downloading the children of folders that contain a lot of children.

And as you might have noticed, there is another commit (68c07ab), that delegates the main filter task to the server. Hopefully this helps.

btw: Does you folder with 90 subfolders also contain files?

[matteocontrini]

btw: Does you folder with 90 subfolders also contain files?

Zero files in that folder.

I should say again that the issue happened in the root directory as well.
And I probably missed the "custom root" option, that I'll set now directly to the folder with 90 subfolders, so that the script avoids making some requests.

I'll reopen this issue if it happens again even with the recent changes.