Interoperability with leading / in key name

[cyberduck]

bccfca5 created the issue

I have a container with a file called "/x/y.pdf"
Cyberduck finds the container and lists the contents, but it shows as y.pdf
When I try to download, sync, view, get status of the file it always fails with a 404 because https://identity.api.rackspacecloud.com//y.pdf

This turns out to be inconsistent handling of the leading /. Rackspace don't treat /x/ as a placeholder folder, but cyberduck does.

Have tagged it as openstack as a guess.

Log drawer not interesting as it doesn't show the error, just successfully getting the files list.

GET /v1/MossoClouxxxxxxxxxxxxxxxxxxxxxxx/xxxxx4742?format=xml&prefix=&limit=10000&delimiter=%2F HTTP/1.1
X-Auth-Token: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Host: storage101.lon3.clouddrive.com
Connection: Keep-Alive
User-Agent: Cyberduck/4.4.5 (14721) (Windows 7/6.1) (x86)
HTTP/1.1 200 OK
Content-Length: 5069
X-Container-Object-Count: 22
Accept-Ranges: bytes
X-Timestamp: 1401738255.23634
X-Container-Bytes-Used: 21453741
Content-Type: application/xml; charset=utf-8
X-Trans-Id: txcf8bxxxxxxxxxxxxxxb7-0053ac31celon3
Date: Thu, 26 Jun 2014 14:44:30 GMT

[cyberduck]

@dkocher commented

In cab6e79 and 59b0901.

[cyberduck]

bccfca5 commented

Hi

Many thanks for the speedy investigation, but it still doesn't work for me.

Not sure what the differences are between your successful test and my failure - so here are some hopefully salient bits..

I'm on the beta build 14906 - hopefully that includes revision 14866 (snapshot is the same build number)

Not sure if the placeholder folder is supposed to be shown in the main tree (it doesn't, it's just a flat view of all files), but on the transfers dialog the URL us shown as clientfiles-4742/x.pdf, ie no placeholder.

I'm using / as the separator in my cloud files, and the files often begin with a leading / but am on Windows (you use path.DELIMETER i think - my java is rusty (>12 years ago) so I don't remember if this is the defined platform value or fixed)

My connection uses the Rackspace (UK) profile - but I get the same with Swift (Openstack) and Rackspace (US). I created a new bookmark in case there was a cached value.

Turned on debug logging. the first ref to my file is this - there is no reference to the placeholder/path.. (may be irrelevant)

2014-07-22 12:43:25,275 [Thread-0] DEBUG RegistryApplicationFinder - GetRegisteredDefaultApplication for filename Address proof xxxxx.pdf

and when I start the transfer.. still no path.

2014-07-22 12:43:25,987 [background-1] DEBUG ch.cyberduck.core.transfer.Queue - Add transfer Transfer{transferred=null, size=null, roots=[TransferItem{local=Local{path='C:\Users\paul\Downloads\Address proof xxxxx.pdf'}, remote=Path{path='/container-nnnnnn/Address proof xxxx.pdf', type=[file]}}], state=stopped, host=Host{credentials=Credentials{user='xxxxxxx'}, hostname='lon.identity.api.rackspacecloud.com', port=443, protocol=Profile{parent=swift, image=Local{path='C:\Users\paul\AppData\Local\Temp\\xxxxxxxxxxxxxico'```}

}}}

..and not surprisingly I get 404.

Any further assistance let me know. 

Paul

[cyberduck]

@dkocher commented

Please update to the latest snapshot build available. Should be fixed with 5fea16e.

[cyberduck]

bccfca5 commented

Really sorry, still not working - same symptoms as before as far as I can see - on snapshot 4.5.2 (14943)

[cyberduck]

@dkocher commented

Reading the ticket description again I now understand that this issue describes that objects in placeholders are actually displayed in the root of the container instead in the browser and the placeholder path is omitted.

[cyberduck]

@dkocher commented

Please post again a current transcript from the log drawer (⌘-L). You should see requests with the format

GET /v1/MossoCloudFS_59113590-c679-46c3-bf62-9d7c3d5176ee/test.cyberduck.ch?format=xml&prefix=<x>%2F&limit=10000&delimiter=%2F HTTP/1.1

[cyberduck]

bccfca5 commented

Arghh! This is frustrating! I'm on 15069. Still NOT working for me.

Certainly the files are displayed flat - with no placeholder dir - I'd say that was a strong(ish) hint that the placeholder dir has been lost processing the file list from rackspace

One oddity is that I can't connect using openswift profile- I use Rackspace US profile (even though the DC is in london). however the log implies the swift helper is being used

It looks to me like the placeholder is not extracted...

2014-09-09 22:44:34,043 [Thread-0] DEBUG ch.cyberduck.ui.AbstractController - Run action WorkerBackgroundAction{worker=SessionListWorker{directory=Path{path='/clientfiles-xxxx', type=[directory, volume]}}, result=null} in background
2014-09-09 22:44:34,044 [Thread-0] DEBUG ch.cyberduck.ui.AbstractController - Synchronize on lock Session{host=Host{credentials=Credentials{user='username'}, hostname='identity.api.rackspacecloud.com', port=443, protocol=Profile{parent=swift, image=Local{path='C:\Users\paul\AppData\Local\Temp\\4bfc20b0-9fb1-4bb6-afd0-56421f079db2.ico'```, state=closed} for action WorkerBackgroundAction{worker=SessionListWorker{directory=Path{path='/clientfiles-xxxx', type=[directory, volume]}}, result=null}
}}}

2014-09-09 22:44:46,308 [background-1] DEBUG ch.cyberduck.core.Profile - No value for key:Scheme ??

I see this..

2014-09-09 22:44:50,667 [background-1] DEBUG ch.cyberduck.core.threading.AbstractBackgroundAction - Finish background task WorkerBackgroundAction{worker=SessionListWorker{directory=Path{path='/clientfiles-xxxx', type=[directory, volume]}}, result=[Path{path='/clientfiles-xxxx/Address proof xxxx.pdf', type=[file]}, ...
}]}

So by this point already the placeholders have been lost/removed

My code uses the .net library openstack, and calling listfiles on the container I get the first file as /Admin/Address proof xxxx.pdf (is the leading / a problem??)

and then on the transfer..

2014-09-09 22:34:02,944 [background-1] DEBUG ch.cyberduck.ui.threading.TransferCollectionBackgroundAction - Finish background action for transfer Transfer{transferred=null, size=null, roots=[TransferItem{local=Local{path='C:\Users\paul\Downloads\Address proof xxxxx.pdf'}, remote=Path{path='/clientfiles-xxxx/Address proof xxxx.pdf', type=[file]}}], state=stopped, host=Host{credentials=Credentials{user='username'}, hostname='identity.api.rackspacecloud.com', port=443, protocol=Profile{parent=swift, image=Local{path='C:\Users\paul\AppData\Local\Temp\b660f76b-bb93-4b9f-a0f5-fe795a51ab8b.ico'```}
}}}

What's the difference with my setup and yours?

[cyberduck]

@dkocher commented

Can you please contact Rackspace support and confirm your layout in the container adheres to the recommendation in Pseudo hierarchical folders and directories.

[cyberduck]

bccfca5 commented

It is in fact the leading / I have on my files that is breaking it.
Removed the leading / and their web interface now presents a hierarchy (never seen it before so I didn't realise it would do that) and it's replicated in Cyberduck.

Well we got there in the end.

Thanks for your help

[cyberduck]

bccfca5 commented

..so there is still a bug, and that is that files beginning with a / can't be downloaded - there's a discrepancy in the handling - you think it's a placeholder folder, but Rackspace don't treat it as such.

[cyberduck]

@dkocher commented

Path names are always normalized. See also #8238.

[cyberduck]

bccfca5 commented

Hi David -
A bit more background - I've been attempting to rename all my files from /x/y/z.pdf to x/y/z.pdf
It looks like the leading / is causing rackspace a problem too - as all my object renames are failing.. although with a non-obvious error.
I've raised a ticket with them and if anything salient comes out of it I'll chuck a link on here.

Paul

[cyberduck]

@dkocher commented

Replying to [comment:20 paulnicklin]:

Hi David -
A bit more background - I've been attempting to rename all my files from /x/y/z.pdf to x/y/z.pdf
It looks like the leading / is causing rackspace a problem too - as all my object renames are failing.. although with a non-obvious error.
I've raised a ticket with them and if anything salient comes out of it I'll chuck a link on here.

Paul

Thanks for the update.

[cyberduck]

@dkocher commented

Replying to [comment:19 dkocher]:

Path names are always normalized. See also #8238.

No longer true as of f029f4c.

[cyberduck]

@dkocher commented

Because we use / as the delimiter for listing keys this cannot be fixed.