no way to query metadata in current master

[michielbdejong]

When you do a getListing on a folder, in current master, you do not get any metadata. We should either restore the behavior that was there in 0.9, or create a new way to query metadata.

Here, metadata is currently: ETag, Content-Type, and Content-Length

[michielbdejong]

leaving this one for next month

[untitaker]

👍 I currently get bitten by this.

Concretely I want to store a file with If-None-Match or If-Match conditions. This itself is not possible AFAIK, but there is no way for me to fetch the original etags in the first place.

[raucao]

Concretely I want to store a file with If-None-Match or If-Match conditions

That's handled automatically by this library.

but there is no way for me to fetch the original etags in the first place.

Yes, the current cached directory listings don't contain the metadata yet. Unfortunately, the code that was written at the time didn't account for metadata being introduced in later spec versions, so it's not trivial to add the feature. Yet, it's kind of important, of course.

@michielbdejong If you change your last comment to "next year", then this would be the perfect time to have a go at it. :D

[untitaker]

That's handled automatically by this library.

How is this possible? If I load a file, parse it and put it into my datamodel, then sometime later modify the datamodel and serialize it back into a file, how would remoteStorage know whether the file object I used is up to date?

Also when creating new files, I want to ensure that my randomly generated filenames don't clash with existing ones. Is there a way to ensure this?

I suspect that I'm doing something wrong, but I can't figure out what, since all I have is the API docs. Is there an example app for this?

[michielbdejong]

IIRC, the problem with this is that in our IndexedDB database, the metadata is stored on the item, not on the containing folder node. So if you do getListing on a folder without metadata, that's one request to IndexedDB, but if you do want the metadata of each item, you have to first retrieve the folder node, and then do one request to IndexedDB for each item in the folder, which might be a lot of items. So rather than making getListing return metadata, it may be nicer to add a getMetaData method where you can retrieve the metadata of one item, or loop over all items if you really need the metadata of all the items in the folder. That gives the app developer some chance to keep things efficient.

Yeah, that 'next month' never happened, sorry. :)

how would remoteStorage know whether the file object I used is up to date?

Ah, it knows because it's very smart! ;) Of course, initially, it doesn't know. But when you push out your local change, it will add an If-Match header on that PUT requests, so that, if the remote data also changed, then the PUT will get rejected. When that happens, remotestorage.js will do a GET to retrieve the new value on the server ('server wins'), and give a change event to the app with source: conflict. After this, the app can write its data again to effectively accomplish 'client wins'. There should be a lot of documentation somewhere in this repo, or maybe also on remotestorage.io, about how to handle conflict events. I called this the 'keep/revert' system at the time. Have a look at this bit, for instance: https://github.com/remotestorage/remotestorage.js/blob/master/doc/data-format.md#keeprevert-conflict-resolution

I want to ensure that my randomly generated filenames don't clash with existing ones

Same system: your local new item creation will be pushed out as a PUT with an If-None-Match: * header, so if a remote item already exists on the same URL, your app will receive a change event which contains the keep/revert conflict. The only thing you can't really detect is if a file was briefly created and deleted on either side (client or server). That will look to the other side (server or client) as if nothing ever happened there. :)

[raucao]

Just FYI: removed bug label, added feature label. Reason: it's not really a bug, but a missing feature in this codebase.

[untitaker]

IIRC, the problem with this is that in our IndexedDB database, the metadata is stored on the item, not on the containing folder node. So if you do getListing on a folder without metadata, that's one request to IndexedDB, but if you do want the metadata of each item, you have to first retrieve the folder node, and then do one request to IndexedDB for each item in the folder, which might be a lot of items. So rather than making getListing return metadata, it may be nicer to add a getMetaData method where you can retrieve the metadata of one item, or loop over all items if you really need the metadata of all the items in the folder. That gives the app developer some chance to keep things efficient.

Fair enough, although in my case I'd have to call that method for every item
returned by the listing anyway.

Ah, it know because it's very smart! ;) Of course, initially, it doesn't know. But when you push out your local change, it will add an If-Match header on that PUT requests, so that, if the remote data also changed, then the PUT will get rejected. When that happens, remotestorage.js will do a GET to retrieve the new value on the server ('server wins'), and give a change event to the app with source: conflict. After this, the app can write its data again to effectively accomplish 'client wins'. There should be a lot of documentation somewhere in this repo, or maybe also on remotestorage.io, about how to handle conflict events. I called this the 'keep/revert' system at the time. Have a look at this bit, for instance: https://github.com/remotestorage/remotestorage.js/blob/master/doc/data-format.md#keeprevert-conflict-resolution

I'm more worried about the following scenario (with remoteStorage.js's caching
client):

• I use getFile, parse the result and store it in my model (and the DOM)
• remoteStorage.js synchronizes the file, changes it (adding new data to the
  file)
• I serialize my model into a blob, use storeFile. Since my model didn't
  contain the new data, it is lost.

As far as I can see, remoteStorage.js can't detect this scenario. I can try to
mitigate this by subscribing to change-events (to update my model ASAP), but
I think there's still a potential for data races:

• I use getFile, parse the result and store it in my model
• remoteStorage.js synchronizes the file, changes it
• remoteStorage.js fires the event
• I serialize the model, use storeFile, data gets lost
• I recieve the event, but it's too late

I have a very limited understanding of Javascript, but as far as I know this
situation is possible.

I want to ensure that my randomly generated filenames don't clash with existing ones

Same system: your local new item creation will be pushed out as a PUT with an If-None-Match: * header, so if a remote item already exists on the same URL, your app will receive a change event which contains the keep/revert conflict. The only thing you can't really detect is if a file was briefly created and deleted on either side (client or server). That will look to the other side (server or client) as if nothing ever happened there. :)

Similar scenario here, since I can't atomically check whether a file exists in
the local cache and simultaneously update it when there isn't one, I still have
a race.

---

Reply to this email directly or view it on GitHub:
#721 (comment)

[untitaker]

I've created a new feature request at #887 because I really derailed the issue here. It also clarifies what I think is missing from remoteStorage.js