File system to Solid mapping

[bblfish]

On gitter @leobard expressed a use case that comes up a lot: placing a Solid server with a root directory on one's local file system, so that one can easily edit the files with tools available on one's desktop.

This is actually how we started programming the web. We could open a browser to view files on the file system and immediately, link them together and see how they worked. But as we allow content negotiation on Solid we can have a local file such as /index.html become /index if we adopt to always remove the extension, which makes a lot of sense with RDF. But then links built on the file system will always contain the extension eg. .html. This is not so troubling with html (as the Solid server should also react to the extension). But with RDF, a user would end up setting a preference for one of the encodings. So all the links would be /card.ttl#me rather than /card#me. If some of the data were then to be edited via the LDP API then the user would end up with links to the content without the extension.

One way one could get it right would be to have symbolic links on the file system that linked to the default extension. So if one has card.ttl one could have a symbolic link from card to card.ttl. This would allow a good editor to jump to the correct relative URL. (We don't have many such editors). So then should such a web server always and such symbolic links?

It seems like one could have a repo and spec just dedicated to this use case.

[bblfish]

It would be interesting to know how things look on different operating systems. Some support attributes on files. Is there an OS which supports attributes where Apps consistently consult those? Those OSes would then provide a particularly easy answer: add the mime type to a file to the standard attributed for it. If all apps on the OS consulted those all would be good.

It would not help that much for exchanging data across OSes. There, one would need to go to the lowest common denominator or create a standard metadata associated with files in the tar).

And it would probably not help for hackers that link to edit files on the FS, with vim as those tend to rely on extensions.

Researching this a bit I found that in Java there is the User Defined File Attribute View, which allows one to set attributes across platform. The implementation of course differs between platforms as these have different Systems (see StackExchange answer). And I think for platforms that don't have the feature Java encodes these properties in a specific way. (what way?) So it would require a format to transfer files with attributes between platforms, if that is important.

[bblfish]

I think @timbl's answer here is a helpful one, especially as it helps avoid problems with .exe attacks on Windows.

URI on the web	Content type	Filename on the system
http://localhost/space/foo.html	text/html	/users/bob/space/foo.html
http://localhost/space/foo.bar	text/html	/users/bob/space/foo.bar$.html
http://localhost/space/foo.exe	text/html	/users/bob/space/foo.exe$.html

But I think it may not get it quite right with the other way around, where people are editing files on the FS and checking them in some browser locally, and want to create hyperlinks between representations.

So for RDF the situation is that one wants to serve a foaf profile card that links card#i to group#friends. Here I think both locally and on the web we don't want to show the extensions.

URI on the web	Content type	Filename on the system
http://localhost/space/card	text/turtle	/users/bob/space/card.ttl
http://localhost/space/group	text/turtle	/users/bob/space/group.ttl

In rww-play I had come to the conclusion that it would help to have a symbolic link card to card.ttl that would help when editing files locally (not that we really have such editors yet!)... But perhaps that creates .exe type problems on Windows? (If a symbolic link hello.exe points to hello.exe.html does it get executed on Windows?).

Anyway, it looks like the symbolic links locally allow one to get the direct mapping answer n1.

[leobard]

Hi @bblfish , I didn't think through all implications outlined above.

assuming i have my files stored locally like /home/bob/documents/projects/bar/foo.html
the foo.html only contains the "content" payload of the data, what a CMS would store internally. Alternatively, think of README.md, a markdown file in the same folder.

my 2c:

• mapping the local filesystem to a web-hosted requires an intelligent webserver
• the webserver will likely have a URI scheme for returning the raw file, something along the lines of https://myserver.com/theapp/**raw**/home/bob/documents/projects/bar/foo.html
• additionally, the webserver will have a GUI that shows an editor for the file, where you get some kind of app that is good in changing the content of exactly this file but not showing much else: https://myserver.com/theapp/**edit**/home/bob/documents/projects/bar/foo.html
• additionally, the webserver will have a GUI that shows a viewer/browser for the file, where you klick links, see related content, have a menu around for navigation, metadata: https://myserver.com/theapp/**view**/home/bob/documents/projects/bar/foo.html
• all three endpoints (raw, edit, view) will check access rights, will serve different content. The view and edit could be JS apps completely running the in the browser client using get/put to "raw" to load and save the content.
• that in mind, I would solve all linking issue with the knowledge that there is an app in between
• all links between files can be relative and will work, especially in the "viewer" context
• if we want to map the URLs to identify the "RDF resource", we can mint another URI and 303 from that away according to Cool URIs for the semantic web. For example, following what I wrote in cool uris, I would use https://myserver.com/theapp/**id**/home/bob/documents/projects

I have no deep knowledge of SOLID architecture, so you probably already have all that. I am noob here, so please take my input with grain of salt.

[bourgeoa]

You could have a look to solid/solid-rest which does this kind of mapping.

[bblfish]

@leobard I think one may be able to avoid 3 namespaces and have just one
https://myserver.com/bob/documents/projects/bar/foo.html

First note that a Solid server should just be a normal HTTP server. So uploading HTML, css, JS, etc should just render as it in normal browsers without needing any JS. This would allow the tradition method of publishing pages, except that content creators can upload their documents directly with LDP's HTTP read-write extension.

So the server should then be able to specify transformations for various file formats such as .md to allow them to be rendered in HTML. Perhaps just requesting an .md file with "text/html" Content-Type would be enough. Perhaps JS clients should add a header specifying they want the raw type... I think it is tricky to work out right but there should be a way to make sure that one does not have parallel URL spaces.

[bourgeoa]

@leobard
NSS uses this module for URI <-->
filename mapping https://github.com/solid/node-solid-server/blob/master/lib/resource-mapper.js

 URI name  content-type    filename
  test.txt       text/turtle <>    text.txt$.ttl
  test.txt.      text/plain  <>.   text.txt

[leobard]

@bblfish you can of course jump through hoops and realize it with one URL using HTTP headers. But most developers today chose different URI schemes because they use modern application-building frameworks where you configure different paths for different ways of handling. So I decide to disagree: going for "one uri scheme to rule them all just because I can" does not help developers here. Looking at how dropbox/googledrive/onedrive do their thing, I see differnt URIs for raw/browser-render/editor and it makes life easier.

I recommend keeping things as simple as possible regarding implementation ability

[bblfish]

Solid already provides a way of viewing and editing one resource using one URL, since it is using LDP. That simplifies a lot of state-management all over the place. The problem you bring up is just how to view non natively viewable web resources such as RDF. Solid there tends to prefer to leave that to an LDP/RDF aware client to present the data as it wishes.
If a non RDF aware client fetches the resource, Solid servers tend to ship an HTML version with embedded JS of the RDF to render it as selected by the server. That embedded JS can also present a UI to allow the user to edit the resource, and the App can use LDP to save it to the same resource. That way there is no unnecessary duplication of links.

[bblfish]

One problem I see reading @RubenVerborgh's resource-mapper.js is that it does not deal with content negotiation. This is well explained by @leobard's Cool URIs for the Semantic Web.

If I read resource-mapper.js's mapUrlToFile correctly then it will find that the mapping from file.exe to file.exe$.html. But it will not find card to map to card.ttl or card.jsld. If furthermore someone asks for "cat.jpg" and there is a "cat.png" and there is a way to transform one to the other, this will not work either.