Consider folder extensions over Index.json types

[webdesserts]

One of the things I loved when I first started using OSX was the way they handled app packages (e.g. Safari.app). In my mind I see a lot of parallels with objectstore and keystore. They are types of folders that the average user shouldn't ever need to view the contents of. I also imagine they will be presented to the user as a custom UI. By giving these folders extensions like .objstore & .keystore and "pretending" that they are just normal files, I believe that it will be much more intuitive for a user to double click these packages and open them in an "objectstore editor/browser" or similar.

protected is different to me since it's purely permissions and as I mentioned in #13 it still feels weird to have file-based permissions.

[pfrazee]

That's a pretty interesting suggestion. I'll think about this.

[webdesserts]

@pfrazee Thinking through #15 a bit more has prompted some further thoughts on storage formats in general. I've tried to summarize all my thoughts on "packages" below and I think this represents why I'm so attracted to the idea.

---

In today's Operating Systems, a file is opened based on a file format. My notes app can open .md files. Excel can open .csv files. Although these applications don't necessarily get together and agree upon how that format is written, they all generally reference a handful of standards to interpret that file. As I mentioned, object stores are a kind of fancy file akin to something like a mac .photoslibrary "package" or an .app "package". However, there are a few ways that object stores are different from normal files:

1. they adhere to a schema Beaker is taking an extra step and trying to use schemas to help apps agree upon & parse the storage format.
2. they are user unique There is only one object store for any given schema per user. There would only be one blog.objectstore as opposed to multiple *.md files scattered throughout the fs.
3. they are tied to a designated folder Object stores are tied to the data folder and (as far as I can tell) cannot leave

So you could consider an object store a "browser-managed, user-unique, schema-adhering folder structure". If a web app ever needs to store and edit data that is not necessarily user-specific, the user-uniqueness of object stores might not be desirable. Unless I'm missing something, at this point the app would have to fall back to using normal files and folders for data storage. In the process of dropping that "user-uniqueness", they would lose the "browser-managed, schema-adhering" aspect of object stores.

I think this is why it might be a good idea to consider a schema-adhering "json-package" concept that exists below the concept of an "object store". These "json-packages" could function much like object stores do, but users could move, duplicate, share and collaborate on these packages with other users. To the average user, it would just look like their moving around and saving files like they always have.

As mentioned earlier, today's apps are all manually agreeing on and adhering to file formats. With object stores, the browser is jumping in and helping developers enforce this process in one specific case. Why not supply developers the means to use this tool for more general use-cases?

---

The above might produce a glossary like so:

• package: a folder that hides its contents from users (likely has a package manifest)
• json-package: a browser managed package that contains a collection of json files that adhere to a given schema.
• object store: a type of json-package where only one store may exist per schema for any given user.

[pfrazee]

If a web app ever needs to store and edit data that is not necessarily user-specific, the user-uniqueness of object stores might not be desirable.

Can you expand on the case where it’s not user-unique?

…

On Nov 4, 2018, at 8:09 PM, Michael Mullins ***@***.***> wrote: @pfrazee Thinking through #15 a bit more has prompted some further thoughts on storage formats in general. I've tried to summarize all my thoughts on "packages" below and I think this represents why I'm so attracted to the idea. In today's Operating Systems, a file is opened based on a file format. My notes app can open .md files. Excel can open .csv files. Although these applications don't necessarily get together and agree upon how that format is written, they all generally reference a handful of standards to interpret that file. As I mentioned, object stores are a kind of fancy file akin to something like a mac .photoslibrary "package" or an .app "package". However, there are a few ways that object stores are different from normal files: they adhere to a schema Beaker is taking an extra step and trying to use schemas to help apps agree upon & parse the storage format. they are user unique There is only one object store for any given schema per user. There would only be one blog.objectstore as opposed to multiple *.md files scattered throughout the fs. they are tied to a designated folder Object stores are tied to the data folder and (as far as I can tell) cannot leave So you could consider an object store a "browser-managed, user-unique, schema-adhering folder structure". If a web app ever needs to store and edit data that is not necessarily user-specific, the user-uniqueness of object stores might not be desirable. Unless I'm missing something, at this point the app would have to fall back to using normal files and folders for data storage. In the process of dropping that "user-uniqueness", they would lose the "browser-managed, schema-adhering" aspect of object stores. I think this is why it might be a good idea to consider a schema-adhering "json-package" concept that exists below the concept of an "object store". These "json-packages" could function much like object stores do, but users could move, duplicate, share and collaborate on these packages with other users. To the average user, it would just look like their moving around and saving files like they always have. As mentioned earlier, today's apps are all manually agreeing on and adhering to file formats. With object stores, the browser is jumping in and helping developers enforce this process in one specific case. Why not supply developers the means to use this tool for more general use-cases? The above might produce a glossary like so: package: a folder that hides its contents from users (likely has a package manifest) json-package: a browser managed package that contains a collection of json files that adhere to a given schema. object store: a type of json-package where only one store may exist per schema for any given user. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[webdesserts]

@pfrazee Ahh, is this supposed to be a solution for ALL data? The world index? Hmmm... can't expand on it right now, but...

I see object stores working well for anything that is associated with a single person:

• Blog posts
• Mini-blogs (twitter)
• Contacts
• Email
• Medical Data

I have a hard time seeing it working for anything associated with a group of people, a company, or an organization:

• Company Documentation
• Mockups
• Financials
• Contracts and other legal work
• Code

I don't "sign in" to my company, the company gives me permissions, but I don't really own any of the things that I work on for that company.

[pfrazee]

Okay right. In that case we will need another mechanism. I’m not sure that alternative mechanism requires us to generalize objects folders as you suggest though. Not sure uet. I’ll keep thinking about this.

…

On Nov 5, 2018, at 9:51 AM, Michael Mullins ***@***.***> wrote: @pfrazee Ahh, is this supposed to be a solution for ALL data? The world index? Hmmm... can't expand on it right now, but... I see object stores working well for anything that is associated with a single person: Blog posts Mini-blogs (twitter) Contacts Email Medical Data I have a hard time seeing it working for anything associated with a group of people, a company, or an organization: Company Documentation Mockups Financials Contracts and other legal work Code I don't "sign in" to my company, the company gives me permissions, but I don't really own any of the things that I work on for that company. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[pfrazee]

The core of this suggestion is:

• Don't have a global index.json file.
• Have folder extensions replace the type field.
• If any metadata is needed, it will be a part of the folder-extension's format.

That's fairly practical. Most of the time, the main reason for the global index.json was to provide the type, and then to provide type-specific metadata.

[webdesserts]

@pfrazee right. For the immediate future, the above would be nice.

I was throwing the package concept in as a way to reuse some of these benefits throughout the filesystem. I was imagining that these stores were an optional storage format for things like settings and user data. However, if your ultimate goal is to move the majority of application data AWAY from normal files and into these object stores, then that wouldn't be worth it.