Application Factory Proposal/Specification

[kitsonk]

Story

In order to make it easy to develop full featured web applications, one way of building the application is describing it in meta-data which then gets "inflated" into an instantiated applications.

Features

The application factory should express the following features:

• Interface to a Dojo Store to retrieve its meta data
• Have a small, concise API surface
• Integrate into Dojo Topics and Events as appropriate to extend/modify behaviour

To bootstrap, ideally you would have something as straight forward as this:

import restStoreFactory from 'dojo-store/rest';
import appFactory from 'dojo-app/factory';

const myApp = appFactory({
    store: restStoreFactory({
        url: '/app/myApp'
    })
});

myApp.start();

Considerations

Some points to consider are:

• Does this supplant/transform the Parser?
• What sort of schema is needed to define the application in data?
• What sorts of features are needed from other parts of Dojo 2 to support this?
• What is the application factory lifecycle?
• How does it integrate into concepts of Actions and Routing?

[kfranqueiro]

I probably don't have the full picture here, but I'm not sure I see the utility in referencing a store for the factory metadata.

• We're not 100% sure what stores will look like (see Dojo 2 Stores Proposal/Specification #12)
• It seems like app factory config ought to be one object, not an array of objects or a store that is just a glorified/bloated object hash - especially with the given rest example, which would presumably result in multiple separate XHRs just to bootstrap the application...

[kitsonk]

It is proposed that Stores (whatever form they take) be one of the foundational components of Dojo 2 applications. They are in theory and abstract concept of persisting data. The needs of an application factory would be one of the drivers for identify the requirements for Dojo 2 Stores. It is a bit of a chicken and an egg.

It is a valid point that the schema for what an application factory's meta data should not dictate that it should contain arbitrary requests, although there are some advantages to being able to "lazy load" and "inflate" certain parts of the application, so the schema should not preclude the ability to incrementally load parts of an application.

In conversations with @pottedmeat we have talked about the concepts of:

• Differentiation between structure and state:
  • The structure of the application is what components it has and how they are wired together
  • Some of the structure will can also have an initial state which can then be set once the structure is instantiated
• Lazy Instantiation - The concept that the application factory can instantiate "stubs" or "handles" which can then be fully inflated upon access

[kfranqueiro]

I'm certainly not against the notion that parts of the application logic can be lazily loaded / instantiated. IIRC Mayhem allowed this to an extent by passing module IDs rather than direct constructors to properties of its application configuration object.

But what I was interpreting above was that you'd be needing to perform store requests, potentially via XHR against a server, just to get metadata to begin with, let alone actually instantiate parts of the application, hence my reservation about prolonged/bloated bootstrapping. This seems like it'd potentially be more painful to eventually optimize into builds as well, as opposed to formulating one or more build layers around various module IDs referenced in the application's configuration.

I agree RE it being a chicken-and-egg situation... to me, needing a store way up front just to provide the application's configuration seems like overkill, but I suspect I'm not seeing what you're really envisioning with this. (Sorry if I'm mentally blanking on parts of the application proposal, I know I read it a few weeks ago...)

[pdfernhout]

As I posted about in this comment to issue #15, I implemented a specification-driven webapp with about forty virtual pages, each page defined by a specification file.

At first the application had the specifications in one big structured text file that my wife, the subject matter expert, filled in (based on a her 700 page textbook which defines a participatory narrative inquiry process step-by-step). Those textual specifications were "compiled" to one big JavaScript modules which defined the specifications in a quickly loadable format. Later, the specifications became lots of small JSON files (thinking I might dynamically load the specifications to reduce loading time, including perhaps from untrusted sources and thus via JSON). In the end, I settled on importing TypeScript modules to be able to easily include comments in the specifications, to have multi-line strings, to have type-checking on the specifications definitions, and to include some code (like for decisions on whether a field should be enabled).

I ultimately solved the latency issue of downloading multiple small specification files by using a bundling step during packaging. A large app pulling its specifications from a store might encounter latency issues depending on how fine-grained the requests were.

One advantage of going the specification route was to make it easier to change underlying GUI widget sets (in this case, from Dijit1/DOM to Mithril/vdom).

Also, in theory, specifications defined this way could be queried and sorted in different ways than just looking at plain code (like reporting on all the field names, or looking at all the display strings). See the book the Discipline of Organizing for ideas about how the same information can be organized in different ways for different purposes.

As I detailed at length in that comment, I had mixed feelings about the result given both benefits and costs. For just one webapp, you have to understand the more complex machinery that turns specifications into behavior, compared to understanding a simpler approach of having specific code in different modules to define a model, GUI, storage, and reporting. If I was going to reuse that framework code to make many such applications (within specific limits of that ad-hoc framework), then it might be a big win to have gone the specification route. But, I just was making one application. It is not clear to me how many applications would fit exactly in those constraints -- although I had hopes that many could. The big value here was that my wife, who was a programmer but did not know JavaScript, could focus on defining the specifications while I could focus on the JavaScript machinery around that, So the approach fit a separation of human "roles" more than a separation of programming "concerns".

One constraint was using a messaging-based triplestore architecture to support multi-user editing compared to a more document-oriented store, where all the application's data is retrieved at startup and queried locally. That download process at startup could take ten seconds or more to download 1000 or more messages that reflected the application's state (with results batched 100 messages at a time by the message store). You could build specification-driven applications without this constraint, but a supporting framework would be somewhat different. I also tried to make the application modeless (so, there is no "submit" button), but I have not had time to finish an undo system which a modeless app really needs.

Another constraint was that ultimately some idiosyncratic behavior (beyond application-specific widgets) was needed for the application. Essentially, there had to be one file (and supporting modules) where strings referenced in the specifications were linked to functional code, I'd expect for most applications you'd end up with something similar. You could keep the source for those functions in a store too, but is that really valuable? How much of that app do you want in a store?

Also, as I discovered on an project done many years earlier, where most application behavior was driven by a tab-delimited spreadsheet in a format I designed (an application that processed literally millions of insurance applications), you still want everything under version control. You can't expect non-programmers to correctly edit structured files (even spreadsheets), let alone deal with version control (even if some can). As a programmer, it's easy to take for granted a lot of behaviors such as not accidentally deleting huge blocks of text by an errant mouse swipe and not noticing it, or not just adding arbitrary explanatory text somewhere a parser won't like it. If programmers are going to be doing the maintenance anyway, why not just go with programmer-friendly approaches like files on disk under version control (or some versioned web-based CMS equivalent)?

Stores introduce another complexity. For an application defined in a store, who backs it up? How do you know what changed? How do you even change anything? How do you share changes? How do multiple people collaborate on the application? There may be answers to these (and I've explored them myself), but they are probably non-trivial answers that impose other constraints on the design.

One benefit in theory of a store approach though (without code -- so JSON specifications) might be to download specifications from untrusted sources to integrate them into a larger app. I toyed with that, but realized it seemed not that important for what I was doing. Also, it seems more likely to me that different special-purpose applications might use a common data store than the same application will pull parts of its behavior from different stores. So I don't see this a compelling argument for stores.

So, between the latency issue, the likely need for some application-specific code somewhere, and the general need for programmers to collaborate on shared code under version control, I'm not sure putting stuff in a store would really pay off as much as just designing an application to be driven by "meta-data" specifications and having those specifications be TypeScript modules. Those modules might or might not include some code, depending on framework assumptions and the typings governing the specification format. Once you have something like that working, then you could move the specifications to a store if you really think it is going to be of great benefit.

[kitsonk]

Issue moved to dojo/app #4 via ZenHub.io