Proposal: Streamable apps #834

Closed
justinbmeyer opened this Issue Feb 20, 2017 · 12 comments

Comments

Projects
None yet
4 participants
@justinbmeyer
Contributor

justinbmeyer commented Feb 20, 2017

Proposal: Stream-able apps

This proposal seeks to add streaming abilities to DoneJS, providing an improved user experience. By streaming, we mean we are able to write HTML out to the user as soon as we have the data for it. Instead of waiting for all the data to return from the database or a service call, we are able to stream records as they are retrieved into meaningful output to a user. Specifically, we seek to enable streaming during initial rendering (SSR) and also once the browser has loaded.

This should have a significant impact on performance, especially SSR performance. When rendering an HTML page on the server, we will be able to stream out the <head> immediately which typically includes <link> tags that load external resources. Getting that HTML to the browser immediately, means that the browser can start loading those resources while the database is still hypothetically returning data.

SSR flow

streaming-ssr

Overview of streaming flow

  1. Browser makes a request to the todos.html page.
  2. SSR server begins rendering an instance of the application.
  3. <head>{{title}}</head> and <link href=“theme.css”> are not waiting on anything asynchronous, so they are streamed to the HTML result.
  4. The <ul> (probably) isn't waiting on any asynchronous behavior, so it is streamd to the result.
  5. The {{#each todosPromise.value}} does the following:
    1. Does a request for Todo.getList({}) which results in a promise that resolves immediately to a DefineList.
    2. This fires off a .fetch() request to /api/todos.ndjson whose results will be streamed to that DefineList.
    3. That DefineList is passed to can-view-live.list.
    4. The DefineList, can-view-live.list and the streaming DOM serializer are able to coordinate in such a way to notify the DOM serializer to wait.
  6. Services server handles the /api/todos.ndjson request.
  7. Services server gets a pooled connection and makes a SELECT query to the database.
  8. As records come back, the services server converts them into newline delimited JSON, and writes the records in chunks back to the SSR server.
  9. The SSR server parses out the new JSON records, .push() each record into the DefineList.
  10. By adding the record into the DefineList, a new <li> is created
  11. The streaming DOM serializer is able to identify this change, serialize the DOM of the <li> and res.write it to the browser.
  12. The browser displays the contents of the <li>.

Steps 8-12 repeat as records come back from the Database.

Once the database records have completed:

  1. The Services server response wil be ended res.end().
  2. The asynchronous behavior associated with the DefineList will complete.
  3. can-view-live.list will mark all of its nodes as being ready for serialization.
  4. The streaming DOM serializer will move onto the closing </ul>.
  5. The SSR server will write out the closing </ul> to the browser.

Things to build

  • - A ndjson stream transformer:
    fetch("todos.ndjson") .then( (response) => {
  
     return ndjsonStream( response.body  );
    
}).then( (todosStream) => {
  
});
  • - A streaming can-connect behavior:
    // get a streaming list of todos
    Todo.getList({}).then(function(todos){
      template({ todos: todos });
    });
    <!-- template updates as todos are returned -->
    <ul>
      {{#each todos}}
        <li>{{name}}</li>
      {{/each}}
    </ul>
  • - A streaming DOM serializer:
    var serialize = require('vdom-streaming-serializer');
    http.createServer(function(request, response){
    
      var document = makeDocument();
    
      // ... ADD APP TO DOCUMENT ...
      myApp(document);
    
      // send back HTML as it is "completed"
      var stream = serialize(document.documentElement);
      stream.pipe(response);
    
    });
  • - A notification mechanism for asynchronous and/or streaming behavior:
    var observationZone = require("can-observation-zone");
    
    // A stateful object
    var todos = new DefineList([
      {id: 1, name: "lawn"}
    ]);
    
    // Indicates the length is going to change in the future
    var updatedTodos = observationZone.add(todos, "length");
    
    var completed = observationZone.completed(todos,"length");
    completed.then(function(){
      console.log("zone is completed");
    })
    
    setTimeout(function(){
      todos.push({id: 2, name: "dishes"});
      updatedTodos();
      console.log("updatedTodos")
    },1);
    
    // logs
    //  - updatedTodos
    //  - zone is completed
  • - Streaming aware live-binding utilities
  • - A demo app that puts it all together
donejs-streaming-dev-server

Links

@matthewp

This comment has been minimized.

Show comment
Hide comment
@matthewp

matthewp Feb 20, 2017

Contributor

For:

A newline delimited JSON writer and parser.

I think we can use njdson instead. As long as your db library can provide streams (or you can turn them into streams) you can do:

app.get('/todos.ndjson', function(request, response){
  db.query('select * from foo')
    .pipe(ndjson.serialize())
    .pipe(response);
});
Contributor

matthewp commented Feb 20, 2017

For:

A newline delimited JSON writer and parser.

I think we can use njdson instead. As long as your db library can provide streams (or you can turn them into streams) you can do:

app.get('/todos.ndjson', function(request, response){
  db.query('select * from foo')
    .pipe(ndjson.serialize())
    .pipe(response);
});

@chasenlehara chasenlehara changed the title from Proposal: Stream-able apps to Proposal: Streamable apps Feb 24, 2017

@Xuefeng-Zhu

This comment has been minimized.

Show comment
Hide comment
@Xuefeng-Zhu

Xuefeng-Zhu Feb 25, 2017

@justinbmeyer Talk to on HackIllinois. It seems that what the project is trying to do is server push, which is implemented in http2 standard

https://www.igvita.com/2013/06/12/innovating-with-http-2.0-server-push/

@justinbmeyer Talk to on HackIllinois. It seems that what the project is trying to do is server push, which is implemented in http2 standard

https://www.igvita.com/2013/06/12/innovating-with-http-2.0-server-push/

@justinbmeyer

This comment has been minimized.

Show comment
Hide comment
@justinbmeyer

justinbmeyer Feb 25, 2017

Contributor
Contributor

justinbmeyer commented Feb 25, 2017

@justinbmeyer

This comment has been minimized.

Show comment
Hide comment
Contributor

justinbmeyer commented Feb 26, 2017

cursor_and_v5_key

@leoj3n

This comment has been minimized.

Show comment
Hide comment
@leoj3n

leoj3n Mar 1, 2017

Contributor

This, is a fine idea.

Is my understanding correct, that SSR is what enables this to work, and it would not be possible without the SSR brokering the transaction? In that case, it makes using SSR that much more convincing. I don't see why this should cause a problem with javascript spidering search engines, but if it did I would imagine the solution could be a trigger offered by the likes of Google, that allows you to tell the engine your page is now complete; not sure if that exists or is necessary though, and maybe I'm overthinking it.

Contributor

leoj3n commented Mar 1, 2017

This, is a fine idea.

Is my understanding correct, that SSR is what enables this to work, and it would not be possible without the SSR brokering the transaction? In that case, it makes using SSR that much more convincing. I don't see why this should cause a problem with javascript spidering search engines, but if it did I would imagine the solution could be a trigger offered by the likes of Google, that allows you to tell the engine your page is now complete; not sure if that exists or is necessary though, and maybe I'm overthinking it.

@matthewp

This comment has been minimized.

Show comment
Hide comment
@matthewp

matthewp Mar 1, 2017

Contributor

@leoj3n There are 2 parts. One is streaming SSR. The other is streaming API requests. The latter does not require that you use SSR. It will be available as a can-connect plugin (can-connect-ndjson) that can be used with any can project. Server rendered or not.

Contributor

matthewp commented Mar 1, 2017

@leoj3n There are 2 parts. One is streaming SSR. The other is streaming API requests. The latter does not require that you use SSR. It will be available as a can-connect plugin (can-connect-ndjson) that can be used with any can project. Server rendered or not.

@justinbmeyer

This comment has been minimized.

Show comment
Hide comment
@justinbmeyer

justinbmeyer Mar 1, 2017

Contributor

@leoj3n google (or anything consuming the http response) would know to wait until the response is complete anyway.

Contributor

justinbmeyer commented Mar 1, 2017

@leoj3n google (or anything consuming the http response) would know to wait until the response is complete anyway.

@leoj3n

This comment has been minimized.

Show comment
Hide comment
@leoj3n

leoj3n Mar 1, 2017

Contributor

That is so cool. Very elegant. I am going to have to play with this, and possibly rig up an objective speed comparison test.

Contributor

leoj3n commented Mar 1, 2017

That is so cool. Very elegant. I am going to have to play with this, and possibly rig up an objective speed comparison test.

@matthewp

This comment has been minimized.

Show comment
Hide comment
@matthewp

matthewp Apr 24, 2017

Contributor

For a demo I would have something with 1) A header 2) A sidebar 3) A list of something (search results is a good candidate since search queries are usually slower)

We could slow down the query so that you can see the header and styles being applied, the sidebar showing up, and then the results appending.

This is for SSR. Client-side streaming might have a better demo for it (although I think this isn't a bad one for that either).

Contributor

matthewp commented Apr 24, 2017

For a demo I would have something with 1) A header 2) A sidebar 3) A list of something (search results is a good candidate since search queries are usually slower)

We could slow down the query so that you can see the header and styles being applied, the sidebar showing up, and then the results appending.

This is for SSR. Client-side streaming might have a better demo for it (although I think this isn't a bad one for that either).

@matthewp

This comment has been minimized.

Show comment
Hide comment
@matthewp

matthewp May 10, 2017

Contributor

Testing Streaming Approaches

Last week I did some testing of two approaches to streaming with HTTP2. In both methods:

  • The JavaScript is pushed.
  • The data requests are pushed.
  • 100 items are streamed from the API, 1 every 10ms.

The difference is in how the DOM rendering happens.

Traditional SPA with streaming data

The first approach was a traditional SPA with streaming data. The timeline looks like this:

screen shot 2017-05-10 at 10 06 22 am

Since the data is streamed we only need to append to the list. CanJS can do this very quickly.

Incremental Rendering

In this method instructions on how to render are streamed from the server (along with the data). The timeline looks like this:

screen shot 2017-05-10 at 10 06 46 am

This method is advantageous because instructions come in incrementally and asynchronously. This is especially good for slower cpus (such as some phones).

Conclusions

Some conclusions I drew from these tests are that:

  • The slowest part of the app is the blocker when streaming. If the slowest part of your app is the data requests, then either method is about the same. But if the JavaScript payload is the slowest part, the incremental rendering approach will be better.
  • Slower CPUs are affected by the amount of JavaScript executed. This favors the incremental approach which:
    • Has a smaller initial payload.
    • Modifies the DOM incrementally (as instructions come in), asynchronously.
  • Incremental rendering can get to first-paint faster as it doesn't have to wait on a large bundle to execute.
Contributor

matthewp commented May 10, 2017

Testing Streaming Approaches

Last week I did some testing of two approaches to streaming with HTTP2. In both methods:

  • The JavaScript is pushed.
  • The data requests are pushed.
  • 100 items are streamed from the API, 1 every 10ms.

The difference is in how the DOM rendering happens.

Traditional SPA with streaming data

The first approach was a traditional SPA with streaming data. The timeline looks like this:

screen shot 2017-05-10 at 10 06 22 am

Since the data is streamed we only need to append to the list. CanJS can do this very quickly.

Incremental Rendering

In this method instructions on how to render are streamed from the server (along with the data). The timeline looks like this:

screen shot 2017-05-10 at 10 06 46 am

This method is advantageous because instructions come in incrementally and asynchronously. This is especially good for slower cpus (such as some phones).

Conclusions

Some conclusions I drew from these tests are that:

  • The slowest part of the app is the blocker when streaming. If the slowest part of your app is the data requests, then either method is about the same. But if the JavaScript payload is the slowest part, the incremental rendering approach will be better.
  • Slower CPUs are affected by the amount of JavaScript executed. This favors the incremental approach which:
    • Has a smaller initial payload.
    • Modifies the DOM incrementally (as instructions come in), asynchronously.
  • Incremental rendering can get to first-paint faster as it doesn't have to wait on a large bundle to execute.
@matthewp

This comment has been minimized.

Show comment
Hide comment
@matthewp

matthewp May 10, 2017

Contributor

Next Steps

I think regardless of whether the incremental rendering approach becomes a real thing that we use in DoneJS apps (and it might!) there are still some important changes that would be useful.

With that being said I think we should:

  1. Add HTTP2 support to done-serve
  2. Change steal-tools to export it's dependency graph (to include a mapping of bundles to all of the split bundles).
  3. Update done-ssr, using the info from (2) to HTTP2 push (when in an HTTP2 setting) the JavaScript.
  4. Update done-ssr to HTTP2 the data requests.
  5. Create the incremental rendering as an option in done-ssr (or possibly as an external plugin, if we want to avoid over-committing to this approach).
Contributor

matthewp commented May 10, 2017

Next Steps

I think regardless of whether the incremental rendering approach becomes a real thing that we use in DoneJS apps (and it might!) there are still some important changes that would be useful.

With that being said I think we should:

  1. Add HTTP2 support to done-serve
  2. Change steal-tools to export it's dependency graph (to include a mapping of bundles to all of the split bundles).
  3. Update done-ssr, using the info from (2) to HTTP2 push (when in an HTTP2 setting) the JavaScript.
  4. Update done-ssr to HTTP2 the data requests.
  5. Create the incremental rendering as an option in done-ssr (or possibly as an external plugin, if we want to avoid over-committing to this approach).
@matthewp

This comment has been minimized.

Show comment
Hide comment
@matthewp

matthewp Oct 30, 2017

Contributor

Going to close in favor of #787, of course we will being making future streaming enhancements in the future.

Contributor

matthewp commented Oct 30, 2017

Going to close in favor of #787, of course we will being making future streaming enhancements in the future.

@matthewp matthewp closed this Oct 30, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment