Skip to content

Server communication

Marcin Warpechowski edited this page Jun 8, 2016 · 53 revisions

PuppeteerJs communication is based on REST principles, HTTP and JSON-patch (RFC 6902) to provide delta updates to morph a page into another page and to allow view model interaction between browser and server.

An example application

Let's assume that we have an email program such as Googles GMail. This application brings the both the benefits of SPA applications (speed and state) and regular hypertext application (back, forward buttons, bookmarks, etc.).

On the left side of the application page we have a list of emails. On the right side, we have the currently focused email. The URL in the browser points to a specific email. When we move the focused item in the list, the URL changes and the subpage of the email is updated. This means that each email acts as a linkable web page that can fully make use of bookmarks and the back and forward browser buttons. At the same time, the web page is not reloaded when the user moves between emails. Everything is fast and the state of the email list is retained as that part of the DOM is not updated.

A step-by-step walkthrough of what happens under the hood

  1. The user enters the URL in the browser (making a fresh request)
  2. The user navigates to a specific email (morphing a page into another page)
  3. The user navigates to a specific email using a fresh request (regular navigation)
  4. The user enters a new recipient for the focused email (input from the user)
  5. The user sends the email (trigger server events)

Making a fresh request

When navigating to a page the normal way (using the address bar, following a regular hyperlink or navigating to a bookmark) the browser will always make a fresh HTTP GET request and create a new DOM and initialise a new Javascript object instance (i.e. a new context where all user variables are gone).

First the browser does its regular GET for the requested web page

GET /emails HTTP/1.1
Accept: text/html

The web server responds with the web page (in the normal way)

HTTP/1.1 200 OK
Content-Type: text/html
Location: /emails

<!doctype html>
<head>
  <script src="lib/puppetjs/puppet.js"></script>
</head>
<script>
  var puppet = new Puppet();
</script>
<ul>
</ul>

After this small document is loaded, PuppetJs constructor will ask the server for the JSON data to the web page (a.k.a. the view model). By default, PuppetJs will use the same URL as for the web page itself, but instead of requesting HTML (i.e. Accept: text/html) it will request JSON (Accept: application/json).

GET /emails HTTP/1.1
Accept: application/json

The server (such as PuppeteerJs or Starcounter) will then return the JSON data pertaining to the URI.

HTTP/1.1 200 OK
Content-Type: application/json
Location: /emails

{"Emails":[{"Recipient":"joe@example.com","Title":"Hi there"},{"Recipient":"joe@example.com","Title":"Hi again"}]}

PuppetJs will expose the object parsed from the JSON data as callback argument, allowing for usage in client-side templates and frameworks such as Polymer, AngularJS, React, Backbone, etc.

PuppetJs and state-full sessions

PuppetJs does not know about sessions and does not have to know about sessions. Http is a state-less protocol. However, some server create implicit sessions when receiving a browser GET request. By respecting the Location header in the response, this is not a problem for PuppetJs. If the view model is unique to the browser tab session, the Location should address a unique view model. This leaves PuppetJs agnostic to any use of sessions and allows PuppetJs to operate within the boundaries of REST principles. I.e. PuppetJs does not know or care about sessions. PuppetJs simply needs to know where the resource is located. It does not know that the resource (the view model) is bound to a particular session. As you will see, the same logic will apply as far as PuppetJs is concerned. In case the resource is unique to a session, the above response might instead look something like this:

HTTP/1.1 200 OK
Content-Type: application/json
Set-Cookie: Location=%2F__default%2FFDCBA8246CA836DD20000000; path=/

{"Emails":[{"Recipient":"joe@example.com","Title":"Hi there"},{"Recipient":"joe@example.com","Title":"Hi again"}]}

:bulb: PuppetJs understands location path provided as Location cookie (which a web browser can read from the text/html request), X-Location header (which a web browser can read from a XHR request) or Location header (which a generic HTTP client can read, but not a web browser because of security constraints).

Note that the format of the URI or URL of the Location is not relevant for PuppetJs. The server might choose any kind of resource locator it chooses.

Morphing a page into another page

Let's assume that the user wants to navigate to a certain email. The URL for that email is /emails/123 and it is perfectly fine to visit that email in the normal way. However, this would mean that the browser will create a new DOM, reset all animations and create a new Javascript instance (i.e. context). For applications this behaviour is often unwanted as it is potentially slow and as application often have state that needs to be preserved (cursor positions, widgets etc.). In a SPA application, we can instead dynamically morph the changes into the current page.

In PuppetJs, this is done by asking for a JSON-Patch array. The array contains only the changes we need to apply to morph the current page to the new page. When this is done, PuppetJs will simply use the HTML5 history API to update the URL in the browser address bar to reflect the new page. In this way we have two options to visit the new page. One is the normal (slow) way and the other is the "patch" way. In our email example, the traffic would look like this:

GET /emails/123 HTTP/1.1
X-Referer: /emails
Accept: application/json-patch+json

:bulb: PuppetJS sends location path provided as X-Referer header, because the web browser does not allowy to specify the value of the Referer header for XHR.

The use of the referer header might at first be puzzling. It is needed as the patch is the delta between the current page and the page you are navigating to. I.e. it is not enough to know the page you are going to, it is also important to know where you are navigating from.

And as PuppetJs does respect the Location header, the server wanting tab unique sessions can simply do so by placing the data (view model) on a unique location. As the referer is the location of representation retrieved in the previous GET, the referer will look as follows:

GET /emails/123 HTTP/1.1
X-Referer: /temps/aW3dAD472hAF12Q
Accept: application/json-patch+json

The server (such as PuppeteerJs or Starcounter) will then prepare a patch that can be used to update the view model and insert any partial HTML into the DOM.

HTTP/1.1 200 OK
Content-Type: application/json-patch+json

[
 {"op":"replace","path":"/Focused","value":{
      "Title":"Hi there",
      "Content":"How are you doing?",
      "$View":"<input value="{{Recipient}}"/><h2>{{Title}}</h2><p>{{Content}}</p>"}}
]

Regular navigation

When the user enters a address in the browser address bar or when a page is linked from the web the browser always creates a new instance (a new context) for Javascript and the DOM document. This means that all state is lost. In this case, the entire page is retrieved. However, if the session cookie is kept, the same session is used. TODO!

Input from the user

Using modern two-way binding frameworks such as Polymer or AngularJS, interacting with the application user interface means that the view-model is updated. Updates to the view-model are propagated to the server using patches.

PATCH /temps/aW3dAD472hAF12Q HTTP/1.1
Accept: application/json-patch+json
Content-Type: application/json-patch+json

[
 {"op":"replace","path":"/Focused/Recipient/0/Name","value":"arthur"}
]

The server responds with a patch that acknowledges the proposed request:

HTTP/1.1 200 OK
Content-Type: application/json-patch+json

[
 {"op":"replace","path":"/Focused/Recipient/0/Name","value":"Arthur Dent"},
 {"op":"replace","path":"/Focused/Recipient/0/Email","value":"a.dent@example.com"}
]

The server can also reject the proposed change by reverting the value at path to the previous state:

HTTP/1.1 200 OK
Content-Type: application/json-patch+json

[
 {"op":"replace","path":"/Focused/Recipient/0/Name","value":""}
]

Trigger server events

Sometimes you need to invoke an action on the server. This could also be reflected as any other view-model change. For example, we could use Boolean or Number to reflect such action. No avoid miss-synchronization when triggering changes without waiting for server confirmation, we provide advanced synchronization features, like Versioning and JSON Patch OT - those features are transparent to the data flow.

Boolean

Consider the below JSON to represent a view model of a "Compose new mail" page:

HTTP/1.1 200 OK
Content-Type: application/json
Location: /temps/aW3dAD472hAF12Q

{"Focused": {
  "Recepient": {
    "Name": "Arthur Dent",
    "Email": "a.dent@example.com"
  },
  "Subject": "Hello",
  "Content": "Is it me you are looking for?",
  "Send": false
}

The above JSON view model specifies that sending process is not running. It can be triggered from the client to the server using a HTTP request:

PATCH /temps/aW3dAD472hAF12Q HTTP/1.1
Accept: application/json-patch+json
Content-Type: application/json-patch+json

[
 {"op":"replace","path":"/Focused/Send","value":true}
]

That means that client triggered send process. Server may accept that request, and send the composed email message and reset the form fields:

HTTP/1.1 200 OK
Content-Type: application/json-patch+json

[
  {"op":"add","path":"/Emails/2","value":{
    "Recepient": {"Name": "Arthur Dent", "Email": "a.dent@example.com"},
    "Subject": "Hello",
    "Content": "Is it me you are looking for?"
  }},
  {"op":"remove","path":"/Focused/Recepient/0"},
  {"op":"replace","path":"/Focused/Subject","value":""},
  {"op":"replace","path":"/Focused/Content","value":""},
  {"op":"replace","path":"/Focused/Send","value":false}
]

Number

In some cases, numbers are more applicable.

Consider the below JSON to represent a view model of a "Order a drink" page:

HTTP/1.1 200 OK
Content-Type: application/json
Location: /temps/aW3dAD472hAF12Q

{
  "Drinks": {
    "AppleJuice": 3,
    "Beer": 4,
    "NukaCola": 0
  },
  "BartenderSays": "",
  "Cash": 100
}

Now the client may order another beer

PATCH /temps/aW3dAD472hAF12Q HTTP/1.1
Accept: application/json-patch+json
Content-Type: application/json-patch+json

[
 {"op":"replace","path":"/Drinks/Beer","value":4}
]

That means that client triggered buy process. Server may accept that purchase

HTTP/1.1 200 OK
Content-Type: application/json-patch+json

[
  {"op":"replace","path":"/Cash","value":95}
]

Or may decide the client is too drunk for another alcoholic drink and

HTTP/1.1 200 OK
Content-Type: application/json-patch+json

[
 {"op":"replace","path":"/Drinks/Beer","value":3}
 {"op":"replace","path":"/BartenderSays","value":"Why don't you try some Nuka Cola?"}
]

TODO!

You can’t perform that action at this time.