From 71f3cebcc61674360177c15cfad161086a74b39a Mon Sep 17 00:00:00 2001 From: tlrobinson Date: Thu, 3 Sep 2009 02:08:44 -0700 Subject: [PATCH] Deployed on 2009-09-03 --- Rakefile | 8 +- _layouts/default.html | 6 + _posts/2009-07-29-hello-0.1.md | 2 +- browser-deployment.md | 294 +++++++++++++++++++++++++++++ docs/.gitignore | 1 + docs/browser-deployment.md | 290 ++++++++++++++++++++++++++++ docs/engines.md | 32 ++++ docs/index.md | 6 +- docs/lib/binary.md | 4 +- docs/lib/binary.wiki | 4 +- docs/lib/file.md | 2 +- docs/lib/file.wiki | 2 +- docs/narwhal.md | 114 +++++------ docs/narwhal/json.md | 121 ++++++++++++ docs/packages-howto.md | 15 +- docs/posts/2009-07-29-hello-0.1.md | 2 +- docs/quick-start.md | 7 +- docs/sea.md | 14 +- engines.md | 36 ++++ index.md | 6 +- lib/binary.md | 4 +- lib/file.md | 2 +- narwhal.md | 114 +++++------ narwhal/json.md | 125 ++++++++++++ packages-howto.md | 15 +- quick-start.md | 7 +- sea.md | 14 +- 27 files changed, 1083 insertions(+), 164 deletions(-) create mode 100644 browser-deployment.md create mode 100644 docs/.gitignore create mode 100644 docs/browser-deployment.md create mode 100644 docs/engines.md create mode 100644 docs/narwhal/json.md create mode 100644 engines.md create mode 100644 narwhal/json.md diff --git a/Rakefile b/Rakefile index 66c6757d..d9a0177f 100755 --- a/Rakefile +++ b/Rakefile @@ -14,6 +14,8 @@ exclude = { "lib/os/popen" => true } +CHECKOUT_BRANCH = 'master' + DEFAULT_LAYOUT_TEMPLATE = '_layouts/default-template.html' DEFAULT_LAYOUT = '_layouts/default.html' @@ -22,10 +24,10 @@ task :default => [:build] task :all => [:checkout, :build, :runserver] task :checkout do - puts "Checking out 'docs' from master" + puts "Checking out 'docs' from #{CHECKOUT_BRANCH}" rm_rf 'docs', :verbose => false - `git checkout master docs` - `git checkout master README.md` + `git checkout #{CHECKOUT_BRANCH} docs` + `git checkout #{CHECKOUT_BRANCH} README.md` `mv README.md docs/index.md` end diff --git a/_layouts/default.html b/_layouts/default.html index ce01041e..6d58180e 100644 --- a/_layouts/default.html +++ b/_layouts/default.html @@ -1,6 +1,10 @@ --- github_url: "http://github.com/tlrobinson/narwhal" articles: +- name: browser deployment + link: "/browser-deployment.html" +- name: engines + link: "/engines.html" - name: introduction link: "/index.html" - name: lib / binary @@ -9,6 +13,8 @@ link: "/lib/file.html" - name: modules link: "/modules.html" +- name: narwhal / json + link: "/narwhal/json.html" - name: narwhal link: "/narwhal.html" - name: packages howto diff --git a/_posts/2009-07-29-hello-0.1.md b/_posts/2009-07-29-hello-0.1.md index 8f8a9d50..d62af9ea 100644 --- a/_posts/2009-07-29-hello-0.1.md +++ b/_posts/2009-07-29-hello-0.1.md @@ -19,5 +19,5 @@ Check out the [quick start guide](http://narwhaljs.org/quick-start.html) for ins * Complete [securable modules](https://wiki.mozilla.org/ServerJS/Modules/SecurableModules) implementation, in JavaScript, with hooks for native module loading. * Various modules, including `file`, `binary`, `os`, `system`, `args`, and many others. * The "tusk" package manager, currently using [Github](http://github.com/) as a package repository. -* Full support for the Rhino interpreter, and partial support for numerous other [platforms](http://narwhaljs.org/platforms.html). +* Full support for the Rhino interpreter, and partial support for numerous other [engines](http://narwhaljs.org/engines.html). * Full support for Mac OS X, Linux, and other Unixes. Preliminary support for Windows. diff --git a/browser-deployment.md b/browser-deployment.md new file mode 100644 index 00000000..16a67259 --- /dev/null +++ b/browser-deployment.md @@ -0,0 +1,294 @@ +--- +layout: default +title: "browser deployment" +--- + +Deploying Modules to Browsers +============================= + +/!\ Warning: most of the techniques described below have not yet been implemented. This document sets forth the design for a variety of solutions. + +There are a myriad of ways to deploy Narwhal modules to web browsers, although none of them are as familiar as adding the module files with script tags. + +The ideal loader would work for both development or production, with static files hosted by your media server or CDN or with files served by your web application, both in modules and in subsequent inline script tags with or without a load event handler, with blocking calls to require and non-blocking calls to require, and with high performance always, both on the initial page load and on subsequent page loads mostly from cache, and with the minimal download size. + + * development or production + * static or dynamic + * synchronous or asynchronous + * bundled, combined, or individual + * minified or debugable + * same origin, media server, CDN + +Since many of these requirements are mutually exclusive, there is not a single loader that will work for every situation. This document is intended to describe the loader you should use for each particular scenario, and to describe how those loaders work. + + +Usage +----- + +If you are using a JSGI server, you can let Narwhal's `narwhal/server` application embed light-weight module loaders in your HTML. Using this application, the syntax for various use cases remains the same across all configurable scenarios. + + var app = ... + var server = require("narwhal/server"); + app = server.App(app, options); + ... + +Wherever you install this App on your request routing, the app will intercept any requests on the "/javascript" path. Otherwise, the application tracks the `env.SCRIPT_NAME` at that point in the request router so it can construct URL's for the modules. The "/javascript" path is configurable. + + app = server.App(app, { + "path": "/.js" + }); + +The most simple use case embeds a loader that will asynchronously load your main module and its transitive dependencies and then "require" the main module. + + + +If you want to asynchronously preload a set of modules and their transitive dependencies, you can embed a nonblocking preloader and observe a ready signal in subsequent inline scripts. + + + + +If you want to syncronously preload a set of modules and their transitive dependencies, you can embed a blocking preloader. Because of technical limitations, this technique is not available in production with modules hosted statically. In other scenarios, this technique is not advisable since the modules are not very cacheable. When used at all, this technique should only be used at the bottom of the HTML `` tag because it prevents most modern browsers from queueing the download of subsequent static resources in the HTML stream. + + + + +You can use "require.async" and the "promise" module to load modules with calculated identifiers (as opposite to statically analyzable String literal identifiers, like `require("foo")`). + + var Q = require("promise"); + + exports.plugin = function (name) { + return require.async("plugins/" + name); + }; + + Q.when(exports.plugin("widget"), function (widget) { + }); + + +Development +----------- + +For debugging, you want line numbers, file names, and the original source code. Performance is not as much of a concern, but you would like it to resemble the user experience. It would also be optimal if you did not have to perform a build step or restart your web server between writing code and debugging your application. + +You can configure the server application to run in debug mode when Narwhal is in debug mode, as set with the `-d` or `--debug` command line switches. + + app = server.App(app, { + "debug": system.debug + }); + +You can pass any `Boolean` value to the `App` options to configure it for debug scenarios. + + +### Server Component ### + +If you are using a JSGI server, you can use Narwhal's server application to host the individual modules dynamically. The application also sends updated versions of your modules when they change. For performance, the server induces browser caching by selecting a unique URL and an expiration date in the distant future. The module loader provides a global "require" function. + +Each individual module will be wrapped by the server in a bit of code that will register the module factory with the loader. This defers execution of the module and adds it to the loader's module factory table. To avoid interfering with line numbers, the server condenses the entire boilerplate onto a single line. + + require.register({ // no newline + "bar/baz": // no newline + function (require, exports, module, system, print) { // no newline + // module text here + // */ and newline here to break out of comments + } + }); + +In the ideal deployment scenario, you have separated your frontend concerns among behavior (JavaScript), content (raw HTML), and presentation (HTML and CSS). This suggests that each page would have a cacheable module for its programmatic beahvior. The Narwhal server application favors the syntax for this use case, enabling you to embed a small preloader with the necessary data about the main module and its static dependency graph, automatically loading the "main" module when all of its dependencies have arrived. + + + +If you prefer to embed behavior in your HTML, you can embed a preloader and use the ready signal in subsequent inline scripts. Each of the modules and their transitive dependencies will be registered asynchronously and the `ready` signal will be sent when they have all been registered. + + + + +If you absolutely must have a blocking script tag for your modules, the application optionally can download all of the dependencies in a single file, but the price is performance and that your line numbers and file names will no longer correspond with the source. You are unlikely to benefit from browser caching, and it would be unwise to use a blocking script download anywhere but the bottom of the page. + + ... + + + + +There is also an option that will allow you to embed the module factories directly in the text of your HTML, that guarantees blocking but prevents caching. + + ... + + + + + +### Build Step ### + +If you can not use a server side component, but you are willing to perform a build step every time you go from writing code to debugging, Narwhal has Tusk commands that will assist deployment. + +One Tusk command generates a script media directory composed of module factories for the modules of higest precedent for each top level identifier in every package, for the browser engine. + + $ tusk browser --debug build [] + +Another Tusk command generates code snippets to include in your HTML to load these modules. + + $ tusk browser --debug require + $ tusk browser --debug block [ [...]] + $ tusk browser --debug nonblock [ [...]] + $ tusk browser --debug embed [ [...]] + + +### Static Files ### + +If you can not use a server side component and are not willing to perform a build step, but are willing to sacrifice meaningful line numbers and file names in many browsers and to constrain yourself to serving modules from the same host as the origin document, Narwhal provides a loader that can even be used to run modules on the local file system directly from the web browser. You will need to run a Tusk command to construct a directory that contains a symbolic link to every Narwhal module path so the client can search for modules. + + $ tusk browser build-path + +This creates "media/js/{n}/" symlinks in numeric order for each of the directories in `require.paths` for a browser deployment. The client has to make a `HEAD` request for the top level identifier of the module in each subdirectory until it finds the module and issues a `GET`. All of these operations are performed with a blocking XML HTTP Request, which is fine for debugging but very low performance, with many round trips, no parallelism or bundling, and the risk of locking up the entire browser user interface in browsers that can't handle a situation where a synchronous HTTP request is made and the server never responds [citation needed]. + +Then loading a main module is a matter of adding a blocking script with the main module as a query string. + + + + +Production +---------- + +In production, all techniques minify modules before sending them to the browser. + +If you can use a server-side component and you are willing to serve your modules from your origin document server instead of a static media server, the method for hosting those files is similar to hosting those files from the server in developer mode. The key difference is that the server minifies the modules. + +With this architecture, in the future it will be possible to write a smarter module loader that will balance the cost of downloading modules serially as a bundle or individually in parallel by maintaining knowledge in persistent session storage about which individual modules have been presumably cached by the browser. Initial page loads would be expedited by bundling. The client would wait an interval and begin redownloading individual modules, which would be tracked by the server in persistent session storage. On subsequent pages, the server would decide, based on what it knows is present in the browser cache, whether to send individual modules or a bundle. + + app = session.App(app); + app = server.App(app, { + "sessionNamespace": "require", + "cacheReloaderDelay": 2000 // miliseconds + }); + +If you are using a CDN, you can use the same technique as above, serving the JavaScript from origin (where the CDN will look for cache misses) and with CDN URLs (so the client looks for the modules on the CDN). You will need to configure the CDN and the Narwhal server to know about each other. The CDN will need to be configured to look for JavaScript modules on your origin server. The Narwhal application will need to be configured to write CDN URLs instead of origin server URLs. + + app = server.App(app, { + "proxy": "http://your.content-delivery.net/work/js" + }); + +With this architecture, it will be possible in the future to implement a smarter loader that can be configured to proactively invalidate URLs from the proxy's cache by issuing `POST` and `DELETE` HTTP requests to the proxy from the origin server. + +If you would like to host certain modules from other URL's, like common libraries from a free CDN, you can configure the server application to use alternate URLs to load particular individual module factories or factory combinations or bundles. This will hypothetically improve cacheability of certain modules across sites. + + app = server.App(app, { + "catalog": { + "jquery": "http://example.net/js/jquery-1.6.js" + } + }); + + +### Build Step ### + +If your solution must be static with no server-side component, as would be needed to serve from a static media server, you will need to use a Tusk build step to overlay the library directories of all packages into a single directory tree. + + $ tusk browser build [] + +You will also need to configure the server to look for individual modules in that directory tree, so it can configure the embedded loaders properly. + + app = server.App(app, { + "static": "/media/js" + }) + +If you cannot use the JSGI application to construct module loader embeddings, you can generate loader snippets with Tusk as well. Each of these snippets can be placed in a ` + +If you want to asynchronously preload a set of modules and their transitive dependencies, you can embed a nonblocking preloader and observe a ready signal in subsequent inline scripts. + + + + +If you want to syncronously preload a set of modules and their transitive dependencies, you can embed a blocking preloader. Because of technical limitations, this technique is not available in production with modules hosted statically. In other scenarios, this technique is not advisable since the modules are not very cacheable. When used at all, this technique should only be used at the bottom of the HTML `` tag because it prevents most modern browsers from queueing the download of subsequent static resources in the HTML stream. + + + + +You can use "require.async" and the "promise" module to load modules with calculated identifiers (as opposite to statically analyzable String literal identifiers, like `require("foo")`). + + var Q = require("promise"); + + exports.plugin = function (name) { + return require.async("plugins/" + name); + }; + + Q.when(exports.plugin("widget"), function (widget) { + }); + + +Development +----------- + +For debugging, you want line numbers, file names, and the original source code. Performance is not as much of a concern, but you would like it to resemble the user experience. It would also be optimal if you did not have to perform a build step or restart your web server between writing code and debugging your application. + +You can configure the server application to run in debug mode when Narwhal is in debug mode, as set with the `-d` or `--debug` command line switches. + + app = server.App(app, { + "debug": system.debug + }); + +You can pass any `Boolean` value to the `App` options to configure it for debug scenarios. + + +### Server Component ### + +If you are using a JSGI server, you can use Narwhal's server application to host the individual modules dynamically. The application also sends updated versions of your modules when they change. For performance, the server induces browser caching by selecting a unique URL and an expiration date in the distant future. The module loader provides a global "require" function. + +Each individual module will be wrapped by the server in a bit of code that will register the module factory with the loader. This defers execution of the module and adds it to the loader's module factory table. To avoid interfering with line numbers, the server condenses the entire boilerplate onto a single line. + + require.register({ // no newline + "bar/baz": // no newline + function (require, exports, module, system, print) { // no newline + // module text here + // */ and newline here to break out of comments + } + }); + +In the ideal deployment scenario, you have separated your frontend concerns among behavior (JavaScript), content (raw HTML), and presentation (HTML and CSS). This suggests that each page would have a cacheable module for its programmatic beahvior. The Narwhal server application favors the syntax for this use case, enabling you to embed a small preloader with the necessary data about the main module and its static dependency graph, automatically loading the "main" module when all of its dependencies have arrived. + + + +If you prefer to embed behavior in your HTML, you can embed a preloader and use the ready signal in subsequent inline scripts. Each of the modules and their transitive dependencies will be registered asynchronously and the `ready` signal will be sent when they have all been registered. + + + + +If you absolutely must have a blocking script tag for your modules, the application optionally can download all of the dependencies in a single file, but the price is performance and that your line numbers and file names will no longer correspond with the source. You are unlikely to benefit from browser caching, and it would be unwise to use a blocking script download anywhere but the bottom of the page. + + ... + + + + +There is also an option that will allow you to embed the module factories directly in the text of your HTML, that guarantees blocking but prevents caching. + + ... + + + + + +### Build Step ### + +If you can not use a server side component, but you are willing to perform a build step every time you go from writing code to debugging, Narwhal has Tusk commands that will assist deployment. + +One Tusk command generates a script media directory composed of module factories for the modules of higest precedent for each top level identifier in every package, for the browser engine. + + $ tusk browser --debug build [] + +Another Tusk command generates code snippets to include in your HTML to load these modules. + + $ tusk browser --debug require + $ tusk browser --debug block [ [...]] + $ tusk browser --debug nonblock [ [...]] + $ tusk browser --debug embed [ [...]] + + +### Static Files ### + +If you can not use a server side component and are not willing to perform a build step, but are willing to sacrifice meaningful line numbers and file names in many browsers and to constrain yourself to serving modules from the same host as the origin document, Narwhal provides a loader that can even be used to run modules on the local file system directly from the web browser. You will need to run a Tusk command to construct a directory that contains a symbolic link to every Narwhal module path so the client can search for modules. + + $ tusk browser build-path + +This creates "media/js/{n}/" symlinks in numeric order for each of the directories in `require.paths` for a browser deployment. The client has to make a `HEAD` request for the top level identifier of the module in each subdirectory until it finds the module and issues a `GET`. All of these operations are performed with a blocking XML HTTP Request, which is fine for debugging but very low performance, with many round trips, no parallelism or bundling, and the risk of locking up the entire browser user interface in browsers that can't handle a situation where a synchronous HTTP request is made and the server never responds [citation needed]. + +Then loading a main module is a matter of adding a blocking script with the main module as a query string. + + + + +Production +---------- + +In production, all techniques minify modules before sending them to the browser. + +If you can use a server-side component and you are willing to serve your modules from your origin document server instead of a static media server, the method for hosting those files is similar to hosting those files from the server in developer mode. The key difference is that the server minifies the modules. + +With this architecture, in the future it will be possible to write a smarter module loader that will balance the cost of downloading modules serially as a bundle or individually in parallel by maintaining knowledge in persistent session storage about which individual modules have been presumably cached by the browser. Initial page loads would be expedited by bundling. The client would wait an interval and begin redownloading individual modules, which would be tracked by the server in persistent session storage. On subsequent pages, the server would decide, based on what it knows is present in the browser cache, whether to send individual modules or a bundle. + + app = session.App(app); + app = server.App(app, { + "sessionNamespace": "require", + "cacheReloaderDelay": 2000 // miliseconds + }); + +If you are using a CDN, you can use the same technique as above, serving the JavaScript from origin (where the CDN will look for cache misses) and with CDN URLs (so the client looks for the modules on the CDN). You will need to configure the CDN and the Narwhal server to know about each other. The CDN will need to be configured to look for JavaScript modules on your origin server. The Narwhal application will need to be configured to write CDN URLs instead of origin server URLs. + + app = server.App(app, { + "proxy": "http://your.content-delivery.net/work/js" + }); + +With this architecture, it will be possible in the future to implement a smarter loader that can be configured to proactively invalidate URLs from the proxy's cache by issuing `POST` and `DELETE` HTTP requests to the proxy from the origin server. + +If you would like to host certain modules from other URL's, like common libraries from a free CDN, you can configure the server application to use alternate URLs to load particular individual module factories or factory combinations or bundles. This will hypothetically improve cacheability of certain modules across sites. + + app = server.App(app, { + "catalog": { + "jquery": "http://example.net/js/jquery-1.6.js" + } + }); + + +### Build Step ### + +If your solution must be static with no server-side component, as would be needed to serve from a static media server, you will need to use a Tusk build step to overlay the library directories of all packages into a single directory tree. + + $ tusk browser build [] + +You will also need to configure the server to look for individual modules in that directory tree, so it can configure the embedded loaders properly. + + app = server.App(app, { + "static": "/media/js" + }) + +If you cannot use the JSGI application to construct module loader embeddings, you can generate loader snippets with Tusk as well. Each of these snippets can be placed in a `