Skip to content
Browse files

[doc] opadoc: open sourcing opadoc resources

  • Loading branch information...
1 parent ae17609 commit c8784653e361714a564e6cbea78a9313b613750b @Aqua-Ye Aqua-Ye committed Mar 21, 2012
Sorry, we could not display the entire diff because it was too big.
View
218 opadoc/cheat-sheet-db.omd
@@ -0,0 +1,218 @@
+WARNING : MOST OF THE FEATURES DESCRIBED BELOW HAVE NOT YET IMPLEMENTED BY DB3
+
+Overview
+============
+Since Opa S4, the compiler handles several database
+backends. Depending (for moment) on the '--database' parameter the compiler
+generates different database accessors.
+
+Currently, the compiler handles two database backends :
+- db3 : Opa database
+- mongo : [http://www.mongodb.org/](http://www.mongodb.org/)
+
+Independently from the database backend, Opa allows :
+- Definition of a database schema, made of a set of typed paths
+- Generic database paths accessors
+- An API that manipulates database paths
+
+But some features can be backend specific. That's why there are also specific API(s)
+
+
+Databases declaration
+===============
+ type stored = {int x, int y, string v, list(string) lst}
+
+ database dbname {
+ int /i
+ float /f
+ string /s
+ stored /r
+ list(string) /l
+ intmap(stored) /imap
+ stored /set[{x}]
+ }
+
+This piece of code declares the database _dbname_ and defines :
+- 3 paths containing basic values (/i, /f, /s)
+- 1 path containing a record (/r)
+- 1 path containing a list of string
+- 1 path to an intmap
+- 1 path to a database collection of _stored_ record. The key of the declared collection is _x_.
+
+Updating
+============
+One of the new things available with Opa S4 is the way one can update a
+database path. Indeed, previously it was only possible to overwrite a path,
+like this :
+
+ /path/to/data <- x
+
+With _x_ a value with its type equals to that of _/path/to/data_
+
+Opa S4 defined new and most powerful ways to update a path. An update
+operation should be atomic.
+
+Int updating
+--------
+ // Set
+ /dbname/i <- 42
+
+ // Increment
+ /dbname/i ++
+ /dbname/i += 10
+
+ // Decrement
+ /dbname/i --
+ /dbname/i -= 10
+
+ // Sure we can go across records
+ /dbname/r/x ++
+
+Record updating
+--------
+ // Overwrite an entire record
+ x = {x : 1, y : 2, v : "Hello, world", lst:[]}
+ /dbname/r <- x
+
+ // Update a subset of record fields
+ /dbname/r <- {x++, y--}
+
+ /dbname/r <- {x++, v : "I just want to update z and incr x"}
+
+List updating
+--------
+ // Overwrite an entire list
+ /dbname/l <- ["Hello", ",", "world", "."]
+
+ // Removes first element of a list
+ /dbname/l pop
+
+ // Removes last element of a list
+ /dbname/l shift
+
+ // Append an element
+ /dbname/l <+ "element"
+
+ // Append several elements
+ /dbname/l <++ ["I", "love", "Opa"]
+
+Database set/map updating
+--------
+The values stored inside database sets and maps can be updated as we see above.
+The difference is how we access to the elements (more details on [querying section](#Querying).
+Futhermore updates can operates on several paths.
+
+ //Increment the field y of the record stored at position 9 of imap
+ /dbname/imap[9] <- {y++}
+
+ //Increment fields y of records stored with key smaller than 9
+ /dbname/imap[< 9] <- {y++}
+
+ //Increment the field y of record where the field x is equals 9
+ /dbname/set[x == 9] <- {y++}
+
+ //Increment the field y of all records where the field x is smaller than 9
+ /dbname/set[x < 9] <- {y++}
+
+Querying
+============
+In the previous section we saw how to update data with Opa S4. Other
+novelties concern the way to access to data stored inside collections.
+
+Previously the database collections were intmap and stringmap, and the
+way to access to data stored inside database map was :
+
+ // Access to the element stored at position 9
+ /dbname/imap[9]
+
+But it was impossible to access all the elements within a path, except by
+using Db.*_fold_range. But it was just a fold on collection keys
+(not on value) and it was not really user friendly.
+
+Opa S4 introduces database sets and a most powerful way to access to
+a subset of database collections.
+
+Querying operators
+--------
+- == expr : equals expr
+- != expr : not equals expr
+- < expr : lesser than expr
+- <= expr : lesser than or equals expr
+- > expr : greater than expr
+- >= expr : greater than or equals expr
+- in expr : in expr where expr is a list
+- q1 or q2 : satisfy query q1 or q2
+- q1 and q2 : satisfy queries q1 and q2
+- not q : doesn't satisfy q
+- {f1 q1, f2 q2, ...} : the record field f1 satisfy q1 and field f2 satisfy q2
+
+Querying options
+--------
+- skip n : where expr should be an expression of type int, skip the first
+- limit n : where expr should be an expression of type int, returns maximum n results
+- order fld (, fld)+ : where fld specify an order. fld can be <ident> or
+ +<ident> to specify staticaly fld ident should be sorted by incr, -<ident> by
+ decr. Or choose dynamicaly by <ident>=<expr> where <expr> should type of {up}
+ or {down}.
+
+Querying set
+--------
+ k = {x : 9}
+ stored x = /dbname/set[k] // k should be type of set keys, returns a uniq value
+ stored x = /dbname/set[x == 10] // Returns a uniq value because {x} is primary key of set
+
+ dbset(stored, _) x = /dbname/set[y == 10] // Returns a database set because y is not a primary key
+
+ dbset(stored, _) x = /dbname/set[x > 10, y > 0, v in ["foo", "bar"]]
+
+ dbset(stored, _) x = /dbname/set[x > 10, y > 0, v in ["foo", "bar"]; skip 10; limit 15; order +x, -y]
+
+ it = DbSet.iterator(x); // then use Iter module.
+
+Querying map
+--------
+ //Access to a uniq value (like S3)
+ stored x = /dbname/imap[9]
+
+ //Access to a submap where keys are lesser than 9 and greater than 4
+ intmap(stored) submap = /dbname/imap[< 9 and > 4]
+
+
+API(s)
+============
+Some features are available on all backends like read, write, remove, etc
+while other feature has available only on one backend (example : path history).
+That's why we have commons types (and API stdlib.core.database module Db) :
+
+ Db.val_path('a, 'engine)
+ Db.ref_path('a, 'engine)
+
+Where 'a corresponding to type of value stored, then 'engine depending to the backend.
+ //Using db3 backend, Db3.val_path is an instance of Db.val_path
+ Db3.val_path(int) vpath = !/dbname/i
+ //You can use the common API
+ x = Db.read(vpath)
+ //And specific db3 API
+ h = Db3.history(vpath)
+
+
+
+ //But if you use mongo backend, DbMongo.val_path is an instance of Db.val_path
+ DbMongo.val_path(int) vpath = !/dbname/i
+ //You can use the common API
+ x = Db.read(vpath)
+ //But you will have a TYPE ERROR if you use specific db3 API
+ h = Db3.history(vpath)
+
+
+Restriction/Todo
+============
+
+db3
+--------
+- dbset are not yet implemeted
+- Most of updating and querying operations are not yet implemented
+
+mongo
+--------
+- Nothing about migration of database schema
View
208 opadoc/manual.omd/about.omd
@@ -0,0 +1,208 @@
+Introducing Opa
+===============
+
+This is a great time to be a developer: As a community, we are
+changing the world, every day, thanks to the web, the cloud and their
+myriads of applications. Web applications and technologies available
+to developers have evolved tremendously since the first days of the
+web.
+
+Opa is a new generation of web development platform that lets you
+write distributed web applications using a single technology.
+Throughout the pages of this book, we will introduce you to the
+many features of Opa. To quote but a few, Opa is concise, simple,
+concurrent, dynamically distributed, and secure.
+
+What problem does Opa solve?
+----------------------------
+
+The web is evolving at a very fast pace. Unfortunately, the _process_
+of web application development has evolved neither quite as far, nor
+nearly as fast. Indeed, developing a web application in 2011 is not so
+different from developing a PC application in 1991: an impressive
+amount of developer time is spent not working on features or
+improvements but on _gluing_ together antagonist technologies that
+were never designed to work together or in this context. In 1991, the
+technologies were the various hardware components, the low-level
+operating system, the high-level operating system and the memory
+itself, which could only be hacked into compliance through copious
+amounts of machine code. Per se, none of these technologies was wrong,
+but they just failed to fit together -- a problem now known as
+_impedance mismatch_.
+
+Today, the technologies involved include the various incompatible
+browsers, the server components or the database components, to quote
+but a few. As in 1991, none of these technologies is wrong, but they
+do not fit together. Because of this impedance mismatch, developers
+need to add to their application logic copious amounts of glue code
+just to get the components to agree on communication mechanisms and
+application-level protocols. Not only is this glue code tedious,
+repetitive and error-prone, but this is also where most safety and
+security issues arise.
+
+In 1991, the answer to impedance mismatch was Software Development
+Kits, and indeed these SDKs proved very useful at allowing developers
+to develop anything at all, but failed to address the underlying
+issues. This was only achieved later, through better operating
+systems, better development tools, and resulted in today's Java
+Platform, C#, Objective-C or Python, for instance.
+
+In 2011, web frameworks, both client-side and server-side, have the
+same role as the SDKs of the 1991s, with the same success but also the
+same shortcomings. But the problem remains: Building web applications
+is hackish, relies on glue code and is barely reusable.
+
+How Opa solves it
+-----------------
+
+With Opa, we set out to provide another, better, way of developing web
+applications for the cloud. Opa is designed to get you to your
+finished app faster, concentrating only on the interesting parts,
+without the hassle of writing the glue or of using a programming
+language against its original design; also without having to keep such
+a constant watch on low-level security issues.
+
+Opa is a programming language and a standard library comprising a
+database management system, a web server, a server-side framework, a
+client-side framework, a distribution middleware, a security audit
+tool, but without the complexity of deployment, administration, or
+impedance mismatch stemming from the use of many different
+technologies.
+
+Developing with Opa is as simple as it gets: First, write your code
+with the Opa programming language -- and only the Opa programming
+language, no JavaScript, no SQL, or anything such. Then compile the
+code, and let Opa analyze security, generate the code for the browser,
+for the server, for the database, and combine it with the web
+server. Finally, launch your executable and every browser, including
+mobile, can connect to your application. There is need to
+independently deploy, administer or maintain a database management
+system, a web server, frameworks or middleware. If you have lots of
+users, deploying in the cloud is just a command away.
+
+Simply put, Opa is a programming language and its runtime,
+designed to carry web developers from start to finish of their development,
+from early prototype to seamless deployment.
+
+![Single binary with Opa](/resources/manual/img/one_binary.png)
+
+What Opa is not
+---------------
+
+While Opa offers all the features that developers have come to expect
+from a high-end web server, a high-end distributed database management
+system, frameworks or distribution engine, Opa does _not_ resemble
+Apache or Tomcat, MySQL or Oracle SQL, Drupal or Ruby on Rails, Google
+AppEngine or Hadoop. With some effort, Opa could certainly be used to
+mimic some of these components, but that is not how or why Opa was
+designed.
+
+To this day, Opa does not offer a command-line database interface, or
+mechanisms to serve web pages from the file system. Opa is probably
+not well suited for writing embedded or desktop applications or
+anything else which does not run in the web.
+
+What is it good for?
+--------------------
+
+Opa was designed for all users, beginners or seasoned developers, students or
+professionals, but not for all applications.
+
+Indeed, Opa was built from the bottom-up for _cloud_ applications, and
+shines most with applications that require scalability, security,
+interactivity, real-time web and complex data manipulation.
+
+Think of a social application. Numerous users communicating in
+real-time by chat, e-mail, forums or other means, through a rich user
+interface, but also through web services that can be accessed by
+third-party websites or native mobile interfaces. Perhaps these users
+can even modify shared appointment tables or work on some documents
+together. Developers need to combine complex information sources,
+extract the results quickly and get them back to the user in
+real-time. As the number of users grows, servers must be added, both
+to store large amounts of data, to manipulate it, and to respond
+quickly to end-users.
+
+Out of the box, Opa features scalable storage, heavily
+concurrent/distributed manipulation of data, service-oriented API and
+load-balanced web serving. Even better, as social applications
+commonly require considerable computing power, the computing
+performance of Opa ensures that fewer machines are required to handle
+crowds than would be required with other development solutions.
+
+Or think of an e-Commerce store. Highly dynamic web pages, generated
+constantly from the database, and a user experience that may need to
+be altered quickly and without disrupting the flow, to match holiday
+seasons or ongoing ad campaigns, but also a base of orders that needs
+to be kept secure from would-be intruders, and safe from accidents,
+including hardware failures.
+
+Opa is also great for these kinds of applications. Its content
+management engine is second to none, whether in flexibility, power or
+security. Opa's built-in security checks guarantee that the
+application will resist most attacks, while the replicated and
+versioned database ensures that you can always rollback to previous
+data or recover from outages.
+
+We have used Opa to build social networks, e-Commerce applications,
+but also multiplayer web games, communication tools,
+developer-oriented tools -- and of course the CMS behind
+[our website](http://opalang.org).
+
+A first peek at Opa
+-------------------
+
+The rest of the book will introduce you in all details to Opa, but let us take
+a sneak peek preview and take a look at the simplest application: Hello, web.
+
+`server = one_page_server("Hello", -> <>Hello, web!</>)`[opa]
+
+This is the full source code of the application. Without any setup (besides the
+installation of the compiler), obtaining a complete, deployable application is
+only a matter of one step:
+
+`opa hello_web.opa`[sh]
+
+This command produces a stand-alone executable binary, containing
+everything required for execution, including database management, compiled
+requests, server code, client code, user interface and everything that may be
+needed to connect them or to handle malicious connections or inputs. Opa generated
+and checked all this code for you. You may now launch the server:
+
+`./hello_web.exe`[sh]
+
+Or, if you prefer, you may deploy it to a cloud:
+
+`opa-cloud hello_web.exe --host localhost --host my@my_server1 --host my@my_server2`[sh]
+
+Your application is launched, load-balanced, and servers share whichever information
+is necessary.
+
+//Welcome to Opa. It really is that simple.
+
+Summary
+-------
+
+In the following chapters, we will introduce you to the various features and
+use-cases of Opa. Each chapter concentrates on writing one specific application, and
+on how best to achieve this using combinations of skills developed in previous and
+current chapter. At the end of the book, additional reference chapters recapitulate
+in detail all the concepts of the language and the platform.
+
+This book assumes some knowledge of programming (any language should do) and web
+pages (knowledge of HTML and CSS will be useful). Any additional knowledge of
+web applications will let you understand better how much Opa makes development
+_right_.
+
+Getting in touch
+----------------
+
+If you have any question or feedback, do not hesitate to contact us.
+
+A few ways to get in touch:
+
+- Opa [mailing list](https://lists.owasp.org/mailman/listinfo/opa);
+- [Stack Overflow](http://stackoverflow.com/questions/tagged/opa), an excellent site for seeking help with programming problems (do not forget to mark Opa related questions with the ``Opa'' tag);
+- through [Twitter](http://twitter.com/opalang), as opalang.
+
+We will be there!
View
387 opadoc/manual.omd/example.omd
@@ -0,0 +1,387 @@
+Example
+=======
+
+List
+----
+
+- un
+- deux
+ {html}
+ <ul>
+ <li>deux point un</li>
+ <li>deux point deux</li>
+ </ul>
+ {html}
+- trois
+
+Code
+----
+
+`one line code`[js]
+
+```{.opa}
+r = {
+ id(x) = x
+ pair(x) = (x, x)
+ one = 1
+}
+> r : {id : 'v0 -> 'v0; pair : 'v1 -> tuple_2('v1, 'v1); one : int} / ... =
+> { id = <fun> ; one = 1 ; pair = <fun> }
+
+i = 5
+> i : int = 5
+f = 3.14159
+> f : float = 3.14159
+s = "foo"
+> s : string = "foo"
+v = {}
+> v : {} / ... = void
+b = true
+> b : bool = { true = void ; }
+```
+
+```{.opa}
+////////////////////////////////////////////////////////////
+// TYPE :
+////////////////////////////////////////////////////////////
+// 1 - Type constant
+type const0 = int
+type const1 = float
+type const3 = string
+
+// 2 - Type record
+type record0 = {int x0, float y0, string z0}
+type record1 = {int x1, float y1, string z1,}
+
+// 3 - Type sum
+type sum0 =
+ {int s00, float s01, string s02}
+or {int s10, float s11, string s12}
+
+type sum1 = sum0 or record0 or record1
+
+// 4 - Type arrow
+type arrow0('a, 'b) = 'a, 'b -> 'b
+
+// 5 - Type named
+type name('a) = arrow0(int, 'a)
+
+// 6 - Type forall
+type arrow1 = forall('a, 'b). arrow0('a, 'b)
+
+// 7 - Type module
+type IModule = module {
+ string get_message()
+ 'a do_message(string -> 'a)
+}
+
+// 8 - Type functor
+type IFunctor = module(option(string)) {
+ string get_message()
+ 'a do_message(string -> 'a)
+}
+
+
+////////////////////////////////////////////////////////////
+// EXPRESSIONS :
+////////////////////////////////////////////////////////////
+// NB : An unbinded expression @toplvl is equivalent to _ = expr
+// 1 - Constant expression
+1
+
+"Hello"
+
+2.
+
+// 2 - Record
+// { (typ? name : expr,)* }
+{x : 1, y : "y", z : 3,}
+
+// Last ; is optionnal
+{x : 1, y : 2, z : 3}
+
+// tilt binding
+{
+ x = 1
+ y = 2
+ z = 3
+ ex1 = ~{x, y, z}
+ ex2 = ~{x, y:8, z}
+ ex3 = {x:10, y:20, ~z}
+ void
+}
+
+// 3 - Extend Record
+{
+ x = 7
+ z = 6
+ record = {x : 1, y : 2, z : 3,}
+ record = {record with x : 0, z : 5}
+ record = ~{record with x, z}
+ void
+}
+
+// 4 - Match
+// match(expr){
+// case p1 : e1
+// case p2 : e2
+// default : e3
+// }
+match({x : 958}){
+ case {x : 2} : 1
+ case {x : 1} : 2
+ case {int ~x} : x
+ case {x : 3, ...} : 9
+ default : 4
+}
+
+// 5 - Expression block
+// {
+// (binding / do_expr)*
+// final_expr
+// }
+{
+ jlog("let x")
+ x = 1
+ jlog("let y")
+ y = 2
+ jlog("let z")
+ z = 3
+ x + y + z
+}
+
+// 6 - Lambda
+// function return_type? (pattern(,pattern)*)block
+function int (x, y, z){
+ x + y + z
+}
+
+function(~{int x, y, z} as r){
+ jlog("{r}")
+ x+y+z
+}
+
+// 7 - Anonymous module
+module{
+ private msg = "This is my message "
+
+ function get_message(){ msg }
+
+ function do_message(string -> void f){ f(msg) }
+}
+
+// 8 - Anonymous functor
+module (option(string) custom){
+
+ private msg = custom ? "This is my message "
+
+ function get_message(){ msg }
+
+ function do_message(string -> void f){ f(msg) }
+
+}
+
+// 9 - Coerce
+// typ expr
+int 89
+
+string "42"
+
+// 10 - Database declaration
+// 10.1 - Grouped
+// database db0 @local("/tmp/tototest0") {
+// int /path0
+// int /path1 = 10
+// }
+
+// 10.2 - Single
+database int /db0/path2 = 999
+
+// 11 - Dom action sugar :
+// action = domselector (insertaction / cssaction)
+//
+// domselector = #id / .class / *selector
+//
+// insertaction = (= / += / -=) expr
+//
+// cssaction = css= css_properties
+private function dom_sugar(){
+
+ #toto = <h1>Replace</h1>
+ #toto += <h1>Prepend</h1>
+ #toto =+ <h1>Append</h1>
+
+ dom_pointer = Dom.select_children(#toto)
+ *dom_pointer += <h2>before</h2>
+ *dom_pointer =+ <h2>after</h2>
+
+ dom_pointer = Dom.select_inside(dom_pointer, Dom.select_tag("h2"))
+ *dom_pointer css={color : blue}
+}
+
+// 12 - Recursives let
+// 12.1 Recursive value
+recursive x = {
+ x : function(){void x.x()}
+}
+
+{
+ // Local recursive lambda
+ recursive function aux(){aux()}
+
+ recursive x = {
+ y : function(){u()}
+ }
+ and z = {
+ y : function(){x.y()}
+ }
+ and function u(){
+ void x.y()
+ }
+
+ recursive x = 1
+
+ aux
+
+}
+
+// 13 - Subsumption
+option(int) :> ({none} {none})
+
+
+////////////////////////////////////////////////////////////
+// DECLARATIONS
+////////////////////////////////////////////////////////////
+
+// 1 - Values
+// var? type? pattern = expr
+var int x = 1
+
+int xcx = 1
+
+var ~{v1, v2, v3} = {v1 : 1, v2 : 2, v3 :3}
+
+// 2 - Function safe replace @publish
+safe function my_fun(){
+ /db0/path0 <- 90
+ jlog("my_fun"); jlog(" is")
+ jlog(" fun")
+}
+
+// 3 - Module
+module MyModule{
+
+ private msg = "This is my message "
+
+ function get_message(){ msg }
+
+ function 'a do_message(string -> 'a f){ f(msg) }
+
+ private module P{
+ function transform(){
+ msg ^ msg
+ }
+ }
+
+}
+
+IModule module MyModule{
+
+ private msg = "This is my message "
+
+ function get_message(){ msg }
+
+ function 'a do_message(string -> 'a f){ f(msg) }
+
+}
+
+// 4 - Functor
+IFunctor module MyFunctor(option(string) custom){
+
+ private msg = custom ? "This is my message "
+
+ function string get_message(){ msg }
+
+ function do_message(f){ f(msg) }
+
+}
+
+package MyModule2 = MyFunctor(some("My custom message"))
+
+MyModule2.do_message(jlog)
+
+
+
+
+
+
+////////////////////////////////////////////////////////////
+// STARTING THE SERVER
+////////////////////////////////////////////////////////////
+// Proposal
+type Server.conf = {
+ int port,
+ ip netmask,
+ Server.encryption encryption,
+ string name,
+}
+
+/**
+ * Different types of request handler.
+ */
+type Server.handler =
+ /** The most simple request handler. It replies to all incoming
+ request. With a title page [title] and given body by the [page]
+ function */
+ or {string title, xhtml page()}
+
+ /** The most configurable request handler. The [custom] parser takes
+ as input the non-decoded uri from incoming requests and should
+ compute corresponding resource. */
+ or {Parser.general_parser(resource) custom}
+
+ /** Request handler on decoded incoming uri. This handler takes as
+ input the decoded uri from incoming requests which through the
+ [filter]. */
+ or {Server.filter filter, resource dispatch(Uri.relative) }
+
+ /** Request handler which performs on non-decoded uri. Returns
+ resource which uri matches into the [bundle] map. */
+ or {stringmap(resource) bundle}
+
+// Sure we can provide Server.compose(Server.handler, Server.handler)
+// With a function : void Server.start(Server.conf, Server.handler)
+http = { port : 8080, netmask : 0.0.0.0, encryption : {no_encryption}, name : "http"}
+
+Server.start(
+ http,
+ {title : "My first opa server", page : function(){<h1>Hello world</h1>}}
+)
+
+Server.start(
+ {service with name:"secure", port:4343},
+ {title : "My first opa server secured", page : function(){<h1>Hello secure</h1>}}
+)
+```
+
+Block
+-----
+
+{block}[TIP]
+###About _inserts_
+Opa provides _inserts_ to insert expressions inside HTML, inside strings and in
+a few other places that we will introduce as we meet _them_
+{block}
+
+OpaCode
+-------
+
+{opa}
+Hello world
+{opa}
+
+OpaFile
+----
+
+opa-file://../../src/main.opa
+
View
636 opadoc/manual.omd/hello_bindings.omd
@@ -0,0 +1,636 @@
+Hello, bindings -- Binding other languages
+==========================================
+
+It is possible to bind Opa with other languages thanks to a *BSL*: Binding System Library.
+Currently, there is a native support to Javascript on the client side, and to OCaml on the server side.
+
+This chapter shows an example of binding between [Opa, OCaml and Javascript](/manual/Hello--bindings----Binding-other-languages/Binding-Ocaml-and-Javascript),
+as well as an example to see [C primitives calls from Opa](/manual/Hello--bindings----Binding-other-languages/Binding-C).
+
+A complete example of binding with an external Javascript library is available in the chapter
+[Hello, reCaptcha](/manual/Hello--reCaptcha-(and-the-rest-of-the-world)).
+
+In Opa, a function written in an other language is called a
+[foreign function](/manual/The-core-language/Foreign-function-interface), a _bypass_,
+or an _external primitive_. All this designations mean that the implementation of the
+function is not written in Opa.
+
+Part of the Opa standard library uses external primitives, such as low level arithmetic
+operations, bindings with the DOM API on the client side, etc.
+
+Using external primitives in Opa applications has two main motivations :
+
+- provide a new functionality not available in the standard library, and the implementation cannot easily be written in Opa (e.g. using low level system call). Although it does not often happen because of the completeness of the Opa standard library, it remains useful for specific applications.
+- bind with existing code: you want to use an external library without reimplementing it in Opa (e.g. numerical computing library), or waiting for it to be rewritten in Opa (e.g. for a quick prototyping, hybrid applications)
+
+Since _OCaml_ supports bindings for many languages, it is possible, on the server side of an Opa applications,
+to use functions written in any language supporting an OCaml bindings (C, Java, Python, etc.)
+
+Although we are working on a native support for binding Opa and _C_ applications,
+this is not available in the distribution of Opa yet. Waiting for this support,
+it is nevertheless possible to bind Opa and _C_ via _OCaml_
+(This chapter contains [an example]((/manual/Hello--bindings----Binding-other-languages/Binding-C)) using a external primitive implemented in _C_,
+using the _OCaml/C_ binding)
+
+{block}[CAUTION]
+### Client/Server distribution
+
+When you define an external primitive, the function is available only on the
+side of the implementation. (Javascript on the client side, _OCaml_, _C_ on the server side).
+If you do not provide both client and server sides of the implementation of the external primitive,
+it has an impact on the client/server distribution of the Opa code.
+You have to think about it before deciding to use an external binding.
+Writing a function in pure Opa gives you the guarantee that the function is available on
+both side.
+{block}
+
+Binding Ocaml and Javascript
+----------------------------
+
+This part shows a complete example, including compilation command line.
+
+ user@computer:~/$ ls
+ hello_binding.opa plugin.js plugin.ml Makefile
+
+##### Makefile
+file://opa_binding_examples/Makefile
+
+### External implementation
+
+Implementations goes into files, and file names should be called the same way between
+server and client side implementation.
+
+The implementation files are files written in the syntax of the bound language,
+adding some Opa directives for registering the primitives and the types exported
+for being used from Opa code.
+
+In Javascript, we use a special syntax for introducing function names (`##args`),
+used for simulating namespaces. Using the directive `##args` following a `##register`
+directive defining the function `stammer` in the file `plugin` will introduce
+the function named `plugin_stammer`. The use of `##args` is optional, we can write instead:
+
+##### plugin.ml
+[ocaml]file://opa_binding_examples/plugin.ml
+
+##### plugin.js
+[js]file://opa_binding_examples/plugin.js
+
+ ##register stammer : string -> string
+ function plugin_stammer(s)
+ {
+ return "client is stammering: " + s + s + s ;
+ }
+
+but it is not advised to handle manually the namespaces.
+Using the directive `##args` implies a check of arity with the registered type.
+
+### Plugin Compilation
+
+The plugin files are compiled using the tool _opa-plugin-builder_,
+distributed with the _Opa_ compiler. You can refer to the documentation of the tool
+for more details, and the part about [opp files](/manual/Filename-extensions/opp).
+_opa-plugin-builder_ produces an _opp_ directory from plugin sources files.
+The name of the plugin should be specified with the option `-o` :
+
+ user@computer:~/$ opa-plugin-builder plugin.js plugin.ml -o plugin
+ user@computer:~/$ ls
+ hello_binding.opa plugin.js plugin.ml plugin.opp
+
+### Plugin browsing
+
+The plugin `plugin.opp` is ready to be linked with an Opa application.
+You can browse the plugin, using the tool _opa-plugin-browser_.
+
+ user@computer:~/$ opa-plugin-browser plugin.opp
+ (you enter a tiny shell-like environment)
+ bslbrowser:/$ ls
+ + module <plugin>
+ bslbrowser:/$ cd plugin
+ bslbrowser:plugin/$ ls
+ + in module <plugin> \:
+ stammer \: string -> string {js, ml}
+ (try also: 'help')
+
+When you are browsing a plugin, you can check that the exported functions are correctly
+stored in the _opp_, and get familiar with the information stored
+about the external primitives (try `ls -v`).
+
+[[key_normalization]]
+#### Key normalization
+
+External Primitives are indexed using a key normalisation. The key is obtained
+from the path (file of declaration + module hierarchy), by concatenation
+of all path elements, using _underscore_ between elements replacing dots by _underscores_,
+and finally converted to lowercase.
+
+In this example, the key of the function `stammer` is `plugin_stammer`.
+This key is used for referring to this function from the Opa code.
+
+For implementing primitives on both sides, it is mendatory for the server and the client primitives
+to have the same name. This implies to choose valid filenames for Ocaml modules.
+For files implementing primitives on the client only, it is possible to use dots and dash in filenames,
+they are transformed for computing the key : `-` is replaced by `_dash_` and `.` is replaced by `_dot_.
+
+Example: `jquery-1.6.1.js` becomes : `jquery_dash_1_dot_6_dot_1.js`
+
+### Binding in Opa
+
+The [syntax](/manual/The-core-language/Foreign-function-interface) for introducing external primitives in Opa
+is `%% key %%`, referring to the `key` of the external primitive.
+
+ plugin_stammer = %% plugin_stammer %%
+
+{block}[TIP]
+The keys found in the syntax are also normalized by the compiler,
+which means that calls are case insensitive, and so the following lines are equivalent:
+{block}
+
+ plugin_stammer = %%plugin.stammer%%
+ plugin_stammer = %% plugin.stammer %%
+ plugin_stammer = %% Plugin.stammer %%
+ plugin_stammer = %% pLuGin_staMmeR %%
+
+You can find there a simple _Opa_ application for calling either the client or the server
+side implementation of the function `stammer`, with an input for giving the argument.
+
+##### hello_bindings.opa
+[opa|fork=opa_bindings_examples]file://opa_binding_examples/hello_bindings.opa
+
+Compiling the application requires an extra argument to be passed to the _Opa_ compiler
+so that it finds the type of the external primitive, and find the
+implementation to link with the server. Note that parameters are not ordered.
+
+ user@computer:~/$ opa plugin.opp hell_bindings.opa
+ user@computer:~/$ ./hell_bindings.exe
+
+Binding C
+---------
+
+This part shows an integration of C code in an Opa application, via an _OCaml_
+binding. A native support is currently in integration, available in a future
+version of Opa.
+
+ user@computer:~/$ ls
+ freq.opa freq.c c_binding.ml Makefile
+
+##### Makefile
+file://opa_binding_examples/c_binding/Makefile
+
+##### freq.c
+[c]file://opa_binding_examples/c_binding/freq.c
+
+##### c_binding.ml
+[ocaml]file://opa_binding_examples/c_binding/c_binding.ml
+
+##### freq.opa
+[opa|fork=opa_bindings_examples]file://opa_binding_examples/c_binding/freq.opa
+
+### Compilation
+
+ user@computer:~/$ ocamlopt.opt freq.c
+ user@computer:~/$ opa-plugin-builder c_binding.ml -o c_binding
+ user@computer:~/$ opa --mllopt $PWD/freq.o c_binding.opp freq.opa
+ user@computer:~/$ ./freq.exe
+
+For more details about integration between C/OCaml, you can refer to the
+manual of Objective Caml.
+
+Documentation
+-------------
+
+This paragraph describes the exhaustive list of directives handled by `opa-plugin-builder` in OCaml and Javascript files.
+The [conventions adopted](/manual/The-core-language/Formal-description) to describe the syntax of theses directives are the same as the one used
+for describing the core language syntax.
+
+The directives are used to declare the types and values to export in Opa.
+Opa is type-checked at compile time, hence it is mandatory to declare the types of the external primitives.
+
+{block}[CAUTION]
+There is a restriction about types available in the interface between Opa and foreign implementations.
+You should keep in mind that there are differences of representation of values between languages, a fortiori between
+Opa, OCaml, and Javascript,
+and a value passed through the register interface should most of the time be translated. For the most common types, an automatic
+projection is done by Opa (constants, bool, option). For more complex values, you will need to handle the projection
+manually, via the ServerLib //<...>
+on the server side, and the ClientLib //<...>
+on the server side.
+When you are designing a new plugin, you should keep in mind that value
+projections may be inefficient, and it often is more convenient to use primitive types only.
+{block}
+
+### General syntax
+
+#### Syntax of types
+
+```
+type ::=
+| int
+| string
+| float
+| void
+| bool
+| option(<type>)
+| <type>* sep , -> <type>
+| <type-var>
+| <type-ident> ( <type>* sep , )?
+| opa[<type>]
+| ( <type> )
+```
+
+#### Syntax of directives
+
+```
+directive ::= ## <properties>? <directive_name> <arguments>
+
+properties ::= [ <property>+ sep , ]
+property ::=
+| <tag>
+| <tag> : <value>
+
+tag ::= <see tags documentation, depends of the directive_name>
+
+directive_name ::=
+| extern-type
+| opa-type
+| module
+| endmodule
+| register
+| args
+| property
+
+arguments ::= <see arguments documentation, depends of the directive_name>
+```
+
+The properties are used for setting options to directives, for modifying their behavior,
+or for attaching extra information to them. The directive `##property` is used for
+changing the behavior of the other directives until the end of the current file.
+The supported tags are specific to the directive.
+In standard cases, you should not need tags and properties (advanced use only)
+
+##### Syntax of arguments
+
+```
+verbatim-ocaml ::= some valid OCaml code
+
+type-ident ::= same as Opa type ident
+
+parameters ::=
+| ( <type-var>+ sep , )
+```
+
+##### Syntax of arguments for [##extern-type](/manual/Hello--bindings----Binding-other-languages/External_types)
+
+
+```
+arguments(##extern-type) ::=
+| <type-ident> <parameters>? ( = <verbatim-ocaml> )?
+```
+
+##### Example:
+
+```
+##extern-type hashset('a) = ('a, unit) Hashtbl.t
+##extern-type black
+```
+
+##### Syntax of arguments for [##opa-type](/manual/Hello--bindings----Binding-other-languages/Opa_types)
+
+```
+arguments(##opa-type) ::=
+| <type-ident> <parameters>?
+```
+
+##### Example:
+
+```
+##opa-type option('a)
+```
+
+##### Syntax of arguments for ##register
+
+```
+arguments(##register) ::=
+| <name> : <type>
+| <name> \ <name> : <type>
+| <name> \ `<verbatim-ocaml>` : <type>
+```
+
+##### Example:
+
+ ##register int_of_string : string -> option(int)
+ ##register int_add \ `Pervasives.( + )` : int, int -> int
+
+{block}[TIP]
+_Backslash_ '*\*' can also be used to dispatch a directive on several lines if
+it does not fit on one line of source, e.g. a function using big types.
+{block}
+
+##### Example:
+
+```
+##register my_function : \
+ (int, int, int, int -> int), \
+ int -> \
+ int
+```
+
+#### Tags
+
+Tags are needed for advanced uses only. The casual user can simply ignore them.
+
+{table}
+{* directive name | supported tags | arguments *}
+{| extern-type | opacapi | type definition |}
+{| opa-type | opacapi | type definition |}
+{| module | - | module name |}
+{| endmodule | - | - |}
+{| register | backend, opacapi, restricted, no-projection, cps-bypass | implementation name, and type of the value |}
+{| args | - | formal parameters |}
+{| const | - | - |}
+{| property | mli, endmli | - |}
+{table}
+
+### About Supported Types
+
+{block}[TIP]
+Compiling OCaml plugins, _opa-plugin-builder_ will perform a type check to ensure that the functions
+defined in plugins have consistent types with respect to their registered definition.
+We are also working on a step of Javascript validation to ensure some consistent properties (functions arities, etc.)
+{block}
+
+##### Example: the following definitions will generate a static error :
+
+```
+##register foo : int, int -> int
+let foo a b = a +. b
+
+##register foo : int, int -> int
+##args(x)
+{
+ return x+x;
+}
+```
+
+The supported types in the interface are :
+
+- void and constants;
+- bool;
+- option;
+- external types;
+- opa types;
+- polymorphic types.
+
+Before writing a new plugin, you should think about the kind of types you will need to manipulate,
+and the side where the values of theses types will be processed: server only, client only, or both.
+The latter case requires types to be serializable.
+
+{block}[CAUTION]
+You should also try not to use functional arguments.
+The Opa code is rewritten in CPS, and the external function are generally not in CPS.
+Opa proceed to a projection of a CPS function into a non CPS function for passing it to the external
+functional primitive, using an hint about the time needed for this function to compute.
+In general it is not possible to know statically a bound of the time needed for a functional argument to compute :
+typically, in case of an asynchronous function (e.g. with client/server I/O) you may get a runtime error when the
+external functional primitive will be executed, meaning that the response of the function is not available at
+the end of the timeout used in the projection.
+Nevertheless, if you really need to have asynchronous functional arguments, you can contact MLstate
+for a support, or browse the source code of the standard library (looking for the tag _cps-bypass_)
+{block}
+
+#### Automatic projections
+
+As often when you write a binding between several languages, you have to deal with the fact that
+values can have different runtime representations in each language. The approach used e.g. in the OCaml/C binding,
+is to manipulated OCaml values using a special set of macros and function (`CAMLparam`, `CAMLreturn`, etc.), and
+to use a manual projection of values between the two languages, e.g. using _Val_int_ for building an OCaml int
+from a C int, etc.
+
+```
+value add_ocaml_int(value a, value b)
+{
+ int ia, ib;
+ CAMLparam2 (a, b);
+ ia = Int_val(a);
+ ib = Int_val(b);
+ CAMLreturn (Val_int(ia + iab));
+}
+```
+
+The code resulting of this manual projection is quite verbose, and the experience has shown that it is easy
+to write incorrect code, e.g. forgetting part of projections, especially if the code is not typed.
+
+From this constatation, we decided in Opa to handle automatically the projections of values for the most common
+types allowed in the interface between Opa and foreign languages.
+
+Let's see an example:
+
+```
+(*
+ In the ServerLib, we have the following manipulation of int values :
+
+ type ty_int
+
+ val wrap_int : int -> ty_int
+ val unwrap_int : ty_int -> int
+*)
+let add a b =
+ let ia = ServerLib.unwrap_int a in
+ let ib = ServerLib.unwrap_int b in
+ let ic = ia + ib in
+ ServerLib.wrap_int ic
+```
+
+This example implements a function adding two opa ints using the OCaml + primitive combined with projection between
+OCaml ints and Opa ints.
+
+Now, we will save the code into the file `int.ml`, try to register this function and building an Opa plugin.
+
+```
+##register add : int, int -> int
+let add a b =
+ let ia = ServerLib.unwrap_int a in
+ let ib = ServerLib.unwrap_int b in
+ let ic = ia + ib in
+ ServerLib.wrap_int ic
+```
+
+Let's try to compile it:
+
+```{.sh}
+user@computer:~/$ opa-plugin-builder int.ml -o int
+> File "intMLRuntime.ml", line 1, characters 0-1:
+> Error: The implementation intMLRuntime.ml
+> does not match the interface intMLRuntime.cmi:
+> Modules do not match:
+> sig
+> val add : ServerLib.ty_int -> ServerLib.ty_int -> ServerLib.ty_int
+> end
+> is not included in
+> sig val add : int -> int -> int end
+> Values do not match:
+> val add : ServerLib.ty_int -> ServerLib.ty_int -> ServerLib.ty_int
+> is not included in
+> val add : int -> int -> int
+```
+
+We get there an error, the code does not type. Why ?
+
+The function manipulates int, and this is pretty clear that if you want to register a function working on int,
+you want to use it in Opa, so you will need to projects values. To avoid this, _opa-plugin-builder_ transforms
+the code and inserts the projections, unless you specify in the interface that you are explictly manipulating
+opa values, and want to handle the projection yourself:
+
+```
+##register add : opa[int], opa[int] -> opa[int]
+let add a b =
+ let ia = ServerLib.unwrap_int a in
+ let ib = ServerLib.unwrap_int b in
+ let ic = ia + ib in
+ ServerLib.wrap_int ic
+```
+
+```{.sh}
+user@computer:~/$ opa-plugin-builder int.ml -o int
+# compilation is successfull
+```
+
+```
+##register add : int, int -> int
+let add a b = a + b
+```
+
+```{.sh}
+user@computer:~/$ opa-plugin-builder int.ml -o int
+# compilation is successfull
+```
+
+{block}[TIP]
+If the version of the compiler knows that some types have the same representation in Opa or in Ocaml,
+projections will not be generated.
+{block}
+
+#### Representation of values
+
+##### OCaml
+
+{table}
+{* type | interface syntax | ocaml representation | opa representation *}
+{| int | int VS opa[int] | int | ServerLib.ty_int |}
+{| float | float VS opa[float] | float | ServerLib.ty_float |}
+{| string | string VS opa[string] | string | ServerLib.ty_string |}
+{| void | void VS opa[void] | unit | ServerLib.ty_void |}
+{| bool | bool VS opa[bool] | bool | ServerLib.ty_bool |}
+{| option | option('a) VS opa[option('a)] | 'a option | 'a ServerLib.ty_option |}
+{table}
+
+##### Javascript
+
+{table}
+{* type | interface syntax | ocaml representation | opa representation *}
+{| int | int VS opa[int] | int | ClientLib_ty_int |}
+{| float | float VS opa[float] | float | ClientLib_ty_float |}
+{| string | string VS opa[string] | string | ClientLib_ty_string |}
+{| void | void VS opa[void] | undefined or null | ClientLib_ty_void |}
+{| bool | bool VS opa[bool] | bool | ClientLib_ty_bool |}
+{| option | option('a) VS opa[option('a)] | None if null or Some 'a otherwise | 'a ClientLib_ty_option |}
+{table}
+
+An API is available for manipulating opa values, on the server side as well as on the client side. //<...>
+
+#### External type VS Opa type
+
+If you want to use an external library providing an API manipulating types such than you do not need
+to see in Opa the implementation of these types, you can refer to the documentation
+of [external types](/manual/Hello--bindings----Binding-other-languages/External_types).
+
+If you want to be able to manipulate Opa values in the external implementation, you can refer to the
+documentation of [Opa types](/manual/Hello--bindings----Binding-other-languages/Opa_types).
+
+[[External_types]]
+### External types
+
+External types are used for manipulating in Opa foreign values without
+manipulating the raw implementation of their types, using a foreign interface.
+Once in Opa, each external type is unifiable only with itself. The implementation is hidden in Opa.
+
+An extern type is generally available only in one side : the side of its implementation.
+Nevertheless, it is possible to define a common API on the client and the server side,
+and providing explicit (de)serialization functions to make types available
+on both side, and able to travel from one side to the other.
+
+{block}[TIP]
+In the standard library, low level arrays are implemented using extern-type definitions.
+{block}
+
+#### Complete example using external types
+
+This example shows how to use the module `Big_int` of the _OCaml_ standard library to add
+2 bigints given by a user via a simple web interface.
+
+ user@computer:~/$ ls
+ bigint.ml bigint.opa Makefile
+
+##### Makefile
+file://opa_binding_examples/external_types/Makefile
+
+##### bigint.ml
+file://opa_binding_examples/external_types/bigint.ml
+
+##### bigint.opa
+file://opa_binding_examples/external_types/bigint.opa
+
+[[Opa_types]]
+### Opa types
+
+Using Opa types in plugin requires a good knowledge of Opa, and can be considered as advanced use.
+The purpose of Opa types is to create or to wander Opa structures in the external implementation.
+
+This is needed when you want to return or to take in argument complex values.
+
+{block}[TIP]
+For people familiar with C/Ocaml bindings, the manipulation of opa values using the server_lib or the client_lib
+correspond to the macro CAMLparam, CAMLreturn, Store_field, etc.
+{block}
+
+//[[client_lib]]
+//#### ClientLib
+//Coming soon...
+// TODO!
+
+//[[server_lib]]
+//#### ServerLib
+//Coming soon...
+// TODO!
+
+#### Complete example using Opa types manipulation
+
+This example shows how to use the opa-type construction from OCaml using the _ServerLib_. //<...>
+We take the opportunity to bind _Opa_ with the NDBM database.
+
+The example is a tiny application for submitting triplets in a persistent external database,
+finding back the information, and guessing the age of the Captain.
+
+For compiling the example you need:
+-ndbm dev library;
+-ndbm OCaml binding.
+
+{block}[CAUTION]
+This example is of course heretic because _Opa_ has its own database.
+The purpose is to take benefits of a small example to show that you can really extend _Opa_
+as much as you want using _OCaml_ bindings (ndbm is implemented originally in C).
+{block}
+
+ user@computer:~/$ ls
+ dbm.ml dbm.opa Makefile
+
+##### Makefile
+file://opa_binding_examples/opa_types/Makefile
+
+##### dbm.ml
+file://opa_binding_examples/opa_types/dbm.ml
+
+##### dbm.opa
+[opa|fork=opa_bindings_examples]file://opa_binding_examples/opa_types/dbm.opa
+
+### Exercise
+Add a primitive which return a opa-list of entries, and add a button to show all entries of the database.
View
779 opadoc/manual.omd/hello_chat.omd
@@ -0,0 +1,779 @@
+Hello, chat
+===========
+
+Real-time web applications are notoriously difficult to develop. They require a
+complex and error-prone infrastructure to handle communications between client
+and server, and often from server to server, in addition to deep security checks
+against specific attacks that may prove quite subtle.
+
+Opa makes real-time web simple. In this chapter, we will see how to program a
+complete web chat application in Opa -- in only 20 lines of code, and without
+compromising security. Along the way, we will introduce the basic concepts of
+the Opa language, but also user interface manipulation, data structure
+manipulation, embedding of external resources, as well as the first building
+bricks of concurrency and distribution.
+
+Overview
+--------
+
+Let us start with a picture of the web chat we will develop in this chapter:
+
+![Final version of the Hello chat application](/resources/manual/img/hello_chat/result.png)
+
+This web application offers one chatroom. Users connecting to the web application
+with their browser automatically join this chatroom and can immediately
+start discussing in real-time. On the picture, we have two users, using regular
+web browsers. For the sake of simplicity, in this application, we choose the name
+of users randomly.
+
+If you are curious, the full source code of the application is listed
+at the end of this chapter. In the rest of the chapter, we will walk
+you through all the concepts and constructions: the communication
+infrastructure for the chatroom, the user interface, and finally, the
+main application.
+
+Setting up communication
+------------------------
+
+A chat is all about communicating messages between users. This means that we need to decide of what _type_ of messages we wish to transmit,
+as follows:
+
+`type message = {string author, string text}`[opa]
+
+This extract determines that each `message` is composed of two fields: an `author` (which is a `string`, in other words, some text)
+and a `text` (also a `string`).
+
+{block}[TIP]
+### About types
+_Types_ are the shape of data manipulated by an application. Opa uses
+types to perform checks on your application, including sanity checks
+(e.g. you are not confusing a length and a color) and security checks (e.g. a
+malicious user is not attempting to insert a malicious program inside a web page
+or to trick the database into confusing informations). Opa also uses types to
+perform a number of optimizations.
+
+In most cases, Opa can work even if you do not provide any type information,
+thanks to a mechanism of _type inference_. However, in this book, for
+documentation purposes, we will put types even in a few places where they are
+not needed.
+{block}
+
+We say that _type_ `message` is a _record_ with two _fields_, `author` and
+`text`. We will see in a few minutes how to manipulate a `message`.
+
+At this stage, we have a complete (albeit quite useless) application. Should you
+wish to check that your code is correct, you can _compile_ it easily. Save your
+code as a file `hello_chat.opa`, open a terminal and enter
+
+###### Compiling Hello, Chat
+`opa hello_chat.opa`[sh]
+
+Opa will take a few seconds to analyze your application, check that everything
+is in order and produce an executable file. We do not really need that file yet,
+not until it actually does something. Opa will inform you that you have no
+_server_ in your application -- at this stage, your application is not really
+useful -- but that is ok, we will add the _server_ shortly.
+
+So far, we have defined `message`. Now, it is time to use it for communications. For
+this purpose, we should define a _network_. Networks are a unit of communication
+between browsers or between servers. As you will see, communications are one of
+the many domains where Opa shines. To define one, let us write:
+
+{block}[TIP]
+### Networks
+A network is a real-time web construction used to broadcast messages from one
+source to many observers. Networks are used not only for chats, but also for
+system event handling or for user interface event handling.
+
+Networks themselves are built upon a unique and extremely powerful paradigm
+of _distributed session_, which we will detail in a further chapter.
+{block}
+
+`room = Network.network(message) (Network.cloud("room"))`[opa]
+
+This extract defines a _cloud network_ called `room` and initially empty. As
+everything in Opa, this network has a type. The type of this network is
+`Network.network(message)`, marking that this is a network used to transmit
+informations with type `message`. We will see later a few other manners of
+creating networks for slightly different uses.
+
+And that is it. With these two lines, we have set up our communication
+infrastructure -- yes, really. We are now ready to add the user interface.
+
+Defining the user interface
+---------------------------
+
+To define user interfaces, Opa uses a simple HTML-like notation for the
+structure, regular CSS for appearance and more Opa code for interactions.
+There are also a few higher-level constructions which we will introduce
+later, but HTML and CSS are more than sufficient for the following few chapters.
+
+For starters, consider a possible skeleton for the user interface:
+
+###### Skeleton of the user interface (incomplete)
+
+```
+<div id=#conversation />
+<input id=#entry />
+<input type="button" value="Post" />
+```
+
+If you are familiar with HTML, you will recognize easily that this
+skeleton defines a few boxes (or `<div>`), with some names (or `id`),
+as well as a text input zone (or `<input>`) called `entry`. We will
+use these names to add interactions and style. If you are not familiar
+with HTML, it might be a good idea to grab
+[a good HTML reference](https://developer.mozilla.org/En/HTML) and check
+up the tags as you see them.
+
+{block}[TIP]
+### About HTML
+There is not much more magic about HTML in Opa than the special
+syntax. For instance, the skeleton that we just defined is a regular
+Opa value, of type `xhtml`. You can for instance inspect its structure
+(with a `match` construct that we will see later), apply to it
+functions accepting type `xhtml`, or use it as the body of a function.
+{block}
+
+Actually, for convenience, and because it fits with the rest of the library, we will
+put this user interface inside a function, as follows:
+
+###### Skeleton of the user interface factorized as a function (still incomplete)
+
+```
+function start() {
+ <div id=#conversation />
+ <input id=#entry />
+ <input type="button" value="Post" />;
+}
+```
+
+This extract defines a _function_ called `start`. This function takes no _argument_
+and produces a HTML-like content. As everything in Opa, `start` has a type. Its
+type is `-> xhtml` .
+
+{block}[TIP]
+### About functions
+_Functions_ are bits of the program that represent a treatment that can be
+triggered as many times as needed (including zero). Functions that can have
+distinct behaviors, take _arguments_ and all functions _produce_ a
+_result_. Triggering the treatment is called _calling_ or _invoking_ the
+function.
+
+Functions are used pervasively in Opa. A function with type `t1, t2, t3 -> u`
+takes 3 arguments, with respective types `t1`, `t2` and `t3` and produces
+a result with type `u`. A function with type `-> u` takes no arguments
+and produces a result with type `u`.
+
+The main syntax for defining a function is as follows:
+```
+function f(x1, x2, x3) {
+ fun_body
+}
+```
+
+Similarly, for a function with no argument, you can write
+
+```
+function f() {
+ fun_body
+}
+```
+To call a function `f` expecting three arguments, you will need to write `f(arg1, arg2, arg3)`. Similarly, for a function expecting no argument, you will write `f()`.
+{block}
+
+{block}[WARNING]
+### Function syntax
+// FIXME: check
+In `function f(x1, x2, x3) { fun_body }` there is no space between `f` and `(`.
+Adding a space changes the meaning of the extract and would cause an error during compilation.
+{block}
+
+At this stage, we can already go a bit further and invent an author name, as follows:
+
+###### Skeleton of the user interface with an arbitrary name (still incomplete)
+
+```
+function start() {
+ author = Random.string(8);
+ <>
+ <div id=#conversation />
+ <input id=#entry />
+ <input type="button" value="Post" />
+ </>;
+}
+```
+
+This defines a value called `author`, with a name composed of 8 random characters.
+
+With this, we have placed everything on screen and we have already taken a few
+additional steps. That is enough for the user interface for the moment, we should
+get started with interactivity.
+
+Sending and receiving
+---------------------
+
+We are developing a chat application, so we want the following interactions:
+
+- at start-up, the application should _join_ the room;
+- whenever a message is broadcasted to the room, we should display it;
+- whenever the user presses return or clicks on the button, a message should be broadcasted to the room.
+
+For these purposes, let us define a few auxiliary functions.
+
+###### Broadcasting a message to the room
+
+```
+function broadcast(author) {
+ text = Dom.get_value(#entry);
+ message = ~{author, text};
+ Network.broadcast(message, room);
+ Dom.clear_value(#entry);
+}
+```
+
+This defines a function `broadcast`, with one argument `author` and performing the following operations:
+
+- read the text entered by the user inside the input called `entry`, call this text `text`;
+- create a record with two fields `author` and `text`, in which the value of field `author` is `author` (the argument to the function) and the value field `text` is `text` (the value just read from the input), call this record `message`;
+- call Opa's network broadcasting function to broadcast `message` to network `room`;
+- clear the contents of the input.
+
+As you can start to see, network-related functions are all prefixed by
+`Network.` and user-interface related functions are all prefixed by `Dom.`. Keep
+this in mind, this will come in handy whenever you develop with Opa. Also note
+that our record corresponds to type `message`, as defined earlier. Otherwise,
+the Opa compiler would complain that there is something suspicious: indeed, we have defined
+our network to propagate messages of type `message`, attempting to send a message
+that does not fit would be an error.
+
+{block}[TIP]
+### About `Dom`
+If you are familiar with web applications, you certainly know about
+the DOM already. Otherwise, it is sufficient to know that DOM, or
+Document Object Model, denotes the manipulation of the contents of a
+web page once that page is displayed in the browser. In Opa, elements
+in the DOM have type `dom`. A standard way to access such an element
+is through the selection operator `#`, as in `#entry` which selects
+the element of id `"entry"` (ids must be unique in the page). A
+variant of the selection operator is `#{id}`, which selects the DOM
+element whose id is the value of `id` (so `id` must be of type
+`string`).
+{block}
+
+Speaking of types, it is generally a good idea to know the type of
+functions. Function `broadcast` has type `string -> void`, meaning that it
+takes an argument with type `string` and produces a value with type
+`void`. Also, writing `{author:author, text:text}` is a bit painful,
+so we added a syntactic sugar for this: it can be abbreviated with
+`{~author, ~text}` or if all fields are constructed with such abbreviation
+even with: `~{author, text}`. So we could have written just as well:
+
+###### Broadcasting a message to the room (variant)
+
+```
+function void broadcast(string author) {
+ text = Dom.get_value(#entry);
+ message = ~{author, text};
+ Network.broadcast(message, room);
+ Dom.clear_value(#entry);
+}
+```
+
+{block}[TIP]
+### About `void`
+Type `void` is an alias for the empty record, i.e. the record with no fields.
+It is commonly used for functions whose result is uninformative, such as
+functions only producing side-effects or sending messages.
+{block}
+
+This takes care of sending a message to the network. Let us now define
+the symmetric function that should be called whenever the network
+propagates a message:
+
+###### Updating the user interface when a message is received
+
+```
+function user_update(message x) {
+ line = <div>{x.author}: {x.text}</div>;
+ #conversation =+ line;
+}
+```
+
+The role of this function is to display a message just received to the screen.
+This function first produces a few items of user interface, using the same
+HTML-like syntax as above, and calls these items `line`.
+It then uses the special `#ID =+ HTML` construction to append the contents of
+`line` at the end of box with the id `conversation` that we have defined earlier.
+Instead of using the `=+` operator one can also use `+=` to _prepend_ the element
+at the beginning of a given DOM node or simply `=` to _replace_ the node with
+a given content.
+
+If you look more closely at the HTML-like syntax, you may notice that the contents
+inside curly brackets are probably not HTML. Indeed, these curly brackets are
+called _inserts_ and they mark the fact that we are inserting not _text_
+`"x.author"`, but a text representation of the _value_ of `x.author`, i.e. the
+value of field `author` of record `x`.
+
+{block}[TIP]
+### About _inserts_
+Opa provides _inserts_ to insert expressions inside HTML, inside strings and in
+a few other places that we will introduce as we meet them.
+
+This mechanism of inserts is used both to ensure that the correct information
+is displayed and to ensure that this information is sanitized if needs be. It is
+powerful, simple and extensible.
+{block}
+
+We are now ready to connect interactions.
+
+Connecting interactions
+-----------------------
+
+Let us connect `broadcast` to our button and our input. This changes function `start` as follows:
+
+###### Skeleton of the user interface connected to `broadcast` (still incomplete)
+
+```
+function start() {
+ author = Random.string(8);
+ <div id=#conversation />
+ <input id=#entry onnewline={function(_) { broadcast(author) }} />
+ <input type="button" onclick={function(_) { broadcast(author) }} value="Post" />
+}
+```
+
+We have just added _event handlers_ to `entry` and our button. Both call function `broadcast`,
+respectively when the user presses _return_ on the text input and when the user clicks on the
+button. As you can notice, we find again the curly brackets.
+
+{block}[TIP]
+### About _event handlers_
+An _event handler_ is a function whose call is triggered not by the application but by the user
+herself. Typical event handlers
+react to user clicking (the event is called `click`), pressing _return_ (event `newline`),
+moving the mouse (event `mousemove`) or the user loading the page (event `ready`).
+
+Event handlers are always connected to HTML-like user interface elements. An event handler
+always has type `Dom.event -> void`.
+
+You can find more informations about event handlers in [online Opa API documentation](http://api.opalang.org)
+by searching for entry `Dom.event`.
+{block}
+
+We will add one last event handler to our interface, to effectively join the network when
+the user loads the page, as follows:
+
+###### Skeleton of the user interface now connected to everything (final version)
+
+```
+function start() {
+ author = Random.string(8);
+ <div id=#conversation onready={function(_) { Network.add_callback(user_update, room) }} />
+ <input id=#entry onnewline={function(_) { broadcast(author) }} />
+ <input type="button" onclick={function(_) { broadcast(author) }} value="Post" />;
+}
+```
+
+This `onready` event handler is triggered when the page is (fully)
+loaded and connects function `user_update` to our network.
+
+And that is it! The user interface is complete and connected to all
+features. Now, we just need to add the `server` and make things a
+little nicer.
+
+{block}[TIP]
+### About `_`
+Opa has a special value name `_`, pronounced _"I don't care"_. It is
+reserved for values or arguments that you are not going to use, to
+avoid clutter. You will frequently see it in event handlers, as it is
+relatively rare to need details on the event (such as the position of
+the mouse pointer), at least in this book.
+{block}
+
+Bundling, building, launching
+-----------------------------
+
+Every Opa application needs a _server_, to determine what is accessible from the web, so let us define one:
+
+{block}[TIP]
+### About servers
+In Opa, every web application is defined by one or more server. A server is an
+entry point for the web, which offers to users a set of resources, such as web
+pages, stylesheets, images, sounds, etc.
+{block}
+
+###### The server (first version)
+
+```
+Server.start(Server.http, {title: "Chat", page: start})
+```
+
+This extract launches a new HTTP server. For that the special `Server.start` construction is used.
+
+{block}[TIP]
+### About `Server.start`
+`Server.start` is, in a way, an equivalent of a `main` function in C/Java/..., as it's the entry point of the program. However, in Opa instead of just being a function to be executed when a program is started, `Server.start` starts an HTTP server to serve resources to the clients.
+
+We will present some ways of defining servers below. You can also look up the `Server` module in [online Opa API](http://api.opalang.org) and learn more there.
+{block}
+
+This is the type of the `Server.start` function:
+
+`void Server.start(Server.conf configuration, Server.handler handler)`[opa]
+
+ It takes two parameters: configuration and a handler which essentialy defines a service.
+
+Configuration, `Server.conf`, is just a simple record, which defines the port on which the server will run, it's netmask, used encryption and name. Most often you will use `Server.http` or `Server.https`; possibly customizing those values depending on your needs.
+
+In this case the server is constructed using `{title: ..., page: ...}` record which builds a one-page server given a function `page` to generate this page and a `title`.
+
+Well, we officially have a complete application. Time to test it!
+
+We have seen compilation already:
+
+###### Compiling Hello, Chat
+`opa hello_chat.opa`[sh]
+
+Barring any error, Opa will inform you that compilation has succeeded and will produce
+a file called `hello_chat.exe`. This file contains everything you need, so you can now
+launch it, as follows:
+
+###### Running Hello, Chat
+`./hello_chat.exe`[sh]
+
+Congratulations, your application is launched. Let us [visit it](http://localhost:8080).
+
+{block}[TIP]
+### About `hello_chat.exe`
+The opa compiler produces self-sufficient executable applications. The
+application contains everything is requires, including:
+
+- webpages (HTML, CSS, JavaScript);
+- any resource included with `@static_resource_directory`;
+- the embedded web server itself;
+- the distributed database management system;
+- the initial content of the database;
+- security checks added automatically by the compiler;
+- the distributed communication infrastructure;
+- the default configuration for the application;
+- whatever is needed to get the various components to communicate.
+
+In other words, to execute an application, you only need to launch this
+executable, without having to deploy, configure or otherwise administer
+any third-party component.
+{block}
+
+{block}[TIP]
+### About 8080
+By default, Opa applications are launched on port 8080. To launch them on a different port,
+use command-line argument `--port`. For some ports, you will need administrative rights.
+{block}
+
+As you can see, it works, but it is not very nice yet:
+
+![Resulting application, missing style](/resources/manual/img/hello_chat/result_without_css.png)
+
+Perhaps it is time to add some style.
+
+Adding some style
+-----------------
+
+In Opa, all styling is done with stylesheets defined in the CSS
+language. This tutorial is not about CSS, so if you feel rusty, it is
+probably a good idea to keep a
+[good reference] (https://developer.mozilla.org/En/CSS) at hand.
+
+Of course, you will always need some custom CSS, specific to your
+application. Still, you can use some standard CSS to get you started
+with some predefined, nice-looking classes. Opa makes this as easy as
+a single import line:
+
+`import stdlib.themes.bootstrap`[opa]
+
+This automatically brings [Bootstrap CSS from Twitter](http://twitter.github.com/bootstrap/)
+to your application, so you can use their predefined classes that will
+just look nice.
+
+A first step is to rewrite some of our simple HTML stubs to give them
+more structure and add classes. The main user interface becomes
+(omitting the event handlers):
+
+###### Main user interface
+
+```
+<div class="topbar"><div class="fill"><div class="container"><div id=#logo /></div></div></div>
+<div id=#conversation class="container"></div>
+<div id=#footer><div class="container">
+ <input id=#entry class="xlarge"/>
+ <div class="btn primary" >Post</div>
+</div></div>
+```
+
+And the update function becomes:
+
+###### Function to update the user interface when a message is received
+
+```
+function user_update(message x) {
+ line = <div class="row line">
+ <div class="span1 columns userpic" />
+ <div class="span2 columns user">{x.author}:</div>
+ <div class="span13 columns message">{x.text}
+ </div>
+ </div>;
+ #conversation =+ line;
+ Dom.scroll_to_bottom(#conversation);
+}
+```
+
+Note that we have also added a call to `Dom.scroll_to_bottom`, in
+order to scroll to the bottom of the box, to ensure that the user can
+always read the most recent items of the conversation.
+
+For custom style, you have two possibilities. You can either do it
+inside your Opa source file or as an external file. For this example,
+we will use an external file with the following contents:
+
+###### Contents of file `resources/css.css`
+[css]file://hello_chat/resources/css.css
+
+Create a directory called `resources` and save this file as `resources/css.css`. It might be a good idea to add a few
+images to the mix, matching the names given in this stylesheet (`opa-logo.png`, `user.png`) also in directory `resources`.
+
+Now, we will want to extend our `Server.start` construct by instructing it to access the directory and to use our stylesheet. We can achieve by supplying `Server.start` with a record defining our server. More forms of records are allowed, see the definition of `Server.start` in the [online API](http://api.opalang.org).
+
+###### The server (final version)
+
+```
+Server.start(
+ Server.http,
+ [ { resources: @static_resource_directory("resources") }
+ , { register: ["resources/chat.css"]}
+ , { title: "Chat", page: start }
+ ]
+)
+```
+
+In this extract, we have instructed the Opa _compiler_ to:
+* embed the files of directory `resources` and offer them to the browser
+* use a custom resource (CSS) located at `resources/chat.css` and
+* start a one-page application with title _Chat_ and the content of the page generated by the +start+ function.
+
+{block}[TIP]
+### About _embedding_
+In Opa, the preferred way to handle external files is to _embed_ them in the executable.
+This is faster, more secure and easier to deploy than accessing the file system.
+
+To embed a directory, use _directive_ `@static_resource_directory`.
+{block}
+
+{block}[TIP]
+### About _directives_
+In Opa, a _directive_ is an instruction given to the compiler. By opposition to _functions_,
+which are executed once the application is launched,
+directives are executed while the application is being built. Directives always start with character `@`.
+{block}
+
+While we are adding directives, let us take this opportunity to inform the
+compiler that the chatroom definition `room` should be visible and directly accessible for the clients.
+
+`exposed @async room = Network.network(message)`[opa]
+
+This will considerably improve the speed of the chat.
+
+We are done, by the way. Not only is our application is now complete, it also looks nice:
+
+![Final version of the Hello chat application](/resources/manual/img/hello_chat/result.png)
+
+As a summary, let us recapitulate the source file:
+
+###### The complete application
+[opa|fork=hello_chat|run=http://chat.opalang.org]file://hello_chat/hello_chat.opa
+
+All this in 20 effective lines of code (without the CSS). Note that, in this
+final version, we have removed some needless parentheses, which were useful
+mostly for explanations, and documented the code.
+
+Questions
+---------
+
+### Where is the `room`?
+Good question: we have created a network called `room` and we haven't given any location information, so where exactly
+is it? On the server? On some client? In the database?
+
+As `room` is shared between all users, it is, of course, on the server, but the
+best answer is that, generally, you do not need to know. Opa handles such
+concerns as deciding what goes to the server or to the client. We will see in a
+[further chapter](/manual/Developing-for-the-web/Client-server-distribution) exactly how Opa has extracted this information
+from your source code.
+
+### Where are my headers?
+
+If you are accustomed to web applications, you probably wonder about the absence
+of headers, to define for instance the title, favicon, stylesheets or html
+version. In Opa, all these concerns are handled at higher level. You have
+already seen one way of connecting a page to a stylesheet and giving it a
+title. As for deciding which html version to use, Opa handles this
+behind-the-scenes.
+
+### Where is my `return`?
+
+You may be surprised by the lack of an equivalent of the `return` command that would
+allow you to exit function with some return value. Instead in Opa always the
+_last expression_ of the function is its return value.
+
+This is a convention that Opa borrows from functional programming languages
+(as in fact Opa itself is, for the most part, functional!). It may feel limiting at
+first, but don't worry you will quickly get used to that and you may even start
+thinking of a disruption of the functions flow-of-control caused by `return` as
+almost as evil as that of the ill-famed `goto`...
+
+### To `type` or not to `type`?
+
+As mentioned earlier, Opa is designed so that, most of the time, you do not need
+to provide type information manually. However, in some cases, if you do not
+provide type information, the Opa compiler will raise a _value restriction
+error_ and reject the code. Database definitions and value restricted
+definitions are the (only) cases in which you need to provide type information
+for reasons other than optimization, documentation or stronger checks.
+
+For more information on the theoretical definition of a _value restriction
+error_, we invite you to consult the reference chapters of this book. For this
+chapter, it is sufficient to say that value restriction is both a safety and a
+security measure, that alerts you that there is not enough type information on a
+specific value to successfully guard this value against subtle misuses or subtle
+attacks. The Opa compiler detects this possible safety or security hole and
+rejects the code, until you provide more precise type information.
+
+This only ever happens to toplevel values (i.e. values that are defined outside
+of any function), so sometimes, you will need to provide type information for
+such values. Since it is also a good documentation practice, this is not a real
+loss. If you look at the source code of Opa's standard library, you will notice
+that the Opa team strives to always provide such information, although it is
+often not necessary, for the sake of documentation.
+
+Exercises
+---------
+
+Time to see if this tutorial taught you something! Here are a few exercises that will have you expand and customize
+the web chat.
+
+### Customizing the display
+
+Customize the chat so that
+
+- the text box appears on top;
+- each new message is added at the top, rather than at the bottom.
+
+You will need to use operator `+=` instead of `=+` to add at start instead of at end.
+
+### Saying "hello"
+
+- Customize the chat so that, at startup, at the start of `#conversation`, it displays the following message to the current user:
+
+ Hello, you are user 8dh335
+
+(replace `8dh335` by the value of `author`, of course).
+
+- Customize the chat so that, at startup, it displays the following message to all users:
+
+ User 8dh335 has joined the room
+
+- Combine both: customize the chat so that the user sees
+
+ Hello, you are user 8dh335
+
+and other users see
+
+ User 8dh335 has joined the room
+
+{block}[TIP]
+### About comparison
+To compare two values, use operator `==` or, equivalently, function `\`==\`` (with the backquotes).
+When comparing `x == y` (or `\`==\`(x,y)`), `x` and `y` must have the same type. The result of
+a comparison is a boolean. We write that the type of function
+`\`==\`` is `'a,'a -> bool`.
+{block}
+
+{block}[TIP]
+### About _booleans_
+In Opa, booleans are values `{true: void}` and `{false: void}`, or, more
+concisely but equivalently, `{true}` and `{false}`.
+
+Their type declaration looks as follow: `type bool = {true} or {false}`.
+Such types, admitting one of a number of variants, are called sum types.
+{block}
+
+{block}[TIP]
+### About sum types
+A value has a _sum type_ `t or u`, meaning that the values of this type are either
+of the two variants: either a value of type `t` or a value of type `u`.
+
+A good example of sum type are the aforementioned boolean values, which are defined
+as `type bool = {false} or {true}`.
+
+Another good example of sum type is the type `list` of linked lists; its definition
+can be summed up as `{nil} or {... hd, list tl}`.
+
+Note that sum types are not limited to two cases. Sum types with tens of cases
+are not uncommon in real applications.
+{block}
+
+Safely determining which variant was used to construct a value of a sum type
+can be accomplished with pattern matching.
+
+{block}[TIP]
+### About pattern-matching
+The operation used to branch according to the case of a sum type
+is called _pattern-matching_. A good example of pattern-matching
+is `if ... then ... else ...` . The more general syntax for pattern matching is
+```
+match (EXPR) {
+ case CASE_1: EXPR_1
+ case CASE_2: EXPR_2
+ default: EXPR_n
+}
+```
+
+The operation is actually more powerful than just determining which case of a
+sum type is used. Indeed, if we use the vocabulary of languages such as Java or
+C#, pattern-matching combines features of `if`, `switch`, `instanceof`/`is`,
+multiple assignment and dereferenciation, but without the possible safety issues
+of `instanceof`/`is` and with fewer chances of misuse than `switch`.
+
+As an example, you can check whether boolean `b` is true or false by using
+`if b then ... else ...` or, equivalently,
+```
+match (b) {
+ case {true}: ...
+ case {false}: ...
+}
+```
+{block}
+
+### Distinguishing messages between users
+
+Customize the chat so that your messages are distinguished from messages by other users: your messages should be displayed with one icon and everybody else's messages should be displayed with the default icon.
+
+// - Now, expand this beyond two icons. Of course, each user's icon should remain constant during the conversation.
+
+### User customization
+
+- Let users choose their own user name.
+- Let users choose their own icon. You can let them enter a URI, for instance.
+
+{block}[CAUTION]
+### More about `xhtml`
+For security reasons, values with type `xhtml` cannot be transmitted from a client to another one.
+So you will have to find another way of sending one user's icon to all other users.
+{block}
+
+### Security
+
+As mentioned, values with type `xhtml` cannot be transmitted from a client to another one. Why?
+
+### And more
+
+And now, an open exercise: turn this chat in the best chat application available on the web. Oh, and do not forget to show
+your app to the community!
View
302 opadoc/manual.omd/hello_database.omd
@@ -0,0 +1,302 @@
+Hello, database
+===============
+
+In this chapter we will explore data storage possibilities. Opa has its own internal database, which is perfect for prototyping and quickly getting off the ground. However, for complex, data-intense applications, requiring scalable database with data replication and complex querying capabilities it is not enough. That is why starting with ver. 0.9.0 (S4), Opa provides extensive support for the popular [MongoDB database](http://www.mongodb.org) and we will focus on it in this chapter, so beware as some of the presented constructions are not (yet) supported by the internal database of Opa.
+
+MongoDB: Quickstart
+-------------------
+
+In this section we will take the simple counter example from the [Opa tour chapter](/manual/A-tour-of-Opa) and run it using MongoDB.
+
+First, unless done already, you need to [download](http://www.mongodb.org/downloads), [install & run](http://www.mongodb.org/display/DOCS/Quickstart) the MongoDB server. Installation essentially means unpacking the downloaded archive. And you can run the server with a command such as:
+
+```{.sh}
+$ mkdir -p ~/mongodata/opa
+$ {INSTALL_DIR}/mongod --noprealloc --dbpath ~/mongodata/opa > ~/mongodata/opa/log.txt 2>&1
+```
+
+which creates directory `~/mongodata/opa` for the data (1) and then runs MongoDB server using that location and writing logs to `~/mongodata/opa/log.txt` (2). The server will take a moment to start and once done you can verify that it works by running `mongo` (client) in another terminal. The response should look something like this (of course version number may vary):
+
+```
+$ {INSTALL_DIR}/mongo
+MongoDB shell version: 2.0.2
+connecting to: test
+>
+```
+
+Let us recapitulate the example from the previous chapter (we will assume it is saved in a file `counter.opa`).
+
+[opa|fork=hello-opa|run=http://hello-opa.tutorials.opalang.org]file://hello-opa/hello-opa.opa
+
+So how do we upgrade this example to MongoDB? All you have to do is to use the compilation switch for setting the DB backend: `--database mongo` (other accepted value is `db3`, Opa's internal database, which is the default)
+
+`opa --database mongo counter.opa`
+
+run it as before:
+
+`./counter.exe`
+
+and... voilà, you have your first Opa application running on MongoDB!
+
+{block}[WARNING]
+Beware, temporarily Opa accepts database declarations without explicit names, so:
+
+```
+database int /counter = 0;
+```
+
+instead of the above:
+
+```
+database mydb {
+ int /counter = 0;
+}
+```
+
+However, programs with such name-less databases will *not* work with MongoDB. This glitch will be removed in the next version of Opa.
+{block}
+
+The above setup assumes that Mongo is running on local machine on its default port 27017; if that's not the case you can run your application (not the compiler!) with the option `--db-remote:db_name host[:port]`. For instance if Mongo was running locally but on port 4242, we'd need to run the application with:
+
+```{.sh}
+./counter.exe --db-remote:mydb localhost:4242
+```
+
+Incidentally, if you use authentication with your MongoDB databases, you can
+provide the authentication parameters on the command line as follows:
+
+```{.sh}
+./counter.exe --db-remote:mydb myusername:mypassword@localhost:4242
+```
+
+Note, however, that we have a slightly simplified view of MongoDB authentication
+in that we only support one database per connection.
+MongoDB implements authentication on a per-database basis and can have
+authentication to multiple databases over the same connection.
+If you have more than one database at the same server you wil have to issue
+multiple `--db-remote` options, one for each target database.
+A potentially useful feature is that if you authenticate with the `admin`
+database then it acts like ''root'' access for databases and you can then access
+all databases over that connection.
+
+So switching to Mongo is very easy. It is worth noting that although Mongo itself is "Schema free", the Opa compiler ensures adherence to program database definitions, therefore providing a safety layer and ensuring type safety for all database operations.
+
+What are the gains of switching database backend? Apart from running on an industry standard DBMS it opens doors for more complex querying, which we will explore in the following section.
+
+Database declarations
+---------------------
+
+One example is worth a thousand words, so in the remainder of this section we will use a running example of a database for a movie rating service. First we need data for service users, which will just be a set of all users, each of them having her `id` of type `user_id` (here explicit `int`, though in a real-life example it would probably be abstracted away), an `int age` field and a `status` which is a simple variant type (enumeration).
+
+```
+type user_status = {regular} or {premium} or {admin}
+type user_id = int
+type user = { user_id id, string name, int age, user_status status }
+
+database users {
+ user /all[{id}]
+ /all[_]/status = { regular }
+}
+```
+
+First thing to notice is the notation for declaring *database sets*. Writing `user /user` declares a single value of type `user` at path `/user`, however `user /all[{id}]` declares a *set of* values of type `user`, where the record field `id` is the *primary key*.
+
+Second important thing is that all database paths in Opa have associated *defaults values*. For `int` values that is `0`, for `string` values that is an empty string. For custom values we either need to provide it explicitly, which is what `/all[_]/status = { regular }` does, or we need to explicitly state that a given records will never be modified partially, which can be accomplished with the following declaration `/all[_] full` -- more on partial record updates below.
+
+Now we are ready to declare movies. First we introduce a type for a (movie) `person`, which in this simple example just consists of a `name` and the year of birth, `birthyear`.
+
+```
+type person = { string name, int birthyear }
+```
+
+Then we introduce a type for movie cast, including `director` and a list of `stars` playing in the movie (in credits order).
+
+```
+type cast = { person director, list(person) stars }
+```
+
+Now a (single) movie has its `title` (`string`), `view_counter` (`int`; how many times a given movie was looked at in the system), its `cast` and a set of ratings which are mappings from `int` (representing the rating) to `list(user_id)` (representing the list of users who gave that rating). It's not very realistic to just represent one single movie, but for our illustration purposes this will do.
+
+```
+database movie {
+ string /title
+ int /view_counter
+ cast /cast
+ intmap(list(user_id)) /ratings
+}
+```
+
+It is worth noticing that we can store complex (non-primitive) values in the database, without any extra effort -- `cast` is a record, which itself contains a list as one if its fields.
+
+Here we notice that complex types -- records (`cast`), `list`s (`stars` field of `cast`) and `intmap` -- can be declared as entries in the database, in the same way as primitive tyes.
+
+In the rest of this section we will explore how to read, query and update values of different types.
+
+Basic types
+-----------
+
+We can read the title and the view counter of a single movie simply with:
+
+```
+string title = /movie/title;
+int view_no = /movie/view_counter;
+```
+
+The `/movie/title` and `/movie/view_counter` are called *paths* and are composed of the database name (`movie`) and field names (`title`/`view_counter`).
+
+Updates can be performed using the database write operator `path <- value`:
+
+```
+/movie/title <- "The Godfather";
+/movie/view_counter <- 0;
+```
+
+What will happen if one tries to read unitialized value from the database? Say we do:
+
+```
+int view_no = /movie/view_counter;
+```
+
+when `/movie/view_counter` was never initialized; what's the value of `view_no`? We already mentioned default values above and the default value for a given path is exactly what will be given, if it was not written before.
+
+This default value can be overwritten in the path declaration as follows:
+
+```
+database movies {
+ ...
+ int /view_counter = 10 // default value will be 10
+}
+```
+
+Another option is to use question mark (`?`) before the path read, which will return an optional value, `{none}` indicating that the path has not been written to yet, as in:
+
+```
+match (?/movies/view_counter) {
+case {none}: "No views for this movie...";
+case {some: count}: "The movie has been viewed {count} times";
+}
+```
+
+For integer fields we can also use some extra operators:
+
+```
+/movie/view_counter++;
+/movie/view_counter += 5;
+/movie/view_counter -= 10;
+```
+
+Records
+-------
+
+With records we can do complete reads/updates in the same manner as for basic types:
+
+```
+cast complete_cast = /movie/cast;
+/movie/cast <- { director: { name: "Francis Ford Coppola", birthyear: 1939 }
+ , stars: [ { name: "Marlon Brando", birthyear: 1924 }, { name: "Al Pacino", birthyear: 1940 } ]
+ };
+```
+
+However, unless a given path has been declared with for full modifications only (`full` modificator explained above), we can also cross record boundaries and access/update only chosen fields, by including them in the path:
+
+```
+person director = /movie/cast/director;
+/movie/cast/director/name <- "Francis Ford Coppola"
+```
+
+Also with updates we can only update some of the fields:
+
+```
+ // Notice the stars field below, which is not given and hence will not be updated
+/movie/cast <- { director: { name: "Francis Ford Coppola", birthyear: 1939 } }
+```
+
+// TODO Explain defaults and need to define them for custom variant types
+
+Lists
+-----
+
+List in Opa are just (recursive) [records](/type/stdlib.core/list) and can be manipulated as such:
+
+```
+/movie/cast/stars <- []
+person first_billed = /movie/cast/stars/hd
+list(person) non_first_billed = /movie/cast/stars/tl
+```
+
+However, as it's a frequently used data-type, Opa provides a numer of extra operations on them:
+
+```
+ // removes first element of the list
+/movie/cast/stars pop
+ // removes last element of the list
+/movie/cast/stars shift
+ // appends one element to the list
+/movie/cast/stars <+ { name: "James Caan", birthyear: 1940 }
+
+person actor1 = ...
+person actor2 = ...
+ // appends several elements to the list
+/movie/cast/stars <++ [ actor1, actor2 ]
+```
+
+Sets and Maps
+-------------
+
+Sets and maps are two types of collections that allow to organize multiple instances of data in the database. Sets represent a number of items of a given type, with no order between the items (as opposed to lists, which are ordered). Maps represent associations from keys to values.
+
+We can fetch a single value from a given set by referencing it by its primary key:
+
+```
+user some_user = /users/all[{id: 123}]
+```
+
+Similarly for maps:
+
+```
+list(user_id) gave_one = /movie/ratings[1]
+```
+
+We can also make various queries using the following operators (*comma* separated).
+
+- `field == expr`: value of `field` is *equal* to the expression `expr`.
+- `field != expr`: value `field` is *not equal* to the expression `expr`.
+- `field < expr`: value of `field` is *smaller than* that of `expr` (you can also use: `<=`, `>` and `>=` with ).
+- `field in list_expr`: value of `field` is *one of* those of the list `list_expr`.
+- `q1 or q2`: satisfies query `q1` *or* `q2`.
+- `q1 and q2`: satisfies queries `q1` *and* `q2`.
+- `not q`: does not satisfy query `q`.
+- `{f1 q1, f2 q2, ...}`: field `f1` satisfies `q1`, field `f2` satisfies `q2` etc.
+
+and possibly some options (*semicolon* separated)
+
+- `skip n`: skip the first `n` results (`n` an expression of type `int`)
+- `limit n`: limit the result to the maximum of `n` results (`n` an expression of type `int`)
+- `order fld (, fld)+`: order the results by given fields. Possible variants include: `fld`, `+fld` (sort ascending), `-fld` (sort descending), `fld=expr` (where `expr` needs to be of type `{up} or {down}`).
+
+Note that if the result of a query is not fully constained by the primary key and hence can contain multiple values then the result of the query is of type [dbset](/module/stdlib.database.mongo/DbSet) (for sets) or of the initial map type (for maps, giving a sub-map). You can manipulate a dbset with `DbSet.iterator` and [Iter](/module/stdlib.core.iter/Iter).
+
+Examples for sets:
+
+```
+user /users/all[id == 123] // accessing a single entry by primary key
+dbset(user, _) john_does = /users/all[name == "John Doe"] // return a set of values
+it = DbSet.iterator(john_does) // then use Iter module
+
+dbset(user, _) underages = /users/all[age < 18]
+dbset(user, _) non_admins = /users/all[status in [{regular}, {premium}]]
+dbset(user, _) /users/all[age >= 18 and status == {admin}]
+dbset(user, _) /users/all[not status == {admin}]
+ // showing second 50 results for users that are below 18 or above 62,
+ // sorted by age (ascending) and then id (descending)
+dbset(user, _) users1 = /users/all[age <= 18 or age >= 62; skip 50; limit 50; order +age, -id]
+```
+
+Examples for maps:
+
+```
+ // users who rated the movie 10
+list(user_id) loved_it = /movie/ratings[10]
+ // users who rated the movie between 7 and 9 (inclusive)
+list(user_id) liked_it = /movie/ratings[>= 7 and <= 9]
+```
View
235 opadoc/manual.omd/hello_distribution.omd
@@ -0,0 +1,235 @@
+Hello, scalability
+==================
+
+// [WARNING]
+// .You may need to upgrade Opa
+// =============
+// Examples in this chapter require a version of Opa dated from May the 25th, 2011, or later
+// i.e. build 28532 or greater.
+// =============
+
+//
+// About this chapter:
+// Main author: ?
+// Paired author:?
+//
+// Topics:
+// - shared networks, shared sessions
+// - executing with distributed sessions
+// - distributed database
+// - executed with distributed database
+// - deploying a load-balancer
+// - creating a web service
+// - accessing a web service
+// - deploying on EC2
+//
+
+From the ground up, Opa was designed for scalability. This means that any
+application written in Opa can (almost) automatically take advantage of
+additional cores or additional servers to distribute treatment, storage
+or delivery. In this chapter, we will see how to adapt, deploy and
+execute our chat and our wiki in a distributed, load-balanced setting.
+As you will see, it is very, very simple.
+
+Prerequisites
+-------------
+
+You are about to distribute instances of your server on other computers.
+To do so, you will need a valid account and an ssh connection.
+Firstly, make sure that you can connect through ssh to computers you want instances to run, without prompting for a password.
+Secondly, check whether these computers have *base64* installed.
+Thirdly, check whether *HAProxy* is installed on your localhost.
+
+If you don't know how to do any of these steps, don't panic and have a look at the [FAQ](/manual/Hello--scalability/Frequently-Asked-Questions)
+
+It's now time to distribute !
+
+Distributing Hello, chat
+------------------------
+
+Done! We have a distributed Hello, chat.
+
+Don't be confused, this is no mistake. All the versions of the chat we have been implementing
+so far are distributed. If you want to try any of them, you just need to launch
+it in distributed mode, with the following command-line:
+
+ opa-cloud --host localhost,2 hello_chat.exe
+
+You can also explicit each host you want an instance to run on e.g. twice on 'localhost':
+
+ opa-cloud --host localhost --host localhost hello_chat.exe
+
+Both lines are equivalent.
+
+You can now connect to http://localhost:8080/ and users connecting to the service will be automatically distributed between
+two processes running on your computer. They can, of course, chat together,
+regardless of the server to which they are effectively connected. You are of
+course not limited to two processes or to a single computer: if you have other
+computers at hand (say, a cloud), with the same binary configuration, you can
+add distant hosts, too:
+
+ opa-cloud --host localhost,2 --host jeff@albertson hello_chat.exe
+
+Not bad for, well, zero lines of code to add or modify!
+
+Distributing Hello, wiki
+------------------------
+
+We are not going to crack the same joke twice, but we could, though.
+Again, don't modify what you have already written for Hello, wiki and merely invoke the following command-line:
+
+ opa-cloud --host localhost,2 hello_wiki.exe
+
+Users are dispatched to servers so as to balance the load between these
+servers. Of course, they share the same database. And, if you have modified the
+wiki so as to show updates as they take place, this feature will keep working
+in a distributed mode.
+
+Examples
+--------
+
+The following command line distributes 6 instances of Hello, chat on `albertson`, with user `jeff`. Each instance will be listening to a port between 7000 et 7005 included.
+
+ opa-cloud --host jeff@albertson:7000,6 hello_chat.exe
+
+The following command line distributes 10 instances of Hello, wiki on an Amazon EC2 instance with key `jeff.pem`. Each instance will listen to a port between 9090 and 9099. The HAProxy binary will be `~/bin/haproxy`.
+
+ opa-cloud hello_wiki.exe --host-pem ubuntu@ec-XXX.amazonaws.com:9090,10 --pem jeff.pem --haproxy ~/bin/haproxy
+
+Frequently Asked Questions
+--------------------------
+
+### How can Hello, chat work be distributed?
+
+We have spent years making sure that it appears as magic, you don't want to ruin the magic now, do you?
+
+For a few more details, we need to start with the network defined in Hello, Chat:
+
+ Network.network(message) room = Network.cloud("room")
+
+Function `Network.cloud` not only constructs a network, but also declares it to
+Opa's built-in distribution directory. In this directory, the network is called
+`"room"` (we could, of course, have given it any other name -- some developers
+prefer writing `@pos`, which gives as name the position in the source
+code). Subsequent calls to `Network.cloud` by _any of the servers you just
+launched_ will return the _same_ network. Consequently, whenever a client calls
+`Network.add_callback` with `room`, the callback is known to all servers.
+
+We will detail this further in the reference chapters.
+
+{block}[TIP]
+### Non-cloud networks
+You can of course declare networks that you do not want to publish in the
+distribution directory. For this purpose, use function `Network.empty`
+rather than `Network.cloud`.
+{block}
+
+### How does Hello, wiki work?
+
+In a few words: it works because Opa's built-in database engine is fully distributed and compatible with `opa-cloud`.
+
+### How can check my ssh connection?
+
+Let say your remote computer is called 'albertson' and your user name is 'jeff'.
+You should be able to connect to albertson with the following command line:
+
+ $ ssh jeff@albertson
+ jeff@albertson's password:
+ jeff@albertson ~ $
+
+Let's get rid of the password prompt by adding your public key to the list of authorized keys. To do so, copy the content of ~/.ssh/id_dsa.pub to ~/.ssh/authorized_keys on 'albertson'.
+
+You can now log off 'albertson':
+
+ jeff@albertson ~ $ exit
+ logout
+ Connection to albertson closed.
+ $
+
+### How can I get rid of the prompt for password?
+
+You must append your public key to the list of authorized keys of the remote host i.e. usually `~user/.ssh/authorized_keys`.
+
+### How can I make sure that my remote computer *albertson* provide *base64* ?
+
+To check whether 'albertson' has this tool, simply type:
+
+ jeff@albertson ~ $ base64 --version
+ base64 1.5
+ Last revised: 10th June 2007
+ The latest version is always available
+ at http://www.fourmilab.ch/webtools/base64
+
+Don't worry if you don't have the same version, the important thing is not to get anything like the following:
+
+ jeff@albertson ~ $ base64 --version
+ -bash: base64: command not found
+
+But if you do, please see the chapter [Getting Opa](/manual/Getting-Opa).
+
+### How can I use this on a machine with a specific public-key?
+
+You can ask `opa-cloud` to connect to servers with a specific public key with options '--pem' and '--host-pem'. The former specifies the key and the latter specifies a host which needs this key. Although you can define only one key, you can specify several hosts:
+
+ opa-cloud --pem ~/.ssh/mykey.pem --host localhost --host-pem user@sver hello_chat.exe
+
+This command line will start two instances of hello_chat.exe, one on your localhost, and one on `sver` connecting to it with `mykey.pem`.
+
+### How can I use this with Amazon Web Services?
+
+Opa's built-in distribution works very nicely with Amazon EC2 -- and just as well with other public or private clouds.
+Amazon allows you to create a pair of key. You will use the public key to connect to your EC2 instance.
+
+* start an instance and make sure to open ports 22, 8081 and 1086 (see the group security option in your AWS Management console)
+* retrieve the url which should look like: ec2-XXX.amazonaws.com
+* use `opa-cloud` to distribute your service on this instance:
+
+ opa-cloud --pem mykey.pem --host-pem ubuntu@ec2-XXX.amazonaws.com hello_chat.exe
+
+You can of course distribute your service on more then one instance, and not only AmazonEC2. The following command line distributes between an instance on your localhost, two instances on server sv1 and 2 on an AmazonEC2 instance with key mykey.pem.
+
+ opa-cloud hello_chat.exe --host-pem ubuntu@ec2-XXX.amazonaws.com,2 --pem mykey.pem --host localhost --host sv1,2
+
+You can, of course, use Amazon's load-balancer instead of Opa's load balancer.
+
+{block}[WARNING]
+Check that each instance can reach every other one e.g. launching EC2 instances from a computer in a local network may raise issues because the EC2 instances won't be able to reach hosts in the sub-network.
+{block}
+
+### My application does not use any database, do I have to get it started anyway?
+
+For some reason, you may not want to start the Opa database server e.g. your service may not need one. You can specify it in the command line with the '--no-db' option:
+
+ opa-cloud --host localhost,3 myapp.exe --nodb
+
+### Can I specify different public keys for different hosts?
+
+No. At the time, `opa-cloud` does not offer this flexibility. Although we are working to implement it because we believe it would be a great feature.
+