Refactor SOTU dataset package

FAQ.md

@@ -4,19 +4,22 @@
 
 
 * [Why numeric computing in JavaScript?](#numeric-computing-in-javascript)
+* [What about WebAssembly?](#web-assembly)
 * [Why reimplement and provide custom Math implementations?](#custom-math-implementations)
 * [Why reimplement module functionality already available on npm?](#reimplementing-existing-packages)
+* [Backward compatibility?](#backward-compatibility)
 * [Promise support?](#promise-support)
+* [ES2015 and beyond?](#es2015)
 * [Why a monorepo?](#monorepo)
 * [How can I contribute?](#contributing)
 
 
----
-
 <!-- <faq-questions> -->
 
 <!-- <faq-question> -->
 
+---
+
 <a name="numeric-computing-in-javascript"></a>
 
 ### Why numeric computing in JavaScript?
@@ -33,6 +36,27 @@
 
 <!-- <faq-question> -->
 
+---
+
+<a name="web-assembly"></a>
+
+### What about WebAssembly?
+
+[WebAssembly][wasm] proposes to fundamentally change the web platform by providing a low-level compilation target, which, in the future, will allow any language to be compiled to run on the Web. Once [WebAssembly][wasm] is ubiquitously supported, web developers will be able to use, e.g., a numeric computation library from, not just JavaScript, but any language, including traditional languages, such as R, Python, Julia, and C/C++ (provided the library can compile). While the possibility of using libraries like [NumPy][numpy] and [scikit-learn][scikit-learn] on the Web is exciting, this is not viewed as an existential threat for the following reasons:
+
+1. __Compilation__: as developers who have used [asm.js][asm] to compile C and C++ libraries to run on the Web can attest, the process is not as simple as defining input and output targets. Often pre-compiled code has to be massaged into a form suitable for compilation, which means work is involved, requiring both time and labor.
+1. __Timescale__: [WebAssembly][wasm] is not likely to be ubiquitous anytime soon (as of 2016) and a need exists now for numeric computation libraries which work on the Web.
+1. __Monoglot__: developers will still build JavaScript applications and most will, all things being equal, want to use a library written in the same idiom. Using a single language stack reduces cognitive overhead and simplifies application development.
+1. __Legacy__: [WebAssembly][wasm] is unlikely to replace JavaScript, but, instead, serve a complementary role. JavaScript has a long and decorated history as part of the web platform. Relegating JavaScript to the dust bin would entail breaking the Web, an outcome which has been and will continue to be untenable, thus securing JavaScript's privileged status. 
+
+
+<!-- </faq-question> -->
+
+
+<!-- <faq-question> -->
+
+---
+
 <a name="custom-math-implementations"></a>
 
 ### Why reimplement and provide custom Math implementations?
@@ -48,6 +72,8 @@
 
 <!-- <faq-question> -->
 
+---
+
 <a name="reimplementing-existing-packages"></a>
 
 ### Why reimplement module functionality already available on npm?
@@ -61,6 +87,32 @@
 
 <!-- <faq-question> -->
 
+---
+
+<a name="backward-compatibility"></a>
+
+### Backward compatibility?
+
+#### Interfaces
+
+From time to time, interfaces may change in incompatible and breaking ways. Software evolves and such changes are part of the normal course of development. In compliance with [semantic versioning][semver], these changes will result in [major version][semver] changes. As long as your [package manager][npm] supports [semantic versioning][semver], libraries and applications which consume this project, or elements of this project, should not break and may independently upgrade whenever is most convenient.
+
+
+#### Engines
+
+This project has every intent on maintaining backward compatibility with older Node.js engines, including those which have reached their end-of-life (EOL) and including those which are pre-ES2015 beginning with Node.js __v0.10.x__. Accordingly, interface changes and new features should __never__ break this compatibility. The reasons for maintaining compatibility are as follows:
+
+1. With regard to the Node.js [long-term release schedule][node-lts], simply because a Node.js version has reached its end-of-life (EOL), this does not mean that a) the Node.js version is no longer used or b) library authors ought to stop supporting that version. As long as libraries use the simplest, lowest level abstraction, the question as to whether a library should support a legacy Node.js version should never arise. The only time where dropping legacy support may be justified is when supporting native [add-ons][node-addons], as maintenance costs can be significantly higher.
+1. Functionality should not only enable the future, but also allow probing the past. In an ideal world, everyone would use the latest and greatest engine; however, in the real world, not everyone can. Legacy systems abound for very valid and practical reasons; that they will continue to exist is not going to change. To achieve the greatest possible reach, functionality should account for these environments. The best approach for doing so is to use the simplest possible primitives which are most likely to be supported across the widest range of environments.
+1. Consumers should have control over their migration schedules. In general, library developers are far too quick to drop support for legacy environments, citing maintenance costs, often as a thinly veiled desire to force consumers to upgrade. This parental and cavalier attitude fails to acknowledge the practical realities that many consumers face when building real-world applications. Once real-world applications are deployed in production environments, they assume lives of their own, becoming critical zero downtime components without concern for a library author's desire for evolution. All too frequently, a developer's desire for modernity (and trendiness) creates needless downstream effects, especially in those instances where the cost of maintenance is effectively zero.
+
+<!-- </faq-question> -->
+
+
+<!-- <faq-question> -->
+
+---
+
 <a name="promise-support"></a>
 
 ### Promise support?
@@ -72,6 +124,33 @@ No.
 
 <!-- <faq-question> -->
 
+---
+
+<a name="es2015"></a>
+
+### ES2015 and beyond?
+
+Only __if__ three conditions are met:
+
+1. ES2015+ features can be used __without__ breaking backward compatibility.
+1. ES2015+ features provide something which is absolutely needed but literally not possible in ES5 and earlier environments.
+1. ES2015+ features can be polyfilled in older environments __without__ transpilation.
+
+The reasons are as follows:
+
+* __Abstraction__: in general, the lower the abstraction, the less magic. Less magic means increased comprehensibility, a smaller surface area, and more control over performance and optimization. Many ES2015+ features are higher-order abstractions for things already possible in ES5. The preference of this project is to eschew higher-order abstractions for the simplest primitives and the greatest clarity.
+* __Control__: unless a transpiler is developed in-house, transpilation requires third party tooling. Transpilers range from the good to the bad, with many generating unoptimized transpiled code. (And why would they? They are designed to be general tools.) Accordingly, efficiency and performance would reside outside the control of this project, which is not an acceptable cost.
+* __Backward Compatibility__: the ability to probe the past is equally as valuable as the ability to build for the future.
+* __Transparency__: source code matches distributed code. This one-to-one correspondence means a) easier debugging without maintenance overhead (e.g., source-maps) and b) individuals reading the source code can better form expectations as to how that code will execute in a deployed environment.
+* __Simplicity__: any additional interface, syntax, or build step adds complexity.
+
+<!-- </faq-question> -->
+
+
+<!-- <faq-question> -->
+
+---
+
 <a name="monorepo"></a>
 
 ### Why a monorepo?
@@ -88,6 +167,8 @@ No.
 
 <!-- <faq-question> -->
 
+---
+
 <a name="contributing"></a>
 
 ### How can I contribute?
@@ -128,6 +209,15 @@ See the [contributing guide][contributing-guide].
 [native-math-bugs]: https://github.com/stdlib-js/stdlib/blob/master/docs/native_math_bugs.md
 [contributing-guide]: https://github.com/stdlib-js/stdlib/blob/master/CONTRIBUTING.md
 
+[wasm]: https://github.com/WebAssembly/spec/
+[asm]: http://asmjs.org/spec/latest/
+[numpy]: http://www.numpy.org/
+[scikit-learn]: http://scikit-learn.org/stable/
+
+[semver]: http://semver.org/
+[node-lts]: https://github.com/nodejs/LTS
+[node-addons]: https://nodejs.org/api/addons.html
+
 [ecma-262]: http://www.ecma-international.org/publications/standards/Ecma-262.htm
 
 <!-- </links> -->

TODO.md

@@ -892,7 +892,7 @@ TODO
 
 115. should `float64-signbit` return a `boolean` (like Julia) or a `1` or `0`?
 
-116. `make` recipe for checking dep licenses
+116. licenses utilities
 
      - [licensee.js](https://github.com/jslicense/licensee.js)
      - [license-checker](https://github.com/davglass/license-checker)
@@ -1079,9 +1079,7 @@ TODO
 
 186. [GNU parallel](https://www.gnu.org/software/parallel/man.html) for distributed tasks
 
-187. FAQ
-
-     - What about web assembly?
+187. 
 
 188. investigate [vm2](https://github.com/patriksimek/vm2) for sandboxing
 
@@ -1124,6 +1122,10 @@ TODO
 
 206. Consider something likes [credits](https://github.com/stefanjudis/credits-cli)
 
+207. NLP [data](https://confluence.cornell.edu/display/NLP/Data/) (note: ambiguous licensing)
+
+208. Fix SOTU raw text where `\&mash;` was converted to a single hyphen `-`
+
 
 ---