Skip to content

Commit

Permalink
Add QuickJS as a Javascript engine option
Browse files Browse the repository at this point in the history
https://bellard.org/quickjs
https://fuchsia.googlesource.com/third_party/quickjs/

Some benefits over SM:

 * Small. We're using 6 or so C files vs 700+ SM91 C++ files.

 * Built with Apache CouchDB as opposed having to maintain a separate SM
   package, like for RHEL9, for instance, where they dropped support for SM
   already. (see apache#4154).

 * Embedding friendly. Designed from ground-up for embedding. SM has been
   updating the C++ API such that we have to keep copy-pasting new versions of
   our C++ code every year or so. (see
   apache#4305).

 * Easy to modify to accept Spidermonkey 1.8.5 top level functions for
   map/reduce code so we don't have have to parse the JS, AST trasform it, and
   then re-compile it.

 * Configurable runtime feature set - can disable workers, promises and other
   API and features which may not work well in a backend JS environment. Some
   users may want more, some may want to disable even Date(time) features to
   hedge again Spectre-style attacks (spectreattack.com).

 * Allow granular time (reduction) trackign if we wanted to provide a runtime
   allowance for each function.

 * Better sandboxing. Creating a whole JSRuntime takes only 300 microsecond, so
   we can afford to do that on reset. JSRuntimes cannot share JS data or object
   between them.

 * Seems to be faster in preliminary benchmarking with small
   concurrent VDU and view builds:
     https://gist.github.com/nickva/ed239651114794ebb138b1f16c5f6758
   Results seem promising:
     - 4x faster than SM 1.8.5
     - 5x faster than SM 91
     - 6x reduced memory usage per couchjs process (5MB vs 30MB)

 * Allows compiling JS bytecode ahead of time a C array of bytes.

QuickJS can be built alongside spidermonkey and toggled on/off at runtime:

```
./configure --dev --js-engine=quickjs
```

This makes it the default engine. But spidermonkey can still be set in the
config option.

```
[couchdb]
js_engine = spidermonkey | quickjs
```

Only tested on MacOS and Linux. All `make check` tests pass there.
  • Loading branch information
nickva committed Mar 31, 2023
1 parent d141255 commit aab990c
Show file tree
Hide file tree
Showing 53 changed files with 82,042 additions and 11 deletions.
24 changes: 24 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -2267,3 +2267,27 @@ ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

For the QuickJS component couch_js/quickjs/quickjs:

QuickJS Javascript Engine

Copyright (c) 2017-2021 Fabrice Bellard
Copyright (c) 2017-2021 Charlie Gordon

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
25 changes: 24 additions & 1 deletion configure
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ run_erlang() {
}

COUCHDB_USER="$(whoami 2>/dev/null || echo couchdb)"
JS_ENGINE=${JS_ENGINE:-"spidermonkey"}
SM_VSN=${SM_VSN:-"91"}
ARCH="$(uname -m)"
ERLANG_VER="$(run_erlang 'io:put_chars(erlang:system_info(otp_release)).')"
Expand Down Expand Up @@ -64,6 +65,7 @@ Options:
--generate-tls-dev-cert generate a cert for TLS distribution (To enable TLS, change the vm.args file.)
--rebar3=PATH use rebar3 by specified path
--erlfmt=PATH use erlfmt by specified path
--js-engine=ENGINE default js engine: spidermonkey or quickjs
EOF
}

Expand Down Expand Up @@ -205,6 +207,24 @@ parse_opts() {
exit 1
;;

--js-engine)
if [ -n "$2" ]; then
eval JS_ENGINE=$2
shift 2
continue
else
printf 'ERROR: "--js-engine" requires a non-empty argument.\n' >&2
exit 1
fi
;;
--js-engine=?*)
eval JS_ENGINE=${1#*=}
;;
--js-engine=)
printf 'ERROR: "--js-engine" requires a non-empty argument.\n' >&2
exit 1
;;

--generate-tls-dev-cert)
echo "WARNING: To enable TLS distribution, don't forget to customize vm.args file."
generate_tls_dev_cert
Expand Down Expand Up @@ -234,7 +254,7 @@ if [ "${ARCH}" = "aarch64" ] && [ "${SM_VSN}" = "60" ]; then
exit 1
fi

if [ "${ERLANG_OS}" = "unix" ]; then
if [ "${JS_ENGINE}" = "spidermonkey" ] && [ "${ERLANG_OS}" = "unix" ]; then
case "${SM_VSN}" in
1.8.5)
SM_HEADERS="js"
Expand Down Expand Up @@ -276,6 +296,7 @@ cat > rel/couchdb.config << EOF
{log_file, "$LOG_FILE"}.
{fauxton_root, "./share/www"}.
{user, "$COUCHDB_USER"}.
{js_engine, "$JS_ENGINE"}.
{spidermonkey_version, "$SM_VSN"}.
{node_name, "-name couchdb@127.0.0.1"}.
{cluster_port, 5984}.
Expand Down Expand Up @@ -304,12 +325,14 @@ with_fauxton = $WITH_FAUXTON
with_docs = $WITH_DOCS
user = $COUCHDB_USER
js_engine = $JS_ENGINE
spidermonkey_version = $SM_VSN
EOF

cat > $rootdir/config.erl << EOF
{with_proper, $WITH_PROPER}.
{erlang_md5, $ERLANG_MD5}.
{js_engine, "$JS_ENGINE"}.
{spidermonkey_version, "$SM_VSN"}.
EOF

Expand Down
1 change: 1 addition & 0 deletions rebar.config.script
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ SubDirs = [
"src/khash",
"src/b64url",
"src/ets_lru",
"src/couch_quickjs",
"src/chttpd",
"src/couch",
"src/couch_event",
Expand Down
8 changes: 8 additions & 0 deletions rel/overlay/etc/default.ini
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,9 @@ view_index_dir = {{view_index_dir}}
; Sets the log level for informational compaction related entries.
;compaction_log_level = info

; Javascript engine. The choices are: spidermonkey and quickjs
;js_engine = spidermonkey

[purge]
; Allowed maximum number of documents in one purge request
;max_document_id_number = 100
Expand Down Expand Up @@ -870,3 +873,8 @@ port = {{prometheus_port}}
; `false`, the expected n value is based on the number of available copies in
; the shard map.
;use_cluster_n_as_expected_n = false

[quickjs]
; Memory limit in bytes. Default is undefined and so the built-in C default
; whic is currently it's 64MB
;memory_limit_bytes = 67108864
2 changes: 2 additions & 0 deletions rel/reltool.config
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
%% couchdb
b64url,
bear,
couch_quickjs,
chttpd,
config,
couch,
Expand Down Expand Up @@ -91,6 +92,7 @@
%% couchdb
{app, b64url, [{incl_cond, include}]},
{app, bear, [{incl_cond, include}]},
{app, couch_quickjs, [{incl_cond, include}]},
{app, chttpd, [{incl_cond, include}]},
{app, config, [{incl_cond, include}]},
{app, couch, [{incl_cond, include}]},
Expand Down
182 changes: 182 additions & 0 deletions share/server/dispatch-quickjs.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,182 @@
// Licensed under the Apache License, Version 2.0 (the "License"); you may not
// use this file except in compliance with the License. You may obtain a copy of
// the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
// WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
// License for the specific language governing permissions and limitations under
// the License.

function create_sandbox() {
var sandbox = {};
sandbox.emit = Views.emit;
sandbox.sum = Views.sum;
sandbox.log = log;
sandbox.toJSON = JSON.stringify;
sandbox.JSON = JSON;
sandbox.provides = Mime.provides;
sandbox.registerType = Mime.registerType;
sandbox.start = Render.start;
sandbox.send = Render.send;
sandbox.getRow = Render.getRow;
sandbox.isArray = isArray;
sandbox.index = Dreyfus.index;
return sandbox;
};

function create_filter_sandbox() {
var sandbox = create_sandbox();
sandbox.emit = Filter.emit;
return sandbox;
};

function seal(obj, flag) {
Object.freeze(obj);
};

// This is a copy from loop.js
var DDoc = (function() {
var ddoc_dispatch = {
"lists" : Render.list,
"shows" : Render.show,
"filters" : Filter.filter,
"views" : Filter.filter_view,
"updates" : Render.update,
"validate_doc_update" : Validate.validate,
"rewrites" : Render.rewrite
};
var ddocs = {};
return {
ddoc : function() {
var args = [];
for (var i=0; i < arguments.length; i++) {
args.push(arguments[i]);
};
var ddocId = args.shift();
if (ddocId == "new") {
// get the real ddocId.
ddocId = args.shift();
// store the ddoc, functions are lazily compiled.
ddocs[ddocId] = args.shift();
print("true");
} else {
// Couch makes sure we know this ddoc already.
var ddoc = ddocs[ddocId];
if (!ddoc) throw(["fatal", "query_protocol_error", "uncached design doc: "+ddocId]);
var funPath = args.shift();
var cmd = funPath[0];
// the first member of the fun path determines the type of operation
var funArgs = args.shift();
if (ddoc_dispatch[cmd]) {
// get the function, call the command with it
var point = ddoc;
for (var i=0; i < funPath.length; i++) {
if (i+1 == funPath.length) {
var fun = point[funPath[i]];
if (!fun) {
throw(["error","not_found",
"missing " + funPath[0] + " function " + funPath[i] +
" on design doc " + ddocId]);
}
if (typeof fun != "function") {
// For filter_view we want the emit() function
// to be overridden and just toggle a flag instead of
// accumulating rows
var sandbox = (cmd === "views") ? create_filter_sandbox() : create_sandbox();
fun = Couch.compileFunction(fun, ddoc, funPath.join('.'), sandbox);
// cache the compiled fun on the ddoc
point[funPath[i]] = fun;
};
} else {
point = point[funPath[i]];
}
};

// run the correct responder with the cmd body
ddoc_dispatch[cmd].apply(null, [fun, ddoc, funArgs]);
} else {
// unknown command, quit and hope the restarted version is better
throw(["fatal", "unknown_command", "unknown ddoc command '" + cmd + "'"]);
}
}
}
};
})();

// This mostly a copy from loop.js handleError
function handleError(e) {
if (e === null) {
// internal error, another possibility when out of memory
// nothing to do except rethrow and let main.c catch it and exit(1)
throw(null);
}
const type = e[0];
if (type == "fatal") {
e[0] = "error"; // we tell the client it was a fatal error by dying
respond(e);
return false;
} else if (type == "error") {
respond(e);
return true;
} else if (e.name == "InternalError") {
// If the internal error is caught by handleViewError it will be
// re-thrown as a ["fatal", ...] error, and we already handle that above.
// Here we handle the case when the error is thrown outside of
// handleViewError, for instance when serializing the rows to be sent
// back to the user
respond(["error", e.name, e.message]);
return false;
} else if (e.error && e.reason) {
// compatibility with old error format
respond(["error", e.error, e.reason]);
return true;
} else if (e.name) {
respond(["error", e.name, e]);
return true;
} else {
respond(["error","unnamed_error", e.stack]);
return true;
}
};

globalThis.dispatch = function(line) {
const cmd = JSON.parse(line);
State.line_length = line.length;
try {
switch (cmd.shift()) {
case "ddoc":
DDoc.ddoc.apply(null, cmd);
break;
case "reset":
State.reset.apply(null, cmd);
break;
case "add_fun":
State.addFun.apply(null, cmd);
break;
case "add_lib":
State.addLib.apply(null, cmd);
break;
case "map_doc":
Views.mapDoc.apply(null, cmd);
break;
case "index_doc":
Dreyfus.indexDoc.apply(null, cmd);
break;
case "reduce":
Views.reduce.apply(null, cmd);
break;
case "rereduce":
Views.rereduce.apply(null, cmd);
break;
default:
// unknown command, quit and hope the restarted version is better
throw(["fatal", "unknown_command", "unknown command '" + cmdkey + "'"]);
}
} catch(e) {
return handleError(e);
};
return true;
};
2 changes: 1 addition & 1 deletion share/server/dreyfus.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ var Dreyfus = (function() {
} else if (err[0] == "fatal") {
throw(err);
}
var message = "function raised exception " + err.toSource();
var message = "function raised exception " + errstr(err);
if (doc) message += " with doc._id " + doc._id;
log(message);
};
Expand Down
2 changes: 1 addition & 1 deletion share/server/render.js
Original file line number Diff line number Diff line change
Expand Up @@ -347,7 +347,7 @@ var Render = (function() {
throw(e);
} else {
var logMessage = "function raised error: " +
e.toSource() + " \n" +
errstr(e) + " \n" +
"stacktrace: " + e.stack;
log(logMessage);
throw(["error", errType || "render_error", logMessage]);
Expand Down
11 changes: 8 additions & 3 deletions share/server/util.js
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ var Couch = {
throw [
"error",
"compilation_error",
"Module require('" +name+ "') raised error " + e.toSource()
"Module require('" +name+ "') raised error " + errstr(e)
];
}
ddoc._module_cache[newModule.id] = newModule.exports;
Expand All @@ -106,7 +106,7 @@ var Couch = {
throw([
"error",
"compilation_error",
err.toSource() + " (" + source + ")"
errstr(err) + " (" + source + ")"
]);
};
if (typeof(functionObject) == "function") {
Expand All @@ -126,13 +126,18 @@ var Couch = {
}
};

function errstr(e) {
// toSource() is a Spidermonkey "special"
return (e.toSource ? e.toSource() : e.toString());
};

// prints the object as JSON, and rescues and logs any JSON.stringify() related errors
function respond(obj) {
try {
print(JSON.stringify(obj));
} catch(e) {
log("Error converting object to JSON: " + e.toString());
log("error on obj: "+ obj.toSource());
log("error on obj: "+ obj.toString());
}
};

Expand Down
2 changes: 1 addition & 1 deletion share/server/views.js
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ var Views = (function() {
} else if (err.name == "InternalError") {
throw(["fatal", err.name, err.message]);
}
var message = "function raised exception " + err.toSource();
var message = "function raised exception " + errstr(err);
if (doc) message += " with doc._id " + doc._id;
log(message);
};
Expand Down
Loading

0 comments on commit aab990c

Please sign in to comment.