PluotSorbet is a J2ME-compatible virtual machine written in JavaScript.
Java TypeScript JavaScript C++ Makefile CSS Other
Latest commit 46ee3e7 Jan 6, 2017 @brendandahl brendandahl committed on GitHub Merge pull request #1878 from brendandahl/shell-perf
Make node shell performance match chrome performance.
Permalink
Failed to load latest commit information.
bench Merge branch 'master' into intex Aug 27, 2015
certs document provenance and location of mekeytool on Mac Jun 19, 2015
config Allow compile thresholds to be set in config. Aug 14, 2015
img add build configuration for RunWhatsApp and RunWeChat (preview) targets Jun 18, 2015
java Merge branch 'intex-jit' of https://github.com/brendandahl/j2me.js in… Aug 11, 2015
jit Fix j2me for type script v1.8.10 and TextEncoder changes. Sep 30, 2016
l10n Remove now unused localization string Jun 15, 2015
libs Merge pull request #1598 from mozilla/intex Dec 5, 2015
midp Always explicitely call getArrayFromAddr; make getHandle work only fo… Aug 26, 2015
polyfill Use main-all.js in jsshell. Aug 5, 2015
shell Make node shell performance match chrome performance. Jan 5, 2017
shumway Use const enum when possible; fix order of references Aug 13, 2015
style remove unused confirm dialog pseudo element Jun 24, 2015
tests Merge pull request #1598 from mozilla/intex Dec 5, 2015
tools zero primitive arrays with memset Aug 14, 2015
vm Fix j2me for type script v1.8.10 and TextEncoder changes. Sep 30, 2016
.gitignore Only force collection when free memory is low. Aug 20, 2015
.gitmodules Add Boehm.js submodule. May 14, 2015
.travis.yml Issues 1873 - Set typescript version for tsc compile error. Dec 7, 2015
LICENSE move GPL to top level and use version at http://www.gnu.org/licenses/… Feb 13, 2015
Makefile Merge pull request #1831 from mbebenita/intex-cherry Aug 24, 2015
README.md Revert "Merge pull request #1721 from mozilla/revert-1712-intex-pass-… Jul 8, 2015
aot-methods.txt Remove testing method from aot-methods.txt Aug 17, 2015
asteroidzone.mk simplify startup screens Apr 28, 2015
benchmark.js Log benchmark times while going. Aug 14, 2015
bindings.ts.in Throw a more detailed error message. Aug 28, 2015
blackBox.js Add missing newlines. Jan 14, 2015
bytecodes.ts Use const enums everywhere. Aug 21, 2015
config.ts.in Only force collection when free memory is low. Aug 20, 2015
game-ui.js Clean up MIDP foreground info May 7, 2015
gc.html Use minimize memory approach and gc in new page between rounds. Mar 25, 2015
gc.js call location.replace to avoid creating history entry while collectin… May 5, 2015
index.html.in remove obsolete config/midlet.js Apr 8, 2015
index.js.in Impl nextTickDuringEvents, rename setZeroTimeout Jul 6, 2015
int.ts Fix j2me for type script v1.8.10 and TextEncoder changes. Sep 30, 2016
jsc.ts Merge branch 'intex-jit' of https://github.com/brendandahl/j2me.js in… Aug 11, 2015
long.ts Revert revert. Jun 5, 2015
main.html.in Compile vm/native/native.cpp and jit/relooper/Relooper.cpp together. … Aug 10, 2015
main.js require initFS before loading non-midlet program Aug 27, 2015
manifest.webapp.in Configure the share mozActivity filters with a Makefile configuration… Jun 10, 2015
metrics.ts Merge. Dec 12, 2014
nat.ts Merge pull request #1831 from mbebenita/intex-cherry Aug 24, 2015
native.js Merge branch 'master' into intex Aug 27, 2015
options.ts Require a message in assert Jul 21, 2015
references-jsc.ts Use const enum when possible; fix order of references Aug 13, 2015
references.ts Use const enum when possible; fix order of references Aug 13, 2015
runwechatpreview.mk add build configuration for RunWhatsApp and RunWeChat (preview) targets Jun 18, 2015
runwhatsapppreview.mk Increase ASMJS_TOTAL_MEMORY for WA because of #1817 Aug 17, 2015
scheduler.ts Make priority a property of Context Jul 21, 2015
string.js Revert "Merge pull request #1721 from mozilla/revert-1712-intex-pass-… Jul 8, 2015
timer.js Fix browser timer. Aug 5, 2015
types.ts Use const enums everywhere. Aug 21, 2015
util.js Remove unused util.js functions Aug 14, 2015
utilities.ts Add a partially java compatible shell. Aug 28, 2015

README.md

PluotSorbet Build Status

PluotSorbet is a J2ME-compatible virtual machine written in JavaScript[1]. The goal of PluotSorbet is to run MIDlets in web apps without a native plugin.

The current goals of PluotSorbet are:

  1. Run MIDlets in a way that emulates the reference implementation of phone ME Feature MR4 (b01)
  2. Keep PluotSorbet simple and small: Leverage the phoneME JDK/infrastructure and existing Java code as much as we can, and implement as little as possible in JavaScript

Install vs Hot runs

PluotSorbet launches MIDlets under two different circumstances.

  • Install run only happens once per device. It spends extra time downloading, optimizing, and precompiling so that subsequent runs will be faster.
  • Cold runs require starting up both foreground and background MIDlets, if needed
  • Hot runs only require switching to the active app

Building PluotSorbet

Make sure you have wget and a JRE installed

You need to install the TypeScript compiler, the easiest way is via NPM: npm install -g typescript.

Get the PluotSorbet repo if you don't have it already

    git clone https://github.com/mozilla/pluotsorbet

Build using make:

    cd pluotsorbet
    make

Running apps & MIDlets, Debugging

index.html is a webapp that runs PluotSorbet. The URL parameters you pass to index.html control the specific behavior of PluotSorbet.

URL parameters

You can specify URL parameters to override the configuration. See the full list of parameters at config/urlparams.js.

  • main - default is com/sun/midp/main/MIDletSuiteLoader
  • midletClassName - must be set to the main class to run. Only valid when default main parameter is used. Defaults to RunTestsMIDlet
  • autosize - if set to 1, j2me app will fill the page.
  • gamepad - if set to 1, gamepad will be visible/available.

Desktop

To run a MIDlet on desktop, you must first start an http server that will host index.html. You can then connect to the http server, passing URL parameters to index.html

    python tests/httpServer.py &
    http://localhost:8000/index.html?jad=ExampleApp.jad&jars=ExampleApp.jar&midletClassName=com.example.yourClassNameHere

Example - Asteroids

    python tests/httpServer.py &
    http://localhost:8000/index.html?midletClassName=asteroids.Game&jars=tests/tests.jar&gamepad=1

Some apps require access to APIs that aren't enabled by default on Desktop Firefox and there is no UI built in to Desktop Firefox to enable them. APIs matching this description include:

  • mozTCPSocket
  • mozContacts
  • mozbrowser
  • notifications

To enable this type of API for a MIDlet you're running, use Myk's API Enabler Addon

FirefoxOS device (or emulator)

To run a MIDlet on a FirefoxOS device, update the launch_path property in manifest.webapp. The midletClassName URL parameter needs to point to an app.

Once you've updated manifest.webapp, connect to the device or emulator as described in the FirefoxOS Developer Phone Guide and select your PluotSorbet directory (the one containing manifest.webapp) when choosing the app to push to device.

Example - Asteroids

    "launch_path": "/index.html?midletClassName=asteroids.Game&jars=tests/tests.jar&logConsole=web&autosize=1&gamepad=1"

Tests

You can run the test suite with make test. The main driver for the test suite is tests/automation.js which uses the CasperJS testing framework and SlimerJS (a Gecko backend for CasperJS). This test suite runs on every push (continuous integration) thanks to Travis CI.

make test downloads SlimerJS for you automatically, but you have to install CasperJS yourself. The easiest way to do that is via NPM: npm install -g casperjs. On Mac, you may also be able to install it via Brew.

If you want to pass additional casperJS command line options, look at the "test" target in Makefile and place additional command line options before the automation.js filename.

gfx tests use image comparison; a reference image is provided with the test and the output of the test must match the reference image. The output is allowed to differ from the reference image by a number of pixels specified in automation.js.

The main set of unit tests that automation.js runs is the set covered by the RunTests class. The full list of RunTests tests available in the tests/Testlets.java generated file. RunTests runs a number of "Testlets" (Java classes that implement the Testlet interface). Testlets that require to be executed in a MIDlet environment (classes that implement the MIDletTestlet interface) are run by RunTestsMIDlet. The full list of these tests is available in the tests/MIDletTestlets.java generated file.

Running a single test

If the test you want to run is a class with a main method, specify a main URL parameter to index.html, e.g.:

    main=gnu/testlet/vm/SystemTest&jars=tests/tests.jar

If the test you want to run is a MIDlet, specify midletClassName and jad URL parameters to index.html (main will default to the MIDletSuiteLoader), e.g.:

    midletClassName=tests/alarm/MIDlet1&jad=tests/midlets/alarm/alarm.jad&jars=tests/tests.jar

If the test you want to run is a Testlet , specify an args URL parameter to index.html. You can specify multiple Testlets separated by commas, and you can use either '.' or '/' in the class name, e.g.:

    args=java.lang.TestSystem,javax.crypto.TestRC4,com/nokia/mid/ui/TestVirtualKeyboard

If the testlet uses sockets, you must start 4 servers (instead of just the http server):

    python tests/httpServer.py &
    python tests/echoServer.py &
    cd tests && python httpsServer.py &
    cd tests && python sslEchoServer.py &

Failures (and what to do)

Frequent causes of failure include:

  • timeout: Travis machines are generally slower than dev machines and so tests that pass locally will fail in the continuous integration tests
  • Number of differing pixels in a gfx/rendering test exceeds the threshold allowed in automation.js. This will often happen because slimerJS uses a different version of Firefox than the developer. This can also happen because the test renders text, whose font rendering can vary from machine to machine, perhaps even with the same font.

gfx/rendering tests will print a number next to the error message. That number is the number of differing pixels. If it is close to the threshold you can probably just increase the threshold in automation.js with no ill effect.

The test output will include base64 encoded images; copy this into your browser's URL bar as a data URL to see what the actual test output looked like.

When running make test, verbose test output will be printed to your terminal. Check that for additional info on the failures that occurred.

Logging

See logConsole and logLevel URL params in libs/console.js

Running PluotSorbet in the SpiderMonkey shell

  1. Download the SpiderMonkey shell
  2. Execute the jsshell.js file as follows:

    js jsshell.js package.ClassName
    

Coding Style

In general, stick with whatever style exists in the file you are modifying.

If you're creating a new file, use 4-space indents for Java and 2-space indents of JS.

Use JavaDoc to document public APIs in Java.

Modeline for Java files:

/* vim: set filetype=java shiftwidth=4 tabstop=4 autoindent cindent expandtab : */

Modelines for JS files:

/* -*- Mode: Java; tab-width: 2; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
/* vim: set shiftwidth=2 tabstop=2 autoindent cindent expandtab: */

Profiling

JS profiling

One way to profile PluotSorbet is to use the JS profiler available in Firefox Dev Tools. This will tell us how well the JVM is working and how well natives work. This type of profiling will not measure time that is taken waiting for async callbacks to be called (for example, when using the native JS filesystem API).

VM profiling

The PluotSorbet VM has several profiling tools. The simplest feature is to use counters. runtime.ts defines several: runtimeCounter, nativeCounter, etc ... these are only available in debug builds.

To use them, just add calls to runtimeCounter.count(name, count = 1). To view accumulated counts, allow the application to run for some time and then click the Dump Counters button. If you want, reset the counter count any time by clicking Clear Counters.

  • Counting events:

    function readBytes(fileName, length) {
      runtimeCounter && runtimeCounter.count("readBytes");
    }
    
  • Counting bucketed events:

    function readBytes(fileName, length) {
      runtimeCounter && runtimeCounter.count("readBytes " + fileName);
    }
    
  • Counting events with larger counts:

    function readBytes(fileName, length) {
      runtimeCounter && runtimeCounter.count("readBytes", length);
    }
    
  • Counting events with caller context: This is useful to understand which call sites are the most common.

    function readBytes(fileName, length) {
      runtimeCounter && runtimeCounter.count("readBytes " + arguments.callee.caller.name);
    }
    

The second, more heavy weight profiling tool is Shumway's timeline profiler. The profiler records enter / leave events in a large circular buffer that can be later displayed visually as a flame chart or saved in a text format. To use it, build PluotSorbet with PROFILE=[1|2|4]. Then wrap code regions that you're interested in measuring with calls to timeline.enter / timeline.leave.

Java methods are automatically wrapped with calls to methodTimeline.enter / methodTimeline.leave. The resulting timeline is a very detailed trace of the application's execution. Note that this instrumentation has some overhead, and timing information of very short lived events may not be accurate and can lead to the entire application slowing down.

Similar to the way counters work, you can get creative with the timeline profiler. The API looks something like this:

timeline.enter(name: string, details?: Object);
timeline.leave(name?: string, details?: Object);

You must pair the calls to enter and leave but you don't necessarily need to specify arguments for name and details.

The name argument can be any string and it specifies a event type. The timeline view will draw different types of events in different colors. It will also give you some statistics about the number of times a certain event type was seen, how long it took, etc..

The details argument is an object whose properties are shown when you hover over a timeline segment in the profiler view. You can specify this object when you call timeline.enter or when you call timeline.leave. Usually, you have more information when you call leave so that's a more convenient place to put it.

The way in which you come up with event names can produce different results. In the profilingWrapper function, the key is used to specify the event type.

You can also create your own timelines. At the moment there are 3:

  • timeline: VM Events like loading class files, linking, etc.
  • methodTimeline: Method execution.
  • threadTimeline: Thread scheduling.

You may have to change the CSS height style of the profileContainer if you don't see all timelines.

Shumway's timeline viewer

Top band is an overview of all the timelines. Second band is the timeline, third is the threadTimeline and finally the fourth is the methodTimeline. Use your mouse wheel to zoom in and out, pan and hover.

The tooltip displays:

  • total: ms spent in this event including all the child events.
  • self: total - total sum of all child events.
  • count: number of events seen with this name.
  • all total and all self: cumulative total and self times for all events with this name.
  • the remaining fields show the custom data specified in the details object.

If you build with PROFILE=2, then the timeline will be saved to a text file instead of being shown in the flame chart. On desktop, you will be prompted to save the file. On the phone, the file will automatically be saved to /sdcard/downloads/profile.txt, which you can later pull with adb pull. Note that no timeline events under 0.1 ms are written to the file output. You can change this in main.js if you'd like.

PROFILE=4 works like PROFILE=2, but allows you to profile a custom "range" of the execution by adding calls to org.mozilla.internal.Sys::startProfile and org.mozilla.internal.Sys::stopProfile.

PROFILE=1 and PROFILE=2 automatically profile (most of) cold startup, from JVM.startIsolate0 to DisplayDevice.gainedForeground0.

Benchmarks

Startup Benchmark

The startup benchmark measures from when the benchmark.js file loads to the call of DisplayDevice.gainedForeground0. It also measures memory usage after startup. Included in a benchmark build are helpers to build baseline scores so that subsequent runs of the benchmark can be compared. A t-test is used in the comparison to see if the changes were significant.

To use:

It is recommended that a dedicated Firefox profile is used with the about:config preference of security.turn_off_all_security_so_that_viruses_can_take_over_this_computer set to true so garbage collection and cycle collection can be run in between test rounds. To do this on a Firefox OS device, see B2G/QA/Tips And Tricks.

  1. Check out the version you want to be the baseline (usually mozilla/master).
  2. Build a benchmark build with RELEASE=1 BENCHMARK=1 make. RELEASE=1 is not required, but it is recommended to avoid debug code from changing execution behavior.
  3. Open the midlet you want to test with &logLevel=log appended to the url and click Build Benchmark Baseline.
  4. When finished, the message FINISHED BUILDING BASELINE will show up in the log.
  5. Apply/check out your changes to the code.
  6. Rebuild RELEASE=1 BENCHMARK=1 make.
  7. Refresh the midlet.
  8. Click Run Startup Benchmark.
  9. Once done, the benchmark will dump results to the log. If it says "BETTER" or "WORSE" the t-test has determined the results were significant. If it says "SAME" the changes were likely not enough to be differentiated from the noise of the test.

Filesystem

midp/fs.js contains native implementations of various midp filesystem APIs.

Those implementations call out to lib/fs.js which is a JS implementation of a filesystem.

Java APIs are sync, so our implementation stores files in memory and makes them available mostly synchronously.

Implementing Java functions in native code

native keyword tells Java that the function is implemented in native code

e.g.:

public static native long currentTimeMillis();

Java compiler will do nothing to ensure that implementation actually exists. At runtime, implementation better be available or you'll get a runtime exception.

We use Native object in JS to handle creation and registration of native functions. See native.js

Native["name/of/function.(parameterTypes)returnType"] = jsFuncToCall;

e.g.:

Native["java/lang/System.arraycopy.(Ljava/lang/Object;ILjava/lang/Object;II)V" = function(addr, srcAddr, srcOffset, dstAddr, dstOffset, length) {...};

The first parameter of every native, addr, is the address in memory of the object on which the native is being called. If the native is static, then the value of addr is Constants.NULL. Call getHandle(addr) to get a handle to a JavaScript object that wraps the Java object with getters/setters for its fields.

If raising a Java Exception, throw new instance of Java Exception class as defined in vm/runtime.ts, e.g.:

throw $.newNullPointerException("Cannot copy to/from a null array.");

If you need implement a native method with async JS calls, the following steps are required:

  1. Add the method to the yieldMap in jit/analyze.ts
  2. Use asyncImpl in native.js to return the asnyc value with a Promise.

e.g:

Native["java/lang/Thread.sleep.(J)V"] = function(addr, delayL, delayH) {
    asyncImpl("V", new Promise(function(resolve, reject) {
        window.setTimeout(resolve, J2ME.longToNumber(delayL, delayH));
    }));
};

The asyncImpl call is optional if part of the code doesn't make async calls. The method can sometimes return a value synchronously, and the VM will handle it properly. However, if a native ever calls asyncImpl, even if it doesn't always do so, then you need to add the method to yieldMap.

e.g:

Native["java/lang/Thread.newSleep.(J)Z"] = function(addr, delayL, delayH) {
    var delay = J2ME.longToNumber(delayL, delayH);
    if (delay < 0) {
      // Return false synchronously. Note: we use 1 and 0 in JavaScript to
      // represent true and false in Java.
      return 0;
    }
    // Return true asynchronously with `asyncImpl`.
    asyncImpl("Z", new Promise(function(resolve, reject) {
        window.setTimeout(resolve.bind(null, 1), delay);
    }));
};

Remember:

  • Return types are automatically converted to Java types, but parameters are not automatically converted from Java types to JS types
  • $ is current runtime and $.ctx current Context
  • Parameter types are specified in JNI

Overriding Java functions with JavaScript functions

To override a Java function with a JavaScript function, simply define a Native as described earlier. Any Java functions can be overridden, not only Java functions with the native keyword.

Overriding Java functions only works in debug mode (RELEASE=0).

Packaging

make app packages PluotSorbet into an Open Web App in output directory. It's possible to simply package the entire contents of your working directory, but these tools will produce a better app.

Compiling With AOT Compiler

make aot compiles some Java code into JavaScript with an ahead-of-time (AOT) compiler.

To use it, first install a recent version of the JavaScript shell.

Compiling With Closure

make closure compiles some JavaScript code with the Closure compiler.


[1] JavaScript is a trademark or registered trademark of Sun Microsystems, Inc. in the U.S. and other countries, used under license.