Inter-Language Persistent Shared Memory
In The Future™, memory will have new capabilities that programmers will make decisions about how best to use:
- Non-Volatility - No more "loading" files into variables in memory, just use the same variables as last time.
- Sharing - Directly sharing objects between programs instead of communicating them through network messages.
- Selectable Latency - Expensive fast memory for frequently accessed variables, cheaper slower memory for less frequently used data.
- Correctness - Very inexpensive bulk memory might give approximate values instead of the exact value stored.
- Location - Allocate memory in another another rack or data center
- Mirroring - Automatic fail-over to backup memory
EMS presently supports the first two and is ready to incorporate new memory capabilities as they become available.
Sharing Objects Between Programs
Sharing EMS data between programs written in different languages is no different than sharing data between programs in the same language, or sharing data between same program run multiple times.
>>> a = [1,2,3] >>> a + "abc" Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: can only concatenate list (not "str") to list >>> a += "abc" >>> a [1, 2, 3, 'a', 'b', 'c'] >>>
> a = [1,2,3] [ 1, 2, 3 ] > a + 'abc' '1,2,3abc' > a += 'abc' '1,2,3abc' >
You may be nonplussed to learn this is The Future™. I, too, was horrified and baffled at this discovery. Alas, this is an inevitable truth of The Future™.
Possible Orders for Starting Processes
Two processes sharing memory may start in any order, meaning programs must take care when initializing to not overwrite another program which may also be trying to initialize, or detect when it is necessary to recover from a previous error.
|Persistent EMS Array File||(Re-)Initialize||Use|
|Already exists||A one-time initialization process truncates and creates the EMS array. The first program to start must create the EMS file with the
||Any number of processes may attach to the EMS array in any order using the attribute
|Does not yet exist||Subsequent programs attach to the new EMS file with the
Running the Example
interlanguage.js are complementary
to each other, and must be started in the correct order:
dunlin> node interlanguage.js JS Started, waiting for Python to start... [ EXECUTION STALLS UNTIL PYTHON PROGRAM STARTS ] Hello from Python ...
Second, start the Python program
Due to a well-intentioned signal mask in the Python runtime meant to provide resiliency in the face of a failing native plug-in module, sometimes Python will appear hung when it is blocked on an EMS operation. In these situations an alternative way to kill the Python interpreter is with SIGQUIT (Control-\ from most shells), or back-grounding the job with Control-Z and sending the job a kill signal.
What the Example is Doing
- Each process writes a hello message to an EMS variable, marking it full.
- Each process reads the variable written by the other process, blocking until it has been written
- A simple point-to-point barrier is demonstrated by handshaking
- Both JS and Py enter a loop that increments a shared counter
- JS checks the results to make sure every iteration occurred once
- Illustrate some quirks of language design in JS and Py
- JS leaves a shared counter running in the background via
- Python enters a loop that occasionally reads the shared counter, showing it being updated
- Python over-writes the counter with
- JS notices the counter has been replaced with
Illustration of sharing data between the Node.js and Python done interactively from the REPLs.