Leam Architecture
Leam is developed as a web application and is implemented using ReactJS and Flask framework. Following figure captures the Leam architecture.
On the client or front end side,
the Leam client is responsible
for capturing user input
on various interface components,
and for rendering the views
based on results returned by the
back-end. Examples of user interactions include
VITAL commands in Code Editor,
operator selection in Operations Menu operator,
interaction with Data View and charts in Chart View.
Given any user interaction
on the front end,
the Leam Request Processor
issues a request to the backend.
The backend Leam Controller
manages the uploaded data and sessions
while propagating user interactions to the session
manager. The session manager interprets the user
interaction---any interactions on the Operations Menu
is sent to a lightweight VTA Compiler
while the VITAL commands on the Code Editor are pushed in a task queue.
The VTA compiler translates the user-selected operator to a VITAL
command which is then executed by the VTA Executor.
Since users can either write single- or multi-line VITAL commands,
Leam backend employs a Task Queue to keep track of the sequence of
tasks. Whenever the VTA executor completes the execution of a VITAL command, the session
manager fetches an unprocessed command from the task queue and sends that to the
executor. The VTA executor leverages the python runtime to access any global or local variables in the session and executes the command on the desired data.
Leam backend utilizes a customized dataframe called VITAFrame for enabling text data analysis operations. VITAFrame columns are complex objects that not only contains the raw data, but also the schema of the column, associated metadata and the metadata schema type.
Such a column specification with schema is necessary to validate a VTA operation. Leam session manger also employs a View Cache to track states of the front end components and always remains in sync with the front end. If the states of a component is updated after a VITAL command execution, the changes are propagated to the front end. Also, Leam employs a Coordination Graph to manage coordination among linked views. For each view in the front end, Leam maintains an adjacency list---for any interaction on a view, all the views in its adjacency list are updated.