-
Notifications
You must be signed in to change notification settings - Fork 0
big.table
this page has been written before main modular architecture decisions. It describes mainly what will reside in big.table module and some parts of integration with some other modules.
You find here the core software architecture of the platform
- dop: stands for Datum Open Platform
We show here all major blocks: upper block needs lower blocks
+----------------------------------------------------------+
| +-------------+ |
| dopGui |shinyWidgets | |
| +-------------+ |
+----------------------------------------------------------+
+-------------------+ +-------------------+ +--------------+ +----------+
|+-------+ +-------+| |+-------+ +-------+| | | | |
||dopNode| |dopNode|| ||dopNode| |dopNode|| | | | |
|+-------+ +-------+| |+-------+ +-------+| | | | | +-------------------+
|+-------+ +-------+| |+-------+ +-------+| | | | |<-----------> | WebApp |
||dopNode| |dopNode|| ||dopNode| |dopNode|| | ... | | | +-------------------+
|+-------+ +-------+| |+-------+ +-------+| | | | | +-------------------+
| | | | | | | |<-----------> | CDN/ReverseProxy |
| dopProject | | dopProject | | | | | +-------------------+
| | | | | | | | +-------------------+
+-------------------+ +-------------------+ +--------------+ | |<-----------> | IOT Device / GW |
+----------------------------+ +---------------------------+ | | +-------------------+
| | | | | | +-------------------+
| dopCodeWriter | | dopPlugins | | REST API |<-----------> | remote OpenDATUM |
| | | | | | +-------------------+
+----------------------------+ +---------------------------+ +----------+
+----------------------------------------------------------+ +----------+
| <-+-+-> |
| GNU R + CRAN libs | |WebServer |
| | | |
+----------------------------------------------------------+ +----------+
+-----------------------------------------------------------------------+
| |
| COMPUTATION NODE (Bare metal/VM/docker) |
| |
+-----------------------------------------------------------------------+
- dopNode is the smallest element used to access a pice of data.
- dopProject is a list of dopNodes with strong links between them
- dopGui is the data exploration front end where dopNodes are represented as circles and links between dopNodes are shown.
- dopCodeWriter is in charge of writing and updating R code in order to reflect the semantic interconnection build with through the dopGui
- WebServer: could be a simple server from CRAN or a more specific one.
+-----------------------------------------------+
| dopNode: LIST OF ALL [dopInstance] |
| |
| +-------------------------------------+1..n |
| | dopTableQualifier | |
| | | |
| | TABLE OF ATTRIBUTES THAT | |
| | DISTINGUISHES EACH dopInstance | |
| | | |
| +-------------------------------------+ |
| |
| +-------------------------------------+1..n |
| | dopInstance | |
| | | |
| | WHAT WE FOCUS ON (semantic level) | |
| | | |
| | = [dopValue] * [dopAttribute] | |
| | | |
| | | |
| +-------------------------------------+ |
+-----------------------------------------------+
Each node is focusing on one semantic element [dopInstance]. At a semantic level a dopInstance shall be replaceable by a noun or a noun phrase.
[dopProject] <=> "Lyon, France"
[dopElement] <=> "Weather Capture"
=> [dopNode] => all "Weather Capture" of the "Lyon, France"
The node gives access to the list of all these elements in the project scope. We use [dopTableQualifier] to distinguish one instance of the semantic element from others (think about a unique key with multiple columns). At semantic level it the adjective that qualify the best the specificity of each [dopInstance] against the others.
[dopElement][1] <=> "Weather Capture" with Date='2017-01-01' and station='Croix-Rousse'
[dopElement][2] <=> "Weather Capture" with Date='2017-01-02' and station='Tete d'Or'
=> [dopTableQualifier] => dopElementId | Date | station
-------------+------------+-------------
1 | 2017-01-01 | Croix-Rousse
2 | 2017-01-02 | Tete d'Or
Finally each [dopInstance] could be a data.frame (table) of any R object. When it is a data.frame, we go further: columns are considered as attributes [dopAttribute] of [dopInstance]. Each [dopInstance] can have several values [dopValue] for the same attributes. One Line is considered as covering one aspect of one [dopElement].
[dopElement][1] <=> Temperature | Wind | Humidity | Visibility | Time
------------+-------+----------+----------- +----------
12 | 4 | 37 | good | 11:00:00
15 | 3 | 55 | medium | 17:00:00
=> 'Temperature' of "Weather Capture" with Date='2017-01-01' and station='Croix-Rousse' is c(12,15)
=> Mean of 'Temperature' of "Weather Capture" with Date='2017-01-01' and station='Croix-Rousse' is 13
Whenever the sentence does not construct well or is not easily understandable, it is time to rearrange the structure of [dopNode] so that it matches common understanding. The will be a new [dopNode] referring to the current one as its data provider (or parent). This data transformation is simplified by the use of [dopGui] and [dopCodeWriter].
+------------+
| |with OpenDATUM
| R Script |library(DOP)
| |
+------------+
|
| +--------------+ +---------------+1..*
| loadProject | |load | |
|-----------> | dopProject |----> | dopNode |
| | | | |
| +--------------+ +---------------+
| | |
| | |
| requestNode | selectNode |
|-----------------> |--------------------> |
| <-----------------| <------------------- |
| | |
| | |
| query | |
|----------------------------------------->|
| | |
| |<-------------------- | query needed data fromparent nodes
| | |
| | | compute local function
| <--------------------------------------- |
| | |
| | |
OpenDATUM is design to easily add new aspects using predefined hooks