Permalink
Browse files

rethinked-restructured

  • Loading branch information...
daitangio committed Dec 16, 2016
1 parent 9fa9652 commit a734829019404c1951eeddc191a50034ee064785
Showing with 145 additions and 122 deletions.
  1. +8 −0 rtype/import-wiz.sql
  2. +77 −120 rtype/rtype-readme-first.txt
  3. +58 −0 rtype/rtype2017-boot.sql
  4. +2 −2 rtype/src/rtype.py
View
@@ -0,0 +1,8 @@
-- Import wiz load data
attach database ":memory:" as loader_area;
.mode csv
.separator ;
.import C:/giorgi/nttdata/std_cost_2014.csv std_cost
-- select * from std_cost;
@@ -3,63 +3,91 @@
#+Author: Giovanni Giorgi
#+Email: jj@gioorgi.com
# ^c^v t to export code
* Wellcome to RType!
+ Data are opaque entity, without nulls, and with distinct values
+ Data are accessed via /Views/
+ Views are bi-dimensional relations
For instance, the file system can be accessed via a View:
#+BEGIN_SRC python :tangle rtypeV1.py
#!/bin/env python
# Example1
import sys
sys.path.append("src")
from rtype import view
def select(view,fname,size):
result=view.orderby(fname)
for line in result:
print line.fname,line.size
# Pass the meta-select function to the viewbuilder
view('filesystem:/var/',select);
# Example2
# Gropup people by birthplace:
#def group(view,name,
#+END_SRC
In RType, data can grow to disk without problem.
The RType system will provide the best relational adapter system (memory, RDBMS, NoSQL db, etc) based on growth usage
Internally RType will guarantee the best efficent way to store the data.
RType is aimed to provied maximum performance for the following operation: put, get, intersect, union
Removal is a lazy operation, and will be performed as suboptimal task.
* No polling: Simple Event engine
RType provide also a event-listener model to attach event to set modifications like:
+ addition of a new element
+ removal of an element
* Concurrent: multi process on shared file system
* RType: your fastest Relational database :tryout:
** A table is described as a /set/ of column names only
Internally the system can be meta-described by ordered set.
#+BEGIN_SRC js
db=RType.openDB(":memory:");
db.createTable("person", "person_id","name","surname","birdth_dt");
// II declaration form:
db.createTable({
"house":[
"location",
"owner_person_id"
],
"car": [
"engine_cc_n",
"year_dt",
"owner_person_id"
]
});
w=db.getInsertWorker(); // <- Autobild a magical interface to your tables, baby
w.insertOnPerson(1,"Bob","Testy", "19740423");
#+END_SRC
Please observe:
1) For every database a Worker is provided
2) The worker is a typed (somewhat) interface, so you must /know/ at
"design" (compile) time the table names (at least).
3) You must always provide your table declaration, or your system will
not work at all.
4) You can provide a partial table declaration (only the segment you
need):
#+BEGIN_SRC js2
db=RType.openDB(":memory:").getWorkerFor(
{ "car":
[ "engine_cc_n","year_dt","model","owner_person_id"]
}
).insertOnCar(1200,"19990101","FordFiesta",1);
#+END_SRC
When you define a RType database you must specify the maximum key length you care about.
The key length is fixed for performance reason and will directly affect some storage performance.
** You must define a set of Regular expression to tie your model
Foreign keys and type are identified by convention:
For example:
*** $name_id -> integer, primary key or foriegn key
*** $n_flg -> boolean (0,1)
*** $number_n -> number value (internally stored as string)
*** $date_dt -> iso date (internally stored as string yyyymmdd)
*** $timestamp_ts -> date + time n GMT format yyymmdd-hhmmss.xyz...
*** $othername -> strings with 4000 char limit
RType storage engine is pluggable.
The reference implementation provide a very fast engine and a trasactional-optional engine (altrougth based on a Global Lock).
* RType secret trick: optimal for simple relational database
** Foreign Key are not enforced
You can proibit insert (dev mode) or allow them (production, less-rigid mode).
** RType is a relationa database supporting joins
RType core is based on embedded SQLite
** RType did not delete :beta:
RType mark elements for deletation only.
An async job re-build tables when system is on low load.
** RType system can manage load peaks, but not constant load peaks
In low-load condition RType do the following actions:
1. Try to reduce to zero the async queue
2.
** No polling: Simple Event engine
RType provide also a event-listener model to attach event to set modifications like:
+ addition of a new element
+ mark-removal of an element
** RType has no transactional semantic :beta:
Storing data has a fixed, predictive cost.
** RType has no replication policy :beta:
Client library manage a pool of replicated RType system.
RType has a async operation model in which all operations are asyncronous.
A standard way of ensuing operation is to do a cascate set of async
operation, one after one.
** RType storage engine is pluggable.
The reference implementation provide a very fast engine
** Symbolic language support
RType is based on /atoms/ which are unique in the system.
* TODO RType secret trick: optimal for simple relational database :resee:
If you have a simple relational database, with a lot of one-to-many relations you can use RType to get the job done.
A small SQL-like declarative language is provided in RType for easy access:
@@ -76,77 +104,6 @@ A small SQL-like declarative language is provided in RType for easy access:
Many2Many relations can be mapped via two pairs of one-to-many relations.
* API
* IDEA2
** Symbol is a declarative language...
with imperative nature.
The expression can contain 2-dimensional relation
#+BEGIN_SRC text
Hello_world.string().print()
=> "Hello world"
person[name,age] = 'Nick,'47, 'Clara,'23
person[name,age].orderby(age).count()
// Lambda guy:
(int x) -> x * x
(int x) -> { ..... }
#+END_SRC
comma ',' is used to separate data.
' is used to quote symbol. Quoted symbol can be seen as string (anyway they are unique).
Escape is obtained via the '\' char
So a comma string is
'\,,
Quoted symbol are string.
String can be transformed into symbol with the symbol function
Nick == ('Nick).symbol()
And symbol can be stringified via string() function so
Nick.string() == "Nick" == 'Nick
Nick.string().symbol() == Nick
Function are directly applied to types,
** File are data, data are files
In symbol file system primitive and data is the same.
You can usually access to file system via a relational view, using a special function wich remap file system:
#+BEGIN_SRC text
fs('/home/jj')[name,creation_time].print()
#+END_SRC
The [ ] is the "select" operator which remap
* Reference
1) http://research.swtch.com/sparse
View
@@ -0,0 +1,58 @@
-- RType SQL Boot script
--
.print "> RType 2017 setup..."
create table if not exists rtype(
key text unique,
value text);
-- delete from rtype;
insert or ignore into rtype values('version',0.1);
insert or ignore into rtype values('product_name','RType');
-- Default FK regexp
insert or ignore into rtype values('fk_regexp','.*_ID');
attach database ":memory:" as fk_validator;
-- Demo table
create table if not exists person(
person_id integer primary key autoincrement,
name text,
surname text
);
create table if not exists car(
car_id integer primary key autoincrement,
owber_person_id integer,
model text,
year integer);
-- Select RANK EMULATION
/*
SELECT Products.Product,
DENSE_RANK() OVER (ORDER BY Products.Code DESC) AS Rank
FROM Products;
In SQLite:
SELECT Product,
(SELECT COUNT()+1 FROM (
SELECT DISTINCT Code FROM Products AS t WHERE Code < Products.Code)
) AS Rank
FROM Products;
*/
.print ">> Demo table ok"
.print ">>> Emitting json configuration"
--json_object('rtype_dump',
-- json_set('{"a":2,"c":4}', '$.c', json_array(97,96)) → '{"a":2,"c":[97,96]}'
select json_set('{"config": "cfg"}','$.config',C.X) from
(select json_group_object(key,value) AS X from rtype
order by key) C;
select * from rtype;
-- .dump
-- .mode cvs
-- .separator ;
-- .import
View
@@ -15,15 +15,15 @@
Minimum Required:Python 2.7.6
Support
https://code.google.com/p/asq/
https://github.com/sixty-north/asq
linq-like for python (over 40 functions)
Demo:
you organize a simple people/friend/place stuff
then you start filling it with data, and when data growth, the system accomodate it
displacing data into different
displacing data into different storages
"""

0 comments on commit a734829

Please sign in to comment.