Skip to content

niclasko/FuseBase

Repository files navigation

FuseBase

Data Convergence Engine

FuseBase is a data convergence server for JDBC-enabled data sources. For inquiries reach out to Niclas Kjäll-Ohlsson (niclas@fusebase.io) or Bjørnar Fjøren (bjornar@fusebase.io).

FuseBase offers the following capabilities:

  • JDBC connections via drop-in drivers for any JDBC data source.
  • Javascript engine to run scripts server-side
  • SQL parser
  • Job Scheduler for registered scripts
  • Fast data offloading to disk (CSV) from any JDBC data source
  • Fast data ingestion via JDBC
  • Data virtualization (via extension)
  • Config done using WEB UI
  • Usage is via API or WEB UI

Quick start:

$ java -jar fusebase.jar [-port=4444]

Then

Script usage example (data offloading to CSV)

  1. Login
  2. Under Connections tab, register JDBC connection as CONNECTION_NAME for data source for offloading data from and click Connect
  3. Under Scripts tab, run or schedule following Javascript-script:
    var query = "select * from bigtable";
    var outputFilePath = "/data/out.csv";
    FuseBase.queryToFile(CONNECTION_NAME, query, outputFilePath, "CSV");
    output.writeBytes("Done");
    

API usage example (SQL parser)

  1. Login

  2. Users tab

  3. For a user click "Client Keys"

  4. Click "Generate client key"

  5. Copy "Authentication Link" and call it (next step)

    curl http://localhost:4444/clientkey/authenticate?clientKey=YOUR_GENERATED_CLIENT_KEY&outputType=JSON

  6. Copy sessionKey and use it in API call (next step)

    curl http://localhost:4444/sql/parse?sql=select%20*%20from%20dual%20a%20inner%20join%20dual%20b%20on%20(1%3D1)&fusebase_sk=YOUR_SESSION_KEY_FROM_PREVIOUS_STEP

  7. Output

    SQL Parse tree:
    
    - LITERAL (ATOM)
    -- SELECT (CLAUSE)
    --- MULTIPLY (BINARY_OPERATOR)
    -- FROM (CLAUSE)
    --- LITERAL (ATOM): dual
    --- LITERAL (ATOM): a
    --- INNER_JOIN (JOIN)
    --- LITERAL (ATOM): dual
    --- LITERAL (ATOM): b
    --- ON (JOIN_FILTER)
    --- BEGIN_PARANTHESES (PARANTHESES)
    --- NUMBER_ATOM (ATOM): 1.0
    --- EQUALS (BINARY_OPERATOR)
    --- NUMBER_ATOM (ATOM): 1.0
    --- END_PARANTHESES (PARANTHESES)
    
    SQL Parse tree as JSON:
    
    {"KEYWORD": "ROOT","children": [{"KEYWORD": "SELECT","TYPE": "CLAUSE","TOKEN": "select","children": [{"KEYWORD": "MULTIPLY","TYPE": "BINARY_OPERATOR","TOKEN": "*"}]},{"KEYWORD": "FROM","TYPE": "CLAUSE","TOKEN": "from","children": [{"KEYWORD": "LITERAL","TYPE": "ATOM","TOKEN": "dual"},{"KEYWORD": "LITERAL","TYPE": "ATOM","TOKEN": "a"},{"KEYWORD": "INNER_JOIN","TYPE": "JOIN","TOKEN": "inner join"},{"KEYWORD": "LITERAL","TYPE": "ATOM","TOKEN": "dual"},{"KEYWORD": "LITERAL","TYPE": "ATOM","TOKEN": "b"},{"KEYWORD": "ON","TYPE": "JOIN_FILTER","TOKEN": "on"},{"KEYWORD": "BEGIN_PARANTHESES","TYPE": "PARANTHESES","TOKEN": "("},{"KEYWORD": "NUMBER_ATOM","TYPE": "ATOM","TOKEN": "1"},{"KEYWORD": "EQUALS","TYPE": "BINARY_OPERATOR","TOKEN": "="},{"KEYWORD": "NUMBER_ATOM","TYPE": "ATOM","TOKEN": "1"},{"KEYWORD": "END_PARANTHESES","TYPE": "PARANTHESES","TOKEN": ")"}]}]}
    

Add JDBC driver

  1. Stop FuseBase if running
  2. Copy JDBC driver to jdbc_drivers directory under runtime directory (directory where fusebase.jar is located)
  3. Add entry for JDBC driver in ./jdbc_drivers/JDBCDrivers.txt. For example:
    [Oracle]
    DriverPath=./jdbc_drivers/Oracle/ojdbc6.jar
    ClassName=oracle.jdbc.driver.OracleDriver
    ConnectStringPattern=jdbc:oracle:thin:@//hostname:port/servicename
    ValidationQuery=SELECT 1 FROM DUAL
    ConnectionPoolSize=5
    
  4. Start FuseBase and use JDBC driver

Build

  1. Build
    $ ./buildjar.sh
    
  2. Copy contents from ./build folder to runtime folder of choice
  3. Run FuseBase