SQL query interface to C++ collections and C data structures. Also configurable as a loadable Linux kernel module and an extension to Valgrind tools.
C Ruby HTML C++ Makefile Shell
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
doc Move doc files from bin to doc Jul 18, 2016
examples
src
LICENSE
Makefile.am
README.mediawiki
configure.ac

README.mediawiki

PiCO QL is a C/C++ plug-in library that sets up an SQL SELECT web-accessible online interactive interface over C/C++ data structures. SQL queries access the application's data structures in place.

This branch contains releases of the generic PiCO_QL plugin for C/C++ applications. It is not guaranteed to work in the kernel or Valgrind contexts.

For C++ applications, PiCO QL supports querying of containers compatible with the C++ SGI forward container and forward iterator concept. The container classes of the STL consist a prime example of such. PiCO QL provides concept checks to verify that containers follow the above concepts. It also supports querying of custom containers (using the USING LOOP directive) and mere objects. See the PiCO QL for C/C++ applications tutorial.

For C applications, PiCO QL supports querying of data structures like arrays and linked lists (see examples/CApp) but also any other data structure for which users will provide a loop variant (using the USING LOOP directive). It also supports querying of mere structs.

PiCO QL is also configurable as a loadable Linux kernel module. It provides both a /proc and a high-level web interface to be able to issue SQL queries against kernel data structures. Currently PiCO QL supports data structures modelling processes (such as the linked list of processes) , files, virtual memory, file system, network, namespaces, devices, busses etc. Support is extensible to include more data structures. See Fast track roll your own probes tutorial in the PiCO QL LKM wiki.

PiCO QL is also configurable as an extension to Valgrind tools, that is MemCheck, CacheGrind, and CallGrind. It provides an online interactive web interface to the data structures employed by Valgrind tools that gather memory operations metadata from an application that Valgrind instruments at the time. In other words, while Valgrind tools gather metadata about how an application is using memory, at the same time users can query the metadata using SQL queries. Synchronization issues are skipped using a simple stop-the-clock implementation to poll for a query waiting to be executed.

Table of Contents

Platforms

   Userspace (built-in examples):
   Tested under Mac OS X 10.6.7-8, 10.8.5, 10.10.5
     Linux 2.6
     Windows 10 via Cygwin (under way)
   Linux kernel module:
     Tested on:
       Architectures: x86_64, amd64
       Distributions - kernels: Fedora 18 - 3.6.9,
                                Debian 7.1 - 3.2.0,
                                Debian 8 - 3.16.7,
                                Ubuntu server 10.04 - 2.6.32
   Valgrind extension:
     Tested under Mac OS X 10.6.7-8, 10.8.5

Testing

  • Userspace: Commits are verified against:
  • Kernel: Commits are verified against an array of SQL queries by executing a script.

Dependencies

  • Ruby Part of the library's source code (the parser/code generator) is written in Ruby.
  • SQLite3 While PiCO QL does not use a database or other store to copy and/or store data, it uses the virtual table (VT) mechanism of Sqlite3 to provide a relational representation of the underlying data model. The PiCO QL source code includes SQLite, so it does not have to be installed in your system.
  • SWILL Once plugged in an application, PiCO QL uses the same or a separate thread and utilises the SWILL library to present a local server with a web-like, user friendly query interface. In case of the kernel configuration this interface wraps the /proc interface.
  • Boost For C++ the library accepts any container iterable in some fashion and provides concept checks (compile-time), implemented by Boost, for containers modelled after the ForwardContainer and ForwardIterator concepts

Installation

   autoreconf --install (only the first time)
   ./configure [--enable-run-tests=yes] [--prefix=...]
   make
   make install
   make test      # if --enable-run-tests has been used

To see the PiCO QL library in action:

  • do not use --enable-run-tests
  • head to the installation directory
  • run one of the example executables: bankapp, chess, capp, vrp, polymoprhism
To compile under Cygwin you might need to use:

   make CFLAGS='-Dcygwin_conv_to_full_win32_path\(path,fullpath\)=cygwin_conv_path\(CCP_WIN_A_TO_POSIX, path, fullpath, PATH_MAX\)'

The C++ API

   int init(const char** pragmas, int npragmas, int port_number, pthread_t *t);
   void register_data(const void *collection, const char * col_name);
   int exec_query(const char *query, stringstream &s,
   		int (*callback)(sqlite3 *, sqlite3_stmt *, stringstream &));
   int step_mute(sqlite3 *db, sqlite3_stmt *stmt, stringstream &s);
   int step_text(sqlite3 *db, sqlite3_stmt *stmt, stringstream &s);
   int step_swill_html(sqlite3 *db, sqlite3_stmt *stmt, stringstream &s);
   int step_swill_json(sqlite3 *db, sqlite3_stmt *stmt, stringstream &s);
   int progress(int n, int (*callback)(void *p), void *p);
   int interrupt();
   int shutdown();
   int create_function(const char *name, int argc, int text_rep, void *p,
   		void (*xFunc)(sqlite3_context*,int,sqlite3_value**),
   		void (*xStep)(sqlite3_context*,int,sqlite3_value**),
   		void (*xFinal)(sqlite3_context*));

See src/pico_ql.h for the full C/C++ API

Package structure

  • At the examples/ directory, users can browse:
Several implemented userspace application example scenarios which can be executed effortlessly as mentioned above.

Behind the scenes a compiler written in Ruby parses a relational specification and creates the source code of the relational interface that is later used to execute queries on the application's data structures. Here is how the compiler is invoked and the arguments it accepts:

        ruby generator.rb pico_ql_dsl.sql [debug] [kernel] [C] [no_mem_mgt] [concept_check]

There are also different configuration alternatives available at the make step configuration options.

  • At the doc/ directory there is the BNF of the DSL that grounds the user description syntax and the wiki pages.
  • At src/ directory, the library's source code is found.