persistent monitor (for static source code analysis, GCC based)
Clone or download
Permalink
Failed to load latest commit information.
attic moved build.ninja & cold.c to attic/ Jul 22, 2017
dependencies adding dependencies Jun 9, 2018
doc explain more issues on single-page applications Sep 16, 2018
gccplugins adding gccplugins/!README Jun 2, 2018
modubin use modubin/ not modulbin/ for binary shared objects of modules May 31, 2018
modules extracting the module bindings in scan_jsexpr°jsvar_object Sep 17, 2018
webroot changed favicon.ico Jul 20, 2018
.gitattributes dump store in name then id order Sep 12, 2017
.gitignore ignore modules/badm*.c files Sep 5, 2018
.gitmodules adding dependencies Jun 9, 2018
.indent.pro adding .indent.pro Jul 20, 2017
BM_makeconst.cc checking ids form for BMK_ constants Aug 22, 2018
COPYING3 copied better-objects branch to master Mar 29, 2018
LICENSE Initial commit Jul 20, 2017
Makefile fully remove indentation of generated module code; Sep 14, 2018
README.md if ninja --version dont work, then ninja is too old Sep 13, 2018
_bm_delim.h add $= <name> inside $[...] construct to bind the current object into… Sep 13, 2018
_bm_global.h improve dump_scan°list_object Jun 13, 2018
_bm_predef.h adding webdict_root predefined Jul 26, 2018
_bm_types.h apply*BM functions use objroutaddr_BM with BMP_function_sig Nov 10, 2017
agenda_BM.c added end-of-file comment to every hand-written file not having some Aug 25, 2018
allocgc_BM.c added end-of-file comment to every hand-written file not having some Aug 25, 2018
assoc_BM.c added end-of-file comment to every hand-written file not having some Aug 25, 2018
bismon.css tabautocomplete_gui_cmd_BM buggy in cornercases Oct 23, 2017
bismon.h do_dynamic_onion_BM is waiting for and sending the response Aug 6, 2018
bismon.ui improve colors for GTK Sep 7, 2018
bismongtk the modules are not built, but should be Aug 10, 2018
bismonion the modules are not built, but should be Aug 10, 2018
bootsyntax.md adding caret delim, used for readmacro-s Sep 15, 2017
build-bismon-doc.sh improve documentation, and show the dump of first_test_module Sep 9, 2018
build-bismon-persistent-module.sh typo in _1ukG2M9NPuZ_2sIG8oHLDHp chunk Sep 3, 2018
build-bismon-temporary-module.sh remove tempmodule & singlemodule make targets so simplify build*sh sc… Jul 14, 2018
cmacros_BM.h improve documentation, and show the dump of first_test_module Sep 9, 2018
code_BM.c dump_value of node or closure is dumping the nil sons Aug 31, 2018
contributors_BM again add myself Basile Starynkevitch as the first contributor Jun 27, 2018
dump_BM.c added end-of-file comment to every hand-written file not having some Aug 25, 2018
emitcode_BM.c emit_module°plain_module dont backup the emitted file, but objstrbuff… Sep 14, 2018
engine_BM.c added end-of-file comment to every hand-written file not having some Aug 25, 2018
fundecl_BM.h add objislocked_BM Aug 31, 2018
gencode_BM.c typo in miniscan_stmt°basiclo_run Sep 12, 2018
generate-ninja-builder.sh fully remove indentation of generated module code; Sep 14, 2018
globals_BM.h start adding create_anonymous_web_session_BM Sep 12, 2018
gui_GTKBM.c add parse debug menu & parsedebugmsg_BM global Sep 2, 2018
guicode_BM.c minor improvements Sep 13, 2018
id_BM.c added end-of-file comment to every hand-written file not having some Aug 25, 2018
inline_BM.h add objislocked_BM Aug 31, 2018
jscompilnotes.md added perhaps incorrectly into _1dGYpQHFvMA_00Js64eAxrZ ... Sep 6, 2018
list_BM.c added end-of-file comment to every hand-written file not having some Aug 25, 2018
load_BM.c clear firstloader_BM in loadergcdestroy_BM; dont warn but info printf Aug 28, 2018
login_ONIONBM.thtml added do_login_redirect_onion_BM & onion_webhost_bm && websession_object Jul 20, 2018
main_BM.c start adding create_anonymous_web_session_BM Sep 12, 2018
misc_BM.cc a module whose *.so binary is stale -older that its source code- is s… Aug 11, 2018
newgui_GTKBM.c add $= <name> inside $[...] construct to bind the current object into… Sep 13, 2018
node_BM.c makeclosure can accept nil sons Aug 31, 2018
object_BM.c added end-of-file comment to every hand-written file not having some Aug 25, 2018
parser_BM.c missing reciever o_jsexpr in jsemit_expression°jsvar_object Sep 3, 2018
primes_BM.c copied better-objects branch to master Mar 29, 2018
scalar_BM.c emit_module°plain_module dont backup the emitted file, but objstrbuff… Sep 14, 2018
sequence_BM.c thingo in setelemindex_BM corrected; the first_misc_module is now com… Aug 30, 2018
store1.bmon adding scan_jsexpr°jvar_object stub Sep 15, 2018
store2.bmon extracting the module bindings in scan_jsexpr°jsvar_object Sep 17, 2018
store7.bmon ,remove command seems buggy; erase sometimes everything... Aug 8, 2018
timestamp-emit.sh add timestamp-emit.sh script Aug 10, 2018
types_BM.h improved datavect routines by using systematically DATAVECT_ASIZ_BM &… Aug 24, 2018
user_BM.c objcontributornamepayl_BM takes a const object pointer Aug 30, 2018
userlogin.md still missing write loop of password file Jun 29, 2018
web_ONIONBM.c minor improvements Sep 13, 2018
work.md copied better-objects branch to master Mar 29, 2018

README.md

Overview and philosophy

bismon (this is a temporary name) is a persistent monitor which (in the future) would interact with the GCC compiler thru plugins. It is (or will become) somehow a successor to my old GCC MELT project; the medium-term goal is also static program source code analysis with the help of GCC.

My (Basile Starynkevitch, employed at CEA, LIST in France) work on bismon is partly funded (from start of 2018 to end of 2020) by the European Union, Horizon H2020 programme, CHARIOT project, under Grant Agreement No 780075. Within CHARIOT I will focus on analysis of some kind of IoT software coded in C or C++ and (cross-) compiled by GCC on some Linux desktop.

Currently (mid-2018) bismon still in pre alpha-stage, and it is free software under GPLv3+ license. It is intended for a Linux x86-64 desktop developer's workstation (won't work on Windows or MacOSX or Android).

Approach

Develop some kind of high-level (more expressive, more declarative) domain-specific language to facilitate static program source code analysis above GCC and generate GCC plugins for it, so same motivations as GCC MELT (and more broadly related -in motivations- to Coccinnelle, Frama-C, Rascal, CrossMiner, Clang analyser, Clasp etc...).

Important features and goals of that domain-specific language and its implementation include:

  • orthogonal persistence; this is a major feature (useful for whole program static source code analysis of an analyzed program made of several translation units, each compiled independently) which was not realistically addable to GCC MELT so requires a complete redesign. The bismon process is expected to be relatively long-lived -server like- and retains most of its data from one run to the next (in store*.bmon textual files). In the future the bismon process will communicate with gcc or g++ (cross-)compilation processes (using some GCC plugin, to be generated by bismon itself)

  • dynamic typing like in Python, Scheme, Ruby, JavaScript, ... (and somehow MELT).

  • multi-threaded but without any GIL. A small thread pool of worker threads is repeatedly and concurrently fetching tasklets from a global agenda. Each tasklet runs quickly (milliseconds) in some worker thread and can update that agenda (e.g. by adding itself to be continued).

  • reflexive

  • homoiconic

  • precisely garbage collected

  • declarative, with good pattern matching and rules formalisms

  • bootstrapped, and compiled to C, perhaps some parts using GCCJIT.

  • so metaprogrammed

  • generating GCC plugins (in C++) which would interact with bismon by some RPC mechanism

Notice that the syntax and semantics (still undocumented) of bismon is work-in-progress and subject to change incompatibly.

When bismon will be completed to point of being usable, it would be typically used (thru some web interface) by a small team of developers working together on some (common) software project coded mostly in C or C++ (and preferably a free software project).

build instructions

On Linux/x86-64 (e.g. Debian/Unstable or recent Ubuntu) only (won't work on 32 bits machines!):

prerequisites

See also the dependencies subdirectory and its README-dependencies.

  • recent GCC (so GCC 7 or preferably GCC 8 ...), with C and C++ and JIT support and plugins enabled (check with gcc -v; if your system gcc and g++ lack support for plugins and libgccjit, you should build and install a recent GCC release from its source code, and configure it to provide them.). An old GCC (e.g. version 6 or earlier) cannot be used.

Most of bismon is in C99 or C11 (including an increasing amount of generated code), but some few code is in C++14.

  • ninja builder, version 1.8 or newer is need (and is started by make). Don't use an older one (check with ninja --version; if that does not work, your ninja is too old).

  • GTK 3.22 or better. This is a temporary dependency (to be deprecated), we hope to have some web interface (above libonion probably) soon.

  • libonion is needed for the web interface (at least 0.8.99 -master- later than june 10th 2018, or 0.9)

  • GNU make 4 or better

  • ccache is used to avoid useless recompilations.

  • libbacktrace - it is sometimes part of your system GCC, but you'll better install it separately (e.g. in your /usr/local/ prefix)

  • markdown is required for some documentation, such as this README.md

  • GNU indent (2.2.x at last) and astyle (3.1 at least) are useful for make indent. indent is now required for generated modules.

  • tardy (a tar file postprocessor) is used to generate documentation.

  • LaTeX -probably as texlive suite or luatex- and related utilities (pdflatex, makeidx, bibtex ...) with inkscape and hevea are needed for documentation.

relevant Debian/Ubuntu build-dependency packages

On Debian/Unstable or Ubuntu, apt-get install build-essential make ninja gcc-8 g++-8 gcc-8-plugin-dev libgccjit-8-dev ccache for the building infrastructure.

We still need apt-get install libgtk-3-dev for GTK 3. This may pull a lot of dependencies.

The libbacktrace is not independently packaged in Debian. I recommend building and installing it from its source code, or else (not recommended, 😒 but doable) edit the Makefile to uses your system GCC compiler's one.

The libonion is not independently packaged in Debian.

apt-get install markdown indent astyle tardy and apt-get install texlive texlive-full hevea

The dependencies/ subdirectory is for git submodule-s related to libbacktrace and libonion

building

Clone this git repository as usual, and build then use bismon inside the obtained source tree.

Look into the Makefile, and edit it if needed; perhaps update the PREPROFLAGS= and LIBES= lines, e.g. to use your system's GCC libbacktrace, by adding $(shell gcc-7 -print-file-name=include) into the PREPROFLAGS= line, and $(shell gcc-7 -print-file-name=libbacktrace.a) to the LIBES= line. Of course replace the 7 by 8 if appropriate. Maybe you need to explicitly set GCC= and GXX= to your specific GCC C and C++ compilers.

Run make or make -j3

The persistency mechanism is tested by make redump

The ./bismon program is (temporarily) still using GTK and libonion. We want to get rid of GTK. It is symlinked as bismongtk and bismonion.

Be sure to run bismon, bismongtk or bismonion with --help at first to understand the program options that are available.

adding yourself as a contributor

bismon (at least its Web version) needs to know about you. So you should register yourself once to it (and that would grow its persistent state). And bismon (more exactly bismonion) has a login form, so you should have a password (specific to bismon, unrelated to any other password you have).

You first need to initialize an empty password file (the default password file is ~/passwords_BM; you could specify another one on the command line). So run the command touch ~/passwords_BM in a terminal (shell) to create that empty file. Then you need to restrict its permission (readable and writable by your Unix user only) with chmod u+rw,go-rwx ~/passwords_BM. At last, check with ls -ls ~/passwords_BM that this file is empty and readable & writable only by you.

Then you'll register yourself. If Alan Turing is your name or pseudo, and alan@fake.email is your main email, with your secondary email being turing@localhost, you can add yourself to bismon by running it (actually ./bismonion) with something similar to --contributor='Alan Turing;alan@fake.email;turing@localhost' and --batch and --dump-after-load=. program options. Of course you should replace Alan Turing, alan@fake.email and turing@localhost by whatever is appropriate for you. The secondary email (also called alias) is optional. If you don't use it (so if you don't provide any alias, which is likely - so replace turing@localhost with an empty string), bismon knows no more about you than git does (see git config for user.email and user.name). You might even code some shell script to extract that personal data information from git and put it into bismon.

At last you need to set your password (it is unrelated to other passwords in your computer or on the web, but should has at least 10 Unicode characters, and digits, letters, punctuations), as known to bismonion, using the --add-passwords option (also with --batch and --dump-after-load=.). To set your initial password to mypass!123456 create some temporary text file (it contains contributor names and their updated passwords in clear text, so you should not keep it), perhaps /tmp/addpassbismon, containing for example Alan Turing:mypass!123456 add run bismonion with --add-passwords=/tmp/addpassbismon --batch then remove /tmp/addpassbismon. Of course you'll better write some shell script wrapping that, and be sure to have a strong enough password which is not in this README.md.

You might perhaps add a few other users (probably less than a dozen) that you trust and that can work with you. Every user could mess or abuse the bismon system as easily as you could. Be then sure to be compliant with the GDPR if any of your users is European (and GDPR compliance is then your responsability). The login form is templated (with code from web_ONIONBM.c) from source file login_ONIONBM.thtml (and you might improve the text inside that file if you want).

using bismon

It is not really usable yet except by me (Basile) in july 2018 (no static source code analysis yet, no generation of GCC plugins yet). If you want to see something using GTK, run ./bismongtk then type the_system in the command window labelled new-bismon, then both Ctrl and Return keys pressed together. If you want to see something thru the web interface, run ./bismonion then open its URL (perhaps localhost:8086/ or whatever has been specified with --web-base) in your web browser.

It is recommended to run make clean once in a while (e.g. daily or weekly). When generating files, bismon makes a backup of their previous version.

Ask me (Basile Starynkevitch, basile@starynkevitch.net or basile.starynkevitch@cea.fr...) for details.