Skip to content
persistent monitor (for static source code analysis, GCC based)
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
attic
dependencies
doc
drafts
gccplugins
misc
modubin
modules
webroot
.gdbinit
.gitattributes
.gitignore
.gitmodules
.indent.pro
BM_makeconst.cc
COPYING3
LICENSE
Makefile
README.md
_bm_global.h
_bm_predef.h
_bm_types.h
agenda_BM.c
allocgc_BM.c
assoc_BM.c
bismon.css
bismon.h
bismon.ui
bismongtk
bismonion
bm_delim.h
bootsyntax.md
build-bismon-doc.sh
build-bismon-persistent-module.sh
build-bismon-temporary-module.sh
cmacros_BM.h
code_BM.c
contributors_BM
dump_BM.c
emitcode_BM.c
engine_BM.c
fundecl_BM.h
gencode_BM.c
generate-ninja-build.scm
generate-ninja-builder.sh
globals_BM.h
gui_GTKBM.c
guicode_BM.c
id_BM.c
id_BM.h
inline_BM.h
jscompilnotes.md
jsonjansson_BM.c
list_BM.c
load_BM.c
login_ONIONBM.thtml
main_BM.c
misc_BM.cc
newgui_GTKBM.c
node_BM.c
object_BM.c
parser_BM.c
primes_BM.c
scalar_BM.c
sequence_BM.c
store1.bmon
store2.bmon
store7.bmon
timestamp-emit.sh
types_BM.h
user_BM.c
userlogin.md
web_ONIONBM.c
webinterface.md
work.md

README.md

Overview and philosophy

bismon (this is a temporary name) is a persistent monitor which (in the future) would interact with the GCC compiler thru plugins. It is (or will become) somehow a successor to my old GCC MELT project; the medium-term goal is also static program source code analysis with the help of GCC. The source code repository (GPLv3+ licensed) is on https://github.com/bstarynk/bismon .. My (Basile Starynkevitch, employed at CEA, LIST in France) work on bismon is partly funded (from start of 2018 to end of 2020) by the European Union, Horizon H2020 programme, CHARIOT project, under Grant Agreement No 780075. Within CHARIOT I will focus on analysis of some kind of IoT software coded in C or C++ and (cross-) compiled by GCC on some Linux desktop. In 2019 - 2021 the development of bismon is also partly funded by the DECODER H2020 project, under Grant Agreement 824231 (related to its Persistent Knowledge Monitor WP1).

Currently (start of 2019) bismon still in pre alpha-stage, and it is free software under GPLv3+ license. It is intended for a Linux x86-64 desktop developer's workstation (won't work on Windows or MacOSX or Android).

Some still incomplete documentation (as a technical report doc/bismon-chariot-doc.pdf in PDF) can be generated (with make then make doc; that command requires you to have a writable $HOME/tmp/ directory). An early draft of it might be available on http://starynkevitch.net/Basile/bismon-chariot-doc.pdf

Potential future usages

While Bismon is currently funded by H2020 projects with static-analysis goals, please notice that Bismon might later -when more mature, after 2024- be probably usable as a somehow generic framework, with a lot of additional work (years, not months), for many other purposes (the most significant limit being your imagination, and the power of your computer), including perhaps even some kind of AI, sophisticated desktop-like graphical environments, many interactive web-based collaborative software tools concurrently used by some team, or object databases applications or relevant problems. Since Bismon is GPLv3+ licensed, any future software using it should, IMHO, practically also be GPLv3+ (but other licensing schemes could be possible, contact me on basile.starynkevitch@cea.fr for more; however, IANAL ...)

Approach

Develop some kind of high-level (more expressive, more declarative) domain-specific language to facilitate static program source code analysis above GCC and generate GCC plugins for it, so same motivations as GCC MELT (and more broadly related -in motivations- to Coccinnelle, Frama-C, Rascal, CrossMiner, Clang analyser, Clasp etc...).

Important features and goals of that domain-specific language and its implementation include:

  • orthogonal persistence; this is a major feature (useful for whole program static source code analysis of an analyzed program made of several translation units, each compiled independently) which was not realistically addable to GCC MELT so requires a complete redesign. The bismon process is expected to be relatively long-lived -server like- and retains most of its data from one run to the next (in store*.bmon textual files). In the future the bismon process will communicate with gcc or g++ (cross-)compilation processes (using some GCC plugin, to be generated by bismon itself)

  • dynamic typing like in Python, Scheme, Ruby, JavaScript, ... (and somehow MELT).

  • multi-threaded but without any GIL. A small thread pool of worker threads is repeatedly and concurrently fetching tasklets from a global agenda. Each tasklet runs quickly (milliseconds) in some worker thread and can update that agenda (e.g. by adding itself to be continued).

  • reflexive

  • homoiconic

  • precisely garbage collected

  • declarative, with good pattern matching and rules formalisms

  • bootstrapped, and compiled to C, perhaps some parts using GCCJIT.

  • so metaprogrammed

  • generating GCC plugins (in C++) which would interact with bismon by some RPC mechanism

Notice that the syntax and semantics (still undocumented) of bismon is work-in-progress and subject to change incompatibly.

When bismon will be completed to point of being usable, it would be typically used (thru some web interface) by a small team of developers working together on some (common) software project coded mostly in C or C++ (and preferably a free software project).

build instructions

Most of bismon is in C99 or C11 (including an increasing amount of generated C code under modules/), but some few hand-written code is in C++14 (notably misc_BM.cc and the BM_makeconst.cc metaprogram generating C headers).

On Linux/x86-64 (e.g. Debian/Unstable or recent Ubuntu) only (Without additional patches or work, Bismon won't work on 32 bits machines!) :

prerequisites

See also the dependencies subdirectory and its README-dependencies.

  • recent GCC (so GCC 7 or preferably GCC 8 ...), with C and C++ and JIT support and plugins enabled (check with gcc -v; if your system gcc and g++ lack support for plugins and libgccjit, you should build and install a recent GCC release from its source code, and configure it to provide them.). An old GCC (e.g. version 6 or earlier) cannot be used.

  • the documentation needs a /usr/bin/mipsel-linux-gnu-gcc-8 cross-compiler provided by gcc-8-mipsel-linux-gnu Debian package. Beware, that package conflicts with gcc-multilib* packages on Debian.

  • ninja builder, version 1.8 or newer is needed (and is started by make). Don't use an older one (check with ninja --version; if that does not work, your ninja is too old).

  • a recent Guile interpreter is required, version 2 or better (preferably Guile 2.2). I don't know and strongly dislike Python and the Guile interpreter is GNU, LGPLv3+, and much more familiar to me. Also, concepts from Lisp dialects such as Scheme (the language implemented by Guile) are strongly relevant to understand Bismon, where Guile is used to generate the build.ninja script used by the ninja builder.

  • GTK 3.22 or better. This is a temporary dependency (to be deprecated), we hope to have some web interface (above libonion probably) soon.

  • Glib 2.58 or better (which is also needed to GTK above), because some low level routines, notably UTF-8 related, are needed (even when GTK won't be used anymore).

  • Jansson 2.12 or better is needed for JSON (used in relation to WebSockets).

  • libonion is needed for the web interface (later than end of march 2019).

  • GNU make 4 or better

  • ccache is used to avoid useless recompilations. If you don't have it, you'll need to edit our Makefile for compilation of generated modules.

  • libbacktrace - it is sometimes part of your system GCC, but you'll better install it separately (e.g. in your /usr/local/ prefix)

  • markdown is required for some documentation, such as this README.md

  • GNU indent (2.2.x at last) and astyle (3.1 at least) are useful for make indent. indent is now required for generated modules.

  • tardy (a tar file postprocessor) is used to generate documentation.

  • LaTeX -probably as texlive suite or luatex- and related utilities (pdflatex, makeidx, bibtex ...) with inkscape and hevea and xindy are needed for documentation.

relevant Debian/Ubuntu build-dependency packages

On Debian/Unstable or recent Ubuntu, apt-get install build-essential make ninja-build gcc-8 g++-8 gcc-8-plugin-dev libgccjit-8-dev ccache gcc-8-mipsel-linux-gnu inkscape hevea texlive-full xindy fonts-inconsolata ttf-ubuntu-font-family fonts-ubuntu ttfautohint guile-2.2 for the building infrastructure; notice that gcc-8-mipsel-linux-gnu is the MIPSel GCC cross-compiler - it is conflicting with gcc-multilib* packages; inkscape, hevea, texlive-full (a huge meta-package), xindy, fonts-inconsolata, fonts-ubuntu, ttfautohint are needed for generating the documentation.

We still need apt-get install libgtk-3-dev for GTK 3. This may pull a lot of dependencies.

And apt-get install libglib2.0-dev libjansson-dev for our Glib and Jansson required dependencies.

The libbacktrace is not independently packaged in Debian. I recommend building and installing it from its source code, or else (not recommended, 😒 but doable) edit the Makefile to uses your system GCC compiler's one.

The libonion is not independently packaged in Debian.

apt-get install markdown indent astyle tardy and apt-get install texlive texlive-full hevea

The dependencies/ subdirectory is for git submodule-s related to libbacktrace and libonion

You may need to add .mjs as a valid suffix or file extension for application/javascript MIME type in your /etc/mime.types system file. Read about Javascript modules on the web and see Debian bug#927300.

Localization

Run locale to be sure that your localization is POSIX compliant or English (British or American) in UTF-8 (it cannot for instance be Italian or Greek or French). So it should be POSIX or C or en_US.UTF-8 or en_GB.UTF-8. You might want to set your environment variables appropriately (at least LANG=en_US.UTF-8 and LANGUAGE=en_US:en and LC_ALL=en_US.UTF-8, with no other LC_* environment variable). See locale(1), locale(7), utf-8(7), locale(5), setlocale(3). But don't ask me to understand more these very subtle internationalizaion and localization details, that topic is very complex. If you still need that work to be done by me, send me an email at work to basile.starynkevitch@cea.fr. But don't expect it to be done by me before 2024.

Additional configuration

The .mjs file extension is for JavaScript modules. If your /etc/mime.types file (used by libonion) does not know about it, you need to add there the following lines (the first line below, with an hash #, is a comment; what is important is the line mentioning mjs).

# in your /etc/mime.types for JavaScript modules
application/javascript                                 mjs

The documentation generation needs a writable $HOME/tmp/ directory.

extra git configuration

You might want (adapt this suggestion and define your custom hunk-header) to improve the interactive output of git diff on store*.bmon persistent data files. Perhaps something inspired from git config diff.bismon.xfuncname '"^«_[0-9A-Za-z_]*$"' might work, but not really.

building

Clone this git repository as usual, and build then use bismon inside the obtained source tree.

Look into the Makefile, and edit it if needed; perhaps update the PREPROFLAGS= and LIBES= lines, e.g. to use your system's GCC libbacktrace, by adding $(shell gcc-7 -print-file-name=include) into the PREPROFLAGS= line, and $(shell gcc-7 -print-file-name=libbacktrace.a) to the LIBES= line. Of course replace the 7 by 8 if appropriate. Maybe you need to explicitly set GCC= and GXX= to your specific GCC C and C++ compilers.

Run make or make -j3

The persistency mechanism is tested by make redump

The ./bismon program is (temporarily) still using GTK and libonion. We want to get rid of GTK. It is symlinked as bismongtk and bismonion.

Be sure to run bismon, bismongtk or bismonion with --help at first to understand the program options that are available.

adding yourself as a contributor

bismon (at least its Web version) needs to know about you. So you should register yourself once to it (and that would grow its persistent state). And bismon (more exactly bismonion) has a login form, so you should have a password (specific to bismon, unrelated to any other password you have).

You first need to initialize an empty password file (the default password file is ~/passwords_BM; you could specify another one on the command line). So run the command touch ~/passwords_BM in a terminal (shell) to create that empty file. Then you need to restrict its permission (readable and writable by your Unix user only) with chmod u+rw,go-rwx ~/passwords_BM. At last, check with ls -ls ~/passwords_BM that this file is empty and readable & writable only by you.

Then you'll register yourself. If Alan Turing is your name or pseudo, and alan@fake.email is your main email, with your secondary email being turing@localhost, you can add yourself to bismon by running it with something similar to --contributor='Alan Turing;alan@fake.email;turing@localhost' and --batch and --dump-after-load=. program options. Of course you should replace Alan Turing, alan@fake.email and turing@localhost by whatever is appropriate for you. The secondary email (also called alias) is optional. If you don't use it (so if you don't provide any alias, which is likely - so replace turing@localhost with an empty string), bismon knows no more about you than git does (see git config for user.email and user.name). You might even code some shell script to extract that personal data information from git and put it into bismon.

At last you need to set your password (it is unrelated to other passwords in your computer or on the web, but should has at least 10 Unicode characters, and digits, letters, punctuations), as known to bismon, using the --add-passwords option (also with --batch and --dump-after-load=.). To set your initial password to mypass!123456 create some temporary text file (it contains contributor names and their updated passwords in clear text, so you should not keep it), perhaps /tmp/addpassbismon, containing for example Alan Turing:mypass!123456 add run bismonion with --add-passwords=/tmp/addpassbismon --batch then remove /tmp/addpassbismon. Of course you'll better write some shell script wrapping that, and be sure to have a strong enough password which is not in this README.md.

You might perhaps add a few other users (probably less than a dozen) that you trust and that can work with you. Every user could mess or abuse the bismon system as easily as you could. Be then sure to be compliant with the GDPR if any of your users is European (and GDPR compliance is then your responsability). The login form is templated (with code from web_ONIONBM.c) from source file login_ONIONBM.thtml (and you might improve the text inside that file if you want).

using bismon

It is not really usable yet except by me (Basile) in july 2018 (no static source code analysis yet, no generation of GCC plugins yet). If you want to see something using GTK, run ./bismongtk then type the_system in the command window labelled new-bismon, then both Ctrl and Return keys pressed together. If you want to see something thru the web interface, run ./bismonion then open its URL (perhaps localhost:8086/ or whatever has been specified with --web-base) in your web browser.

It is recommended to run make clean once in a while (e.g. daily or weekly). When generating files, bismon makes a backup of their previous version.

Ask me (Basile Starynkevitch, basile@starynkevitch.net or basile.starynkevitch@cea.fr...) for details.

Caveat

This git repository also contains, for convenience, the jqueryui 1.12 framework (which has a different MIT license) under webroot/jquery-ui

N.B.

This is unreleased free software

The license (GPLv3+) is likely to not change (it is contractually specified in the CHARIOT grant). The copyright notice is probably wrong: in generated files, it is not yet conforming to GPL expectations. These details are pending bugs, but there are much more other bugs too.

P.S.

The author is fully aware -in mid 2019- of several bugs in this bismon system - a research project -, but correcting them is not a priority (since continuing the progressive bootstrap of Bismon is much more important, and is not yet achieved). The GTK interface is buggy by design and crashes often (that won't be corrected), and should be as quickly as possible replaced by a more usable web interface - and that is work in progress in mid 2019. Please contact by email the author to get possible workarounds. Notice that such temporary bugs known to the author are not registered as Bismon gitlab issues, since this software is yet unreleased.

You can’t perform that action at this time.