It is conceptually related to GCC MELT, a Lispy like domain specific language to customize the GCC compiler, implemented mostly by a GPLv3 meta-plugin (the MELT plugin for GCC) that I also developed and that is FSF copyrighted, like GCC is. Read GCC MELT sheet (two pages) for more (and, this French page (single page, in French)....)
The MELT system
The MELT system (still partially a dream) refers to a mix of this MELT monitor and GCC MELT customized compilations (including some future MELT extensions for GCC not yet developed; these extensions will communicate with this MELT monitor using some textual ad-hoc protocol on Unix sockets). The long-time goal of the MELT system is to provide a software development environment and static analysis tool for code compiled by GCC (putative free-software equivalent of non-sound static code analysis tools like Coverity, ParaSoft, etc...) and be able to help exploring and understanding and navigating source code of many software components at once (in our wildest dreams, all the free software packaged in a Debian distribution and compiled with GCC, and some large proprietary software provided by funding partners).
In my vision, a user community (e.g. a team of dozens of developers
working on some mid-sized software project compiled by
GCC...) would compile its software collection
with a MELT enhanced GCC (this should be as
simple as adding
CFLAGS += -fplugin=melt -fplugin-melt-arg-mode=super in their
Makefile or other build
machinery; of course that might slow down a bit the compilation but
won't change the habits of the developers...) and use the MELT
system (notably the web interface provided by the MELT monitor) to
query, understand, analyze, navigate, and even discuss about the
development of that software.
You can use the gcc-melt GoogleGroup to discuss technical matters related to the MELT system (so both MELT plugin and MELT monitor)
Goals of the MELT monitor
The goals of the MELT monitor includes notably
provide a web interface (HTML5 compliant, for recent Firefox) to the MELT system (it took years for me to admit that people want a web interface, or at least a GUI one, and that most software developers -notably those working in industry- don't use the command line anymore, like I am used to do....). And with a Web interface, even non-Linux users could use it (although the MELT system targets only Linux)... That interface would also be useful to browse (but not edit!) a large collection of user source code (compiled by a MELT enhanced
gcc), and most of GCC internal representations (in its middle-end) of that user code (perhaps even explaining some of the GCC compiler's decisions, e.g. why that function was inlined here and not there...).
Provide a web interface to edit some abstract syntax tree (ASTs) of some future DSL (e.g. formalizing coding rules), to search into MELT system & GCC compiled software, and give him the ability to document things in a Wiki way. We don't want the user to type some DSL syntax (like the Lispy MELT DSL inside our MELT plugin) as text, we want to assist him in constructing ASTs; Old 1980s MENTOR system from INRIA (or some Smalltalk systems) is inspirational. Notice that the MELT monitor is a specialized Web server with very few simultaneous users (usually one, in the future a small team working on the same software project, e.g. a dozen of users).
provide a persistent machinery to the MELT system, that is a long-term store usable by MELT extensions to keep, enhance, and provide static analysis information. In other words, we need a cluster of MELT monitors which checkpoints periodically its state and which we can later restart smoothly from that persistent state: conceptually the MELT system should run during an entire software project using it (like your version control system does), because it is part of the entreprise know-how. Hence, dynamic software updating of the MELT system is a must.
provide a high-level declarative language framework for static analysis, above the MELT plugin, and for the MELT system ; we want some high-level rule based language (e.g. to express coding rules) ...
be able to take advantage of multi-core systems by using several threads and work in parallel
if possible, take advantage and be able to run on a small cluster or cloud (of e.g. a few dozens of Linux servers), so MELT monitors should be able to run concurrently and communicate ...
Not all of them are implemented in 2015. My roadmap includes, like I did in the MELT plugin, development of bootstrapped specific languages with a metaprogramming (and metaknowledge, inspired by J.Pitrat's work and vision approach). The MELT monitor will have some DSL and more and more of its code will become self-generated.
(I am advancing very slowly; things are difficult; this needs a Linux system to be run and used)
The following software components are needed (or are very likely to be
needed later, I am listing -for reference- also some packages that are
very likely to be useful in the future and that I have looked
into). When packaged in Debian(/Sid), I just name the Debian
package. See the
a recent GCC compiler (so
gcc-5Debian package), in particular because even when running both MELT plugin and MELT monitor are generating code.
the GCCJIT library, so
as for the MELT plugin, all the source dependencies of the latest GCC (so
aptitude build-dep gcc-5)
the MELT plugin itself
Boehm's garbage collector thru
libgc-devDebian package; I am aware of GC techniques, but coding a multi-threaded GC is too much work (I did spend months debugging the single-thread GC in the MELT plugin, and before that I coded Qish and some proprietary GC inside Fluctuat so I consider myself experimented in garbage collection techniques.). Ravenbrook Memory Pool System might be considered as an alternative, at least after bootstrapping is completed.
The Glib from GTK is a little bit used. so
sqlite might be used (for persistency), so
codemirror is embedded under
webroot/, to nicely show some C code in a web page.
redis might be used (for persistency), so
PostGreSQL might be used (for persistency), so probably
0mq might and probably will be used for messaging purposes, so probably
libonion is an HTTP server library that is absolutely required, but it is not packaged in Debian. So compile its latest source from David Moreno
oniongithub; I did contribute some small patches to it (sometimes, when I need a patch not yet accepted in
libonionby David Moreno, you'll need to compile my github fork of onion...). I need at least the version 0.8 (may 2016) of
I am using some HTML5 things (e.g. JQuery, etc...) but I am copying them under my
webroot/subdirectory, which also contains some of my own code in my own source files (or generated files) there.
This list only things that are (at least partly) implemented.
A node value has an item connective and a sequence of son values (a bit like Prolog terms). It might have some fixed metadata (a metaitem and a metarank), which is ignored for comparison, equality, hashing. It is immutable. Nodes are also having a closure role (the connective giving the code, the sons being the closed values).
Items are externally identified by unique item names (internally, they are identified by their address). An item name has a radix (C-identifier like, but double or initial or terminal underscores are forbidden) and an optional suffix preceded with two underscores. For example,
commentis an item name (without suffix). And
web_state__0dmo5BBWK0mbtvis an item name of radix
A persistent machinery, so the entire state of the MELT monitor, including its agenda, can be checkpointed and restarted on disk (persisted preferably in textual form for MELT monitor data, friendly to
gitversion control, and for user data such as static analysis information, in REDIS store or PostGreSQL databases). That code should also be generated but it is not yet.
An agenda mechanism:
the_agendapredefined item contains as its payload a queue of tasklet items. A fixed number (typically 3 to 10) of working threads are repeatedly fetching some tasklet from the agenda and running it. Of course tasklets are added (either in front, or at the end) to
the_agenda's queue, either by running tasklets or by external events (such as some HTTP requests, an external process completing, etc...).
A very incomplete web interface. I'm struggling learning more HTML5 techniques, and most of my recent questions on StackOverflow are related to this. I'm understanding more and more that metaprogramming techniques are practically essential. HOP is inspirational.
Here is a description of the source files that I have wrote. Other files available on github are required, but are either copied from elsewhere or generated. Some generated files are mentioned, however. In the long term, most (ideally, all!) source files should be generated, but I have not yet bootstrapped my MELT monitor (like I mostly did for the MELT plugin), but it is a major subgoal (notably because that would ease adding new data structures in the MELT monitor).
meltmoni.his the only (hand-written) header; it is including many other system or third-party libraries header, and some
_mom*.hgenerated headers (in particular, the generated
_mom_predef.his listing predefined items).
mi19937ar.cis a near-copy of some MIT licensed Mersenne Twister pseudo-random number generator by Makoto Matsumoto and Takuji Nishimura; I modified some function names by prefixing them with
momrandand making them more thread-friendly.
agenda.cis the agenda mechanism
primes.ccontain many primes, and
mom_prime_below (int64_t)functions. Prime numbers are e.g. useful as hash table sizes, etc.
webroot/canbedit.htmlis a (probably aborted) tentative of using HTML5 canvases for web interface.
global.mom-and perhaps other
*.momfiles, is the persistent state in textual form, parsed with some stack machine algorithms.
hashed.cimplements various hash tables, notably hashed sets, hashed associations, hashed maps, ... payloads.
libonionto implement the web interface. Files (notably some
webroot/are statically served. Dynamic content (and AJAX) is related to the
item.chas item related code.
main.cis the main program and utilities.
filebuf.cis for file & string buffers
webroot/microedit.jsis a micro editor (HTML5 contenteditable!) to be able to edit values and use them. I am not happy with it, so I coded again the ....
webroot/microedit.jsis a nano editor (no contenteditable)
modules/should contain generated C code.
_mom_predef.his generated and contains the predefined items.
state.chas the persistence machinery (loading and saving state notably in
global.momtextual file). We'll need to persist to some REDIS store or some PostgreSQL database as soon as we'll be running in a distributed context (cloud or cluster).
value.chandles immutable values
It should be stressed that generating most of the above code is much simpler than hand-coding it.
Install all the required dependencies, then run
make -j to build (in
the source directory containing the
Makefile and this
Run the MELT monitor inside the source & build directory (the one
It is alpha-stage software (and that is the third time I'm coding
some MELT monitor). You won't be able to run it usefully in 2015. But
if you want to help (e.g. to answer some questions I am asking on some
forums) or see what is running, build it, then run
to get some command line help. Later, try something like
./monimelt -Drun,web -W localhost.localdomain:8086/ -J 3 in the terminal and use your
browser on URLs like
be able to type some C code and run it, or
http://localhost.localdomain:8086/miniedit.html to try the nano
editor (very alpha, not working yet).
To simply dump the state and test the persistency machinery, try
./monimelt -d (perhaps also
-Dload,dump). Modified files are
backed up (e.g. old
global.mom is backed up as
Killing the monitor with
SIGTERM might trigger a dump (of the
persistent state). Killing it with
This is currently an informal research work. Please help me to find funding for it (e.g. collaborative R&D projects where this work could fit). Otherwise, I might be asked to stop working on this. I could be interested in having my MELT system working on your proprietary software source code (with some funding available), as long as all my own (Basile Starynkevitch's) work remains free software, GPLv3 compatible. Notice that GCC runtime license strongly and legally discourage compiling proprietary code with proprietary GCC extensions.
Alternatively (but unlikely, because it is so specific), contribute some code to this. But since eventually this will be integrated into MELT -as part of GCC- (with a copyright transfered to FSF) you should follow the Contribute to GCC in particular its demanding legal prerequisites.
(the mailbox I read more often) for technical and scientific
discussions or questions.
(the mailbox I read at office hours only) for funding or business
related discussions: collaborative R&D consortiums, industrial
I (Basile Starynkevitch) am French, living near Paris, working at CEA, LIST, the IT part (nearly a thousand persons) of CEA (a 15000 persons, 4.4 billion euros/year budget, French government owned applied research agency, see Commissariat a l'Energie Atomique et aux energies alternatives wikipage). FWIW, the Frama-C, Papyrus, Unisim, Gatel (and of course this MELT monitor and the MELT plugin...) tools are developed in the same DILS department (100+ persons) where I work.