xconfig implements a C++ API together with utility programs for parsing configuration files.
xconfig configurations integrates closely with a UNIX type environment. Access to environment variables as well as access to output from UNIX commands is transparenly reflected in the configuration language. A C++ API together with a command line program are used for extracting data from configuration files.
Eventhough xconfig is a first cut implementation of an idea that needs many improvements it is useable as it stands right now. Several updates and extensions are planned. For example, a Python API mirroring the C++ API will be implemented. The configuration language will also be updated to support various features.
Type or copy/paste the following into test1.cfg:
namespace system{
user = $USER # user from environment variable
pwd = `pwd` # execute bash command - current work directory
userathost = @"%{user}@`hostname`" # string interpolation - concatenate variable with `hostname`
}Print configuration using xconfig:
xconfig test1.cfgThe output is:
system_pwd="/home/user/hansewetz"
system_user="hansewetz"
system_userathost="hansewetz@dumbo"A short explanation:
- environment variables are accessed using 'dollar' notation
- bash commands can be executed using 'backquote' notation
- configuration variables are accessed using 'percentage' notation or directly by the name
- environment variables, configuration variables and bash commands can be evaluated using string interpolation by prefixing the string with an 'at' sign (@)
- names can be scoped using namespaces
The following example shows how a configuration file can be read using the C++ API:
#include "xconfig/XConfig.h"
#include <iostream>
using namespace std;
using namespace xconfig;
int main(){
// read and evaluate configuration file and print variables
XConfig xfg("test1.cfg");
for(auto&&[name,value]:xfg())cout<<name<<": "<<value<<endl;
}To compile/link the program do:
g++ -o test1 test1.cc -m64 -fPIC -Wall -pedantic -Werror -std=c++2a -Wno-deprecated -Wno-register -D_GLIBCXX_USE_CXX11_ABI=0 -I<xconfig-install-path>/include -L<xconfig-install-path>/lib -lxconfiglWhen running the program we get this output:
system.pwd: /home/user/hansewetz
system.user: hansewetz
system.userathost: hansewetz@dumboThese instructions will get you a copy of the project up and running on your local machine for development and testing purposes.
What things you need to install the software:
- gcc supporting C++17 (tested with gcc 9.0 but should work with gcc 8.X)
- cmake version 3.12.2 or higher
- libboost_program_options.so 1.57 or higher
Installing gcc 9.x
If you don't have gcc 8 or 9 installed, you can pull the latest of trunc and compile it:
$>export INSTALLHOME=<your-installation-path>
$>
$># ------- install isl
$># download isl-isl-0.19
$>mkdir build && cd build
$>.././configure --prefix=${INSTALLHOME}
$>make && make check && make install
$>
$># ------- install gmp
$># download gmp-6.1
$>mkdir build && cd build
$>.././configure --prefix=${INSTALLHOME}
$>make && make check && make install
$>
$># ------- install mpfr
$># download mpfr-4.0.1
$>mkdir build && cd build
$>.././configure --prefix=${INSTALLHOME} --with-gmp=${INSTALLHOME}
$>make && make check && make install
$>
$># ------- install mpc
$># download mpc-1.1.0
$>mkdir build && cd build
$>.././configure --prefix=${INSTALLHOME} --with-gmp=${INSTALLHOME}
$>make && make check && make install
$>
$># ------- install gcc latest
$>mkdir gcc-trunk && cd gcc-trunk
$>svn checkout svn://gcc.gnu.org/svn/gcc/trunk .
$>mkdir build && cd build
$>././configure --prefix=${INSTALLHOME} --with-isl-include=${INSTALLHOME}/include --with-isl-lib=${INSTALLHOME}/lib --disable-multilib --with-gmp=${INSTALLHOME} --with-mpfr=${INSTALLHOME} --with-mpc=${INSTALLHOME}
make && make check && make installInstalling libboost_program_options.so.1.58.0
If you don't have libboost_program_options installed, please follow these instructions:
$># download boost v1.58 or higher
$>export INSTALLHOME=<your-installation-path>
$> ./bootstrap.sh --prefix=${INSTALLHOME} --with-libraries=program_options
$>./b2
$>./b2 installInstalling cmake-3.12
If you don't have cmake version 3.12 or higher installed, please follow these instructions:
$># download cmake-3.12.2 or higher
$>export INSTALLHOME=<your-installation-path>
$>mkdir build && cd build
$>cmake -DCMAKE_INSTALL_PREFIX=${INSTALLHOME} ..
$>make && make installStep by step recipie for compiling and installing xconfig (assuming gcc 9.x, cmake 3.12, libboost_program_options.so are installed):
$>mkdir xconfig && cd xconfig
$>git clone https://github.com/hansewetz/xconfig.git .
$>mkdir build && cd build
$>export CXX=<full path to g++ compiler> # /usr/bin/g++
$>export CC=<full path to gcc compiler> # /usr/bin/gcc
$>cmake .. -DCMAKE_INSTALL_PREFIX=<install-path> ..
$>make && make installThe installation directory is populated with:
.
├── ...
├── <install-path>
│ ├── bin # xconfig binaries
│ ├── lib # xconfig libraries
│ ├── include # xconfig C++ header files
│ ├── examples # sample programs
└── ...The version of the xconfig API is located in the xconfig/version.h header file. The version is specified by two int C++ variables:
constexpr const static int XCONFIG_VERSION_MAJORconstexpr const static int XCONFIG_VERSION_MINOR
You can print the version of the current installation by executing: xconfig --version.
The language follows a C like style where variable names consists of alphanumeric characters including the underscore character. Variables can be assigned string or integer values:
user = "hansewetz" # 'user' is a string
maxthread = 20 # 'maxthreads' is an integerEnvironment variables are accessed using a bash style notation:
user = ${USER} # 'user' is assigned the value of environment variable 'USER'
user1 = $USER # works without braces alsoA configuration variable is accessed by directly using the variable name or by prefixing the name with '%':
user = @"${USER}" # 'user' is assigned the value of environment variable 'USER'
userAtMachine= @"%{user}@`hostname -i`" # access configuration variable 'user' inside a stringCommands can be executed by enclosing the the command in back-quotes:
machine = `hostname` # get name of machineThe plus operator concatenate strings and adds integers:
user = "hans"+"ewetz" # user = "hansewetz"
minthreads = 10
maxthreads = minthreads + 8 # maxthreads = 18An expression on the right hand side of the assignment operator can access a variable using dot separated namespaces:
namespace system{
user = $USER # user = "hansewetz"
machine = `hostname -i` # machine = "10.0.0.4"
}
userAtMachine = @"%{system.user}@%{system.machine}" # userAtMachine = "hansewetz@10.0.0.4"As the previous example showed strings can be interpolated. During interpolation environment variables, normal variables as well as commands are evaluated. A variation of the previous example is shown here:
namespace system{
user = @"$USER" # hansewetz
machine = `hostname -i` # machine = "10.0.0.4"
}
userAtMachine = @"%{system.user}@%{system.machine}" # userAtMachine = "hansewetz@10.0.0.4"There are some restrictions on how variables can be used. A variable cannot be assigned to once it has a value. A namespace qualified variable cannot be used on the left hand side of the assignment operator.
Not yet done - for right now, please see:
The process of evaluating a configuration file is relatively straight forward:
- bison/flex is used for parsing the configuration
- during parsing a program is written for a virtual machine
- the program is executed by the virtual machine
- an extractor is used for retrieving information from the virtual machine
An overview of the design is shown here:
The virtual machine is implemented as a simple stack machine tailored specifically for this project. The name of the virtual machine is MMVM - Mickey Mouse Virtual Machine. The MMVM currently supports 13 opcodes. Among them are simple operation such as 'push value on stack' or 'store value in memory'. More complex operations such as 'evaluate a command in a shell and store output on stack' are also supported.
One might wonder if it is not overkill to implement a VM just for parsing a configuration file. As it turns out, having a VM available makes it almost effortless to support and experiment with new features in the configuration language. Since the MMVM implementation is only around 330 lines of C++ code it can be implemented in only a few hours.
The MMVM currently supports strings and integers. The support is not extensive - two integers can be added, two strings can be concatenated and strings and integers can be added.
Evaluation of a configuration file is done in two steps:
- compile the file into a program(generate a program consisting of op codes for MMVM)
- execute program (sequence of opcodes
Namespaces are used to scope variables. The full name of a variable is a series of dot separated namespaces followed by the name of the variable. It is not necessary to use the full name of a variable. When namespaces are left out, the compiler looks for the variable in the enclosing scope(s) . For example:
# test1.cfg
a = 18
namespace ns{
namespace ns1{
b = 19
namespace ns2{
c = b # 'b' is found one step up
d = ns1.b # same as previous
e = a # 'a' is found in outer most scope
}
}
}
The output from executing: xconfig test1.cfg:
a="18" ns_ns1_b="19" ns_ns1_ns2_c="19" ns_ns1_ns2_d="19" ns_ns1_ns2_e="18"
When interpolating strings I would like to be able to handle namespaces the same way as they are handled when referencing variables outside string interpolation. In order to reference variables without specifying the full namespace path, we need a table that contains information about the location of a variables within namespaces. This is done by building the table at compilation time and using it at runtime during string interpolation.
The following is planned:
- add git commit SHA to 'version.h' file (via cmake)
- implement Python based API
- extend language to support repeated groups of structurally similar blocks of definitions
- better/more documentation
- possibly - port to lower version of gcc (i.e. C++11)
This project is licensed under the MIT License - see the LICENSE.md file for details
- author: Hans Ewetz
- email: hansewetz@hotmail.com
- blog: https://hansewetz.blogspot.com