Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Beginners Questions #3472

Closed
poldi871 opened this issue Aug 31, 2020 · 17 comments
Closed

Beginners Questions #3472

poldi871 opened this issue Aug 31, 2020 · 17 comments

Comments

@poldi871
Copy link

Hello,

I am evaluating libelektra for an embedded system under BuildRoot. And as noted in the FAQ of libelektra "ask before you search too long", I'll do that now ;-)
In advance, after reading a lot (but not yet all) of the documentation and viewed the youtube videos, it seems to be a great work, even a kind of a developer's dream, if it will work for me.

Here are my problems / questions:

  • My evaluation system is a Leap-15.2. Unfortunately there are only versions of 0.8.20 out there. I've searched for SRPM's of newer versions, because I would not use an old version for my system. I have some "hen egg problem" here. On the one hand my build under BuildRoot was successful at the first look, but at runtime there are still problems (warnings C01200), so the build recipe will need some more efforts. On the other hand I don't want to invest too much, in case it turns out that libelektra is not suitable after all. I've also looked through "libelektra4 aus Projekt home:bekun:devel", but there are also only old versions. Is there any RPM or SRPM for the 0.9.2 version?

  • By the way BuildRoot:
    I found a recipe for OpenWrt. Does a recipe exist for BuildRoot already?

  • On a first simple test, I have an entry in a mounted "ini"-file like this:
    Parameter = "[10,20]; [11,35]"
    Reading the value with "kdb get" results in: [10,20]; [11,35]
    Writing that value back to the configuration file results in: [10,20]; [11,35] (without the quotes)
    It is not possible to quote this entry with \" or "". If I do a double quoting, double quotes are written:
    kdb set /users/Parameter ""[10,20]; [11,35]""
    writes: Parameter = ""[10,20]; [11,35]"" to the file.

  • On a second test I was not able to process structured "ini"-files.
    Example:
    [Services]
    [Services:html]
    port = 80
    [Services:ssh]
    port = 22
    On a "kdb ls /user/example" I get:
    ...
    /user/example/Services/[Services:html]
    but I don't get the "port" key.
    If I do a "kdb get /user/example/Services/[Services:html]" I get a "duplicated key entry in INI file"
    What's wrong?

Thanks in advance.

Poldi

@markus2330
Copy link
Contributor

markus2330 commented Sep 1, 2020

Thank you for writing! Always good to hear when someone starts using Elektra 💖

May I ask you where you intend to use Elektra?

"ask before you search too long"

🚀

even a kind of a developer's dream

Very nice to hear 🎆

Is there any RPM or SRPM for the 0.9.2 version?

This is more @darddan and @mpranj's domain. We have spec files (e.g. #2519) which worked across different versions and our CI system also uses Fedora. I am not sure if our CI produces (S)RPMs, though.

I found a recipe for OpenWrt.

Yes, this is very well maintained by @haraldg.

Does a recipe exist for BuildRoot already?

I do not know.

@darddan can you look into BuildRoot?

What's wrong?

The INI plugin is a bit quirky. @bauhaus93 develops a TOML plugin #3292 which is superior in many ways and will replace the INI plugin for most use-cases. @bauhaus93 can you maybe answer the question? Does the TOML plugin makes this quoting/structure shown above as wanted?

@bauhaus93
Copy link
Contributor

bauhaus93 commented Sep 1, 2020

@poldi871 @markus2330
I just checked it with the toml plugin, and it seems to produce the correct quotings and structure. Did I translate the second example correctly into toml?

I created the following script for checking. Command output is annotated with #>.
(The kdb file command is used to get the file path of a certain mount point)

#!/bin/sh

# Mount file
sudo kdb mount example.toml user/example toml type

# Write first example to file
echo 'Parameter = "[10,20]; [11,35]"' > `kdb file user/example`

kdb get 'user/example/Parameter'
#> [10,20]; [11,35]

kdb set 'user/example/Parameter' '[10,20]; [11,35]'
#> Set string to "[10,20]; [11,35]"

cat `kdb file user/example`
#> Parameter = "[10,20]; [11,35]"

# Write second example to file
echo -e '[Services.html]\nport = 80\n[Services.ssh]\nport = 22' > `kdb file user/example`

cat `kdb file user/example`
#> [Services.html]
#> port = 80
#> [Services.ssh]
#> port = 22

kdb ls user/example
#> user/tests/storage/Services/html
#> user/tests/storage/Services/html/port
#> user/tests/storage/Services/ssh
#> user/tests/storage/Services/ssh/port

# Get port values
kdb get user/example/Services/html/port
#> 80
kdb get user/example/Services/ssh/port
#> 22

# Cleanup
kdb rm -r user/example
sudo kdb umount user/example

@poldi871
Copy link
Author

poldi871 commented Sep 1, 2020

Hello,

thanks for the quick and good answers!

  • "Thank you for writing! Always good to hear when someone starts using Elektra sparkling_heart":
    I'm making embedded SW for decades now, and it annoyed me all this time to have no configuration at a certain abstraction level. I had a very similar idea (build a registry of keys and values) a long time ago when I was chief developer of a little company, but never got the man power to realize it. At that time the majority of decision-makers decided to implement a proprietary webmin based system. Some times later I became a free developer and as a "one man band" didn't have even less resources for such a task ;-)
    Now and then I've looked for configuration frameworks which are usable for embedded systems, but obviously I never stumbled upon this project. The search always resulted in projects which were (in my mind) too proprietary, or too specialized, also preventing an ability to choose a practical migration strategy.
    Elektra seems to meet all of these requirements and I wonder why it hasn't become a standard tool yet.

  • "May I ask you where you intend to use Elektra?":
    I'm about to implement a next generation of an embedded gateway application for a customer now using "systemd" as the init system built with BuildRoot. The base design of that application reaches back until the late 1990s (strong arm SA110), and evolved over some generations up to now running now on an x86 based system.

  • RPM and SRPM:
    I took a quick look at it and it seems to be helpful for evaluation. But meanwhile my focus is already on the BuildRoot recipe.

  • Recipe for BuildRoot:
    I had a closer look on the OpenWrt recipe. I think the information there is enough to generate a BuildRoot recipe from it. At the moment there exists no recipe in BuildRoot. I asked, because sometimes you can find BuildRoot recipes outside of BuildRoot itself.

  • "ini" plugin:
    Thanks I will try the suggestions.

Poldi

Addendum:
the recipe for BuildRoot works.
Not very "nice" at the moment, but a first step:

################################################################################

libelektra

################################################################################

LIBELEKTRA_VERSION = 0.9.2
LIBELEKTRA_SITE_METHOD = file
LIBELEKTRA_SITE = $(BR2_EXTERNAL)/package/TestBed/libelektra
LIBELEKTRA_INSTALL_STAGING = YES
LIBELEKTRA_CONF_OPTS =
-DTARGET_PLUGIN_FOLDER=""
-DBUILD_FULL=OFF
-DBUILD_STATIC=OFF
-DBUILD_DOCUMENTATION=OFF
-DFORCE_IN_SOURCE_BUILD=ON
-DBUILD_TESTING=OFF
-DKDB_DEFAULT_RESOLVER=resolver_fm_pb_b
-DKDB_DEFAULT_STORAGE=ini
-DENABLE_OPTIMIZATIONS=OFF
-DPLUGINS="ALL;-multifile"
-DICONV_FIND_REQUIRED=ON

LIBELEKTRA_SUPPORTS_IN_SOURCE_BUILD = NO

$(eval $(cmake-package))

@mpranj
Copy link
Member

mpranj commented Sep 1, 2020

Thank you so much for your interest and for sharing your BuildRoot recipe with us.

Is there any RPM or SRPM for the 0.9.2 version?

Unfortunately we don't have any official ones now, but they are planned. We do offer current DEBs and packages for macOS (via homebrew).

The INI plugin is a bit quirky.

I also agree. If your scenario allows it, wait for us to publish the TOML plugin, which is superior and almost ready. If your project timeline does not allow it, then use what is available.

If you have any further questions, feedback and bugreports (or even success stories) we are happy to hear from you.

@poldi871
Copy link
Author

poldi871 commented Sep 2, 2020

@poldi871 @markus2330
I just checked it with the toml plugin, and it seems to produce the correct quotings and structure. Did I translate the second example correctly into toml?

I created the following script for checking. Command output is annotated with #>.
(The kdb file command is used to get the file path of a certain mount point)

#!/bin/sh

# Mount file
sudo kdb mount example.toml user/example toml type

# Write first example to file
echo 'Parameter = "[10,20]; [11,35]"' > `kdb file user/example`

kdb get 'user/example/Parameter'
#> [10,20]; [11,35]

kdb set 'user/example/Parameter' '[10,20]; [11,35]'
#> Set string to "[10,20]; [11,35]"

cat `kdb file user/example`
#> Parameter = "[10,20]; [11,35]"

# Write second example to file
echo -e '[Services.html]\nport = 80\n[Services.ssh]\nport = 22' > `kdb file user/example`

cat `kdb file user/example`
#> [Services.html]
#> port = 80
#> [Services.ssh]
#> port = 22

kdb ls user/example
#> user/tests/storage/Services/html
#> user/tests/storage/Services/html/port
#> user/tests/storage/Services/ssh
#> user/tests/storage/Services/ssh/port

# Get port values
kdb get user/example/Services/html/port
#> 80
kdb get user/example/Services/ssh/port
#> 22

# Cleanup
kdb rm -r user/example
sudo kdb umount user/example
  • sorry, another foolish question:
    How to get the TOML plugin working?
    With 0.9.2 i get:
    kdb mount OtaModemOperatorDaemon.conf user/OtaModemOperatorDaemon toml type The command kdb mount terminated unsuccessfully with the info: Was not able to load such a plugin!

I downloaded the git master branch of today. The build shows the plugins in- and excluded. "toml" is mentioned nowhere.
A file search 'toml' through the source tree only shows some references within the rust binding.

"On stupid" I included "toml" in "-DPLUGINS" -> no chance ;-)

... beg your pardon, I still did not rtfm completely up to now -> should do it, but at the moment it is too exciting to experiment with libelekta ;-))

@mpranj
Copy link
Member

mpranj commented Sep 2, 2020

You can not test it yet comfortably, because it is not released and not even merged into master.

The development branch can be seen in #3292, but we do not recommend basing any other work on top of development branches.

@poldi871
Copy link
Author

poldi871 commented Sep 2, 2020

I checked out "bauhaus93, libelektra-66cbf47" and found the toml plugin there ;-)

  • Build: successful

  • Tested 'Parameter = "[10,20]; [11,35]"':
    successful.

  • Tested "structured ini file":
    the second example was not exactly reproduced by marcus2330, but it worked nevertheless.

Example-File:

[Services] 
    [Services.html] 
            port = 80 
    [Services.ssh] 
            port = 22

root@Vd:~/.config# kdb ls user/example/Services
user/example/Services
user/example/Services/html
user/example/Services/html/port
user/example/Services/ssh
user/example/Services/ssh/port

  • Comments:
    Comments in the file remain intact, that's fine!

  • But there was a little "inelegance"
    Setting a value destroys the formatting of the file (indents get lost).
    Command:
    "kdb set user/example/Services/html/port 81"

` And now the file looks like:

[Services]
[Services.html]
port = 81
[Services.ssh]
port = 22

`

Next beginner' problem - export to xml:
I used:

[Services]
[Services.html]
# This is a test comment on Services.html
port = 81
[Services.ssh]
# This is a test comment on Services.ssh
port = 22

The command was: "kdb export user xmltool >example.xml"

The resulting file was:

<?xml version="1.0" encoding="UTF-8"?>
<!-- Generated by Elektra API. Total of 5 keys. -->
<keyset xmlns="https://www.libelektra.org"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="https://www.libelektra.org elektra.xsd"
>
<key name="user/example/Services">

     <comment><![CDATA[]]></comment>
</key>

<key name="user/example/Services/html">

     <comment><![CDATA[]]></comment>
</key>

<key name="user/example/Services/html/port"
	value="81">

     <comment><![CDATA[]]></comment>
</key>

<key name="user/example/Services/ssh">

     <comment><![CDATA[]]></comment>
</key>

<key name="user/example/Services/ssh/port"
	value="22">

     <comment><![CDATA[]]></comment>
</key>

</keyset>

--> CDATA's are empty.

Background:
At the moment the plan is to create a xml-file at the center and file it to the gateway. On the gateway side there may be some additional processing and the generation of the original config files for those apps, which were not "electrified" from the beginning or are not "electrifiable" at that time. Therefore it would be great to have the comments intact.

By the way:
The manual on "https://doc.libelektra.org/api/current/html/doc_help_kdb-export_md.html" says in the "examples" section:

To view a keyset stored in user/keyset in the XML format:
kdb export user/keyset xmltool

But within the example itself the following is used:

kdb set user/tests/kdb-export/one one
kdb set user/tests/kdb-export/two two
 
kdb export user/tests/kdb-export/ xml
                                  ^^^

-> my be a typo, because trying that form leads to an error message (no such plugin).

  • once again concerning the toml plugin:
    is there a plan when it will be merged? It's no problem for me to start working on the "hidden" bauhaus93-release, but it should be available when the other side in the computing center will start testing. I think this will be in a few weeks.

Thanks for your patience!

Poldi
ps: what a damn editor. Writing this text took more time than the tests ;-)

@markus2330
Copy link
Contributor

markus2330 commented Sep 2, 2020

Setting a value destroys the formatting of the file (indents get lost).

I agree that this would be a nice-to-have for TOML as the files otherwise easily get hard-to-read.

says in the "examples" section

Thank you for pointing this out. This is not a good example, as xmltool is only legacy (to import Elektra 0.7 configs). Tip: With kdb plugin-info xmltool (shows what https://www.libelektra.org/plugins/xmltool contains) you would have seen that the plugin is discouraged.

my be a typo, because trying that form leads to an error message (no such plugin).

It should use the xerces plugin (see also below in that man page), maybe you did not include it, as it needs the xerces XML parser. See https://www.libelektra.org/plugins/xerces

is there a plan when it will be merged? It's no problem for me to start working on the "hidden" bauhaus93-release, but it should be available when the other side in the computing center will start testing. I think this will be in a few weeks.

This should be easily doable. Can you sponsor development to speed things up?

This was referenced Sep 2, 2020
@poldi871
Copy link
Author

poldi871 commented Sep 2, 2020

Thank you for pointing this out. This is not a good example, as xmltool is only legacy (to import Elektra 0.7 configs). Tip: With kdb plugin-info xmltool (shows what https://www.libelektra.org/plugins/xmltool contains) you would have seen that the plugin is discouraged.

my be a typo, because trying that form leads to an error message (no such plugin).

It should use the xerces plugin (see also below in that man page), maybe you did not include it, as it needs the xerces XML parser. See https://www.libelektra.org/plugins/xerces

Thanks, will try it tomorrow. Hope that xerces is buildable in BuildRoot.

is there a plan when it will be merged? It's no problem for me to start working on the "hidden" bauhaus93-release, but it should be available when the other side in the computing center will start testing. I think this will be in a few weeks.

This should be easily doable. Can you sponsor development to speed things up?

Unfortunately not. As I already mentioned, I'm a "one man band". I'm spending my time on my own risk at the moment and this will continue until the result of my evaluation will end in a positive decision. And the risk for me is not only spending my time, but also get an unjustifiable delay in my own project. Even in the positive case I have to prepare a realizable proposal to convince the customer. From the view of the customer, this will be only a very little part of the project and I think he will not even pay for (yes, I'm a technician, not a sales man). I see my part in helping to make the project a little better known by trying to integrate libelektra into my application because i am convinced of this approach. And hand on heart, after all the resources that are already in this project, there are still problems with ini files in the officially available release? I think the reality is, that most evaluators would have stopped evaluation at that point. And I'm still a long way from the end of my tests...

@markus2330
Copy link
Contributor

I'm spending my time on my own risk

💖

Even in the positive case I have to prepare a realizable proposal to convince the customer.

Can you also write here what your goals and requirements are? Maybe we can tell you if and how much implementation effort will be needed and what we might be able to do.

I see my part in helping to make the project a little better known

🚀

And hand on heart, after all the resources that are already in this project, there are still problems with ini files in the officially available release?

Yes, this was also very surprising for me that this is such a hard problem. In the beginning we were hoping that there would already be some library that would simply read&write configuration files and to integrate this library to Elektra. But we did not find such a library, they always fail in either:

  • round-tripping (not any KeySet can be represented)
  • preserving what the user wrote (comments, spaces, order)
  • giving a data structure describing the configuration and not the syntax of the file

The INI plugin was the first attempt to solve all three requirements but it became so complex that it is unmaintainable now. That is why @bauhaus93 wrote the TOML plugin using better parser technology.

I think the reality is, that most evaluators would have stopped evaluation at that point.

We have quite a lot of customers in the embedded systems world (e.g. Broadcom, Toshiba, Kapsch, ...), as in embedded systems human-editable configuration files are usually not a big topic. There are plenty of plugins in Elektra that reliably serialize and deserialize any KeySet with whatsoever weird configurations, e.g. dump, quickdump or mmapstorage. Alternatively, the ni plugin writes anything reliably to and from INI files but does not care about preserving what the user did. But I agree with you, what the admin usually wants is what the toml plugin does: provide all three properties at the same time in a best-effort. But afaik Elektra is the only software that does this at the moment at all, others do not even try properly, e.g. hyperrealm/libconfig#35 or jbeder/yaml-cpp#154.

@poldi871
Copy link
Author

poldi871 commented Sep 3, 2020

Can you also write here what your goals and requirements are? Maybe we can tell you if and how much implementation effort will be needed and what we might be able to do.

Background:
At the moment a gateway is configured by a proprietary system, which was adapted very often in the history of this long running project. And all adaptions had to be downward compatible. You may imagine the number of exceptions the system has to deal with meanwhile.
Driven by the eol of some tools and packets in the current version the decision was made to build a "next generation" based on "systemd" and "DBUS". Because the last and current versions are/were running nearly "bug free" for a long period there is enough time (even for a "one man band") to generate that next generation. The introduction of "systemd" and "DBUS" leads to a quite different system construct. As a consequence the idea was born to take the chance and also clean up the configuration handling and introduce the base for further migration. At least this task alone would have been worth to generate a project by itself. But that's life ;-)

Back to your question:
At the moment I can't give you a serious answer. But thanks for your offer. There are a bunch of standard configuration files (host, hosts, ipsec,...) which are generated at the center and sent to a special directory on the device. The device knows that files and has links established from the standard location to the file in the special directory. There are also a bunch of configuration files necessary for the customer apps. These apps are configured by simple ini files at the moment and would be the candidates for "electrifying" them immediately or during a migration step. The possibility of migration is necessary because not only the device is affected by a change but also at least the whole organization on the center side. In the past we have made too good experience with that strategy, even at next generation steps, and although the current generation is absolutely different from former generations from an implementation point of view.
At the moment my plan is to generate a xml file on the center side and send it to the device for further processing. At first glance an unnecessary intermediate processing, because the base structure of the configuration is quite the same for all devices, but there are exceptions caused by network infrastructure and other reasons that even cause the need to generate quite different files. At the moment this processing is done in the center with a big effort, although the device could handle those exceptions much more transparent and therefore much more effective. For the non exception cases the device would simply load the xml file into the virtual configuration filesystem of libelektra and write that parts to the configuration files of the "non electrified" applications.

The INI plugin was the first attempt to solve all three requirements but it became so complex that it is unmaintainable now. That is why @bauhaus93 wrote the TOML plugin using better parser technology.

I agree concerning the processing of ini files. Last week I studied a lot of competitors, even whole distributions and all have their problems.

We have quite a lot of customers in the embedded systems world (e.g. Broadcom, Toshiba, Kapsch, ...), as in embedded systems human-editable configuration files are usually not a big topic.

Nice to hear that (customers)
human-readable configuration files: in our case this is a big topic.

@markus2330
Copy link
Contributor

markus2330 commented Sep 3, 2020

DBUS

We spent quite some time into DBUS integration https://www.libelektra.org/plugins/dbus and are currently improving it for KDE&GNOME elektrifikation.

At the moment my plan is to generate a xml file on the center side and send it to the device for further processing.

Such CM tasks are very well supported by Elektra, you can use Elektra for both parsing the XML and for changing the system configuration. Basically you would need to implement something similar to curl | kdb import. For the importing you could either simply overwrite configuration (if you do not care about changes done on that device) or by using a 3-way merge https://www.libelektra.org/tutorials/merge-configuration, which was reimplemented in C by @Chemin1. We recommend to use this new implementation, even though it is currently not yet used in kdb import itself, see #3131.

For the non exception cases the device would simply load the xml file into the virtual configuration filesystem of libelektra and write that parts to the configuration files of the "non electrified" applications.

Yes, Elektra could help in both situations: for the non-elektrified applications you would mount their configuration files. If you can you are better of by patching them. Usually elektrifying applications is not so hard because often there is some internal API which you can reimplement using Elektra. If you try to change these APIs, however, things get very time-consuming, e.g. lcdproc/lcdproc#146 by @kodebach 🚀

human-readable configuration files: in our case this is a big topic.

Good to hear that our efforts for human-read/writeable config files are appreciated 💖

@poldi871
Copy link
Author

poldi871 commented Sep 4, 2020

DBUS

We spent quite some time into DBUS integration https://www.libelektra.org/plugins/dbus and are currently improving it for KDE&GNOME elektrifikation.

At the moment my plan is to generate a xml file on the center side and send it to the device for further processing.

Such CM tasks are very well supported by Elektra, you can use Elektra for both parsing the XML and for changing the system configuration. Basically you would need to implement something similar to curl | kdb import. For the importing you could either simply overwrite configuration (if you do not care about changes done on that device) or by using a 3-way merge https://www.libelektra.org/tutorials/merge-configuration, which was reimplemented in C by @Chemin1. We recommend to use this new implementation, even though it is currently not yet used in kdb import itself, see #3131.

For the non exception cases the device would simply load the xml file into the virtual configuration filesystem of libelektra and write that parts to the configuration files of the "non electrified" applications.

Yes, Elektra could help in both situations: for the non-elektrified applications you would mount their configuration files. If you can you are better of by patching them. Usually elektrifying applications is not so hard because often there is some internal API which you can reimplement using Elektra. If you try to change these APIs, however, things get very time-consuming, e.g. lcdproc/lcdproc#146 by @kodebach rocket

human-readable configuration files: in our case this is a big topic.

Good to hear that our efforts for human-read/writeable config files are appreciated sparkling_heart

Thanks for the hints.
First I want to get xerces running. Couldn't finish it yesterday. Xerces is available as BuildRoot package, but there seem to be some problems building the libelektra plugin. I just saw some message with "#include_next <stdlib.h>" on the first sight, but could not deal with it, because there was a phone conference outstanding which lasted until midnight. Don't worry about this, i think I simply will have to make a new build from scratch, because there was some craft work in the last build.

@poldi871
Copy link
Author

poldi871 commented Sep 6, 2020

After a few day and night shifts, here are more experiences of a beginner ;-)

"#include_next" problem:
It wasn't as easy as it looked at first glance. Seems it's a problem of GNU gcc "-isystem"-setting.
I admit, up to that moment I didn't even know the "#include_next" directive ;-)
There seem to be many ways to circumvent it on embedded systems, but also seem to be very side effect associated.
My solution was to patch "ElektraCompiling.cmake" with the following line to force the build also to search the include paths flagged with "-I" for the C++ system includes without causing system wide side effects:

if (CMAKE_COMPILER_IS_GNUCXX)
	execute_process (COMMAND ${CMAKE_C_COMPILER} -dumpversion OUTPUT_VARIABLE GCC_VERSION)
	if (WIN32)
    ...
	endif (WIN32)
	set (CMAKE_INCLUDE_SYSTEM_FLAG_CXX "")   <=====
endif (CMAKE_COMPILER_IS_GNUCXX)

Not very elegant either, but now the build was successfull.

Now, at runtime I've got:

root@Vd:/usr/lib/gconv# kdb plugin-list 
Could not load a transcoding service

That was a problem for me (language settings and copying gconv libs from the toolchain).

Now, basic "kdb" commands were successfull, but now I couldn't load an ini file with the "toml" plugin.
There were no error messages but "kdb ls user/example" listed no entries anymore.

I made a lot of "build from scratch" (BfS) removing every change step by step until "toml" worked again.
I didn't investigate it deeper, but it seems that it has something to do with the "xerces" plugin or with gconv libs (wrote it on my checklist).

I didn't investigate further, because I noticed issues that I hadn't noticed before. But I wasn't sure if I just overlooked it or if there were still hidden bugs in my build (multiple entries for the plugins).

root@Vd:~# kdb plugin-list 
… 
glob 
globbing 
globbing 
globbing 
…
highlevel 
highlevel 
highlevel 
...
ini 
invoke 
invoke 
invoke 
io 
io 
io 
…
mathcheck 
merge 
merge 
merge 
...
opts 
opts 
opts 
passwd 
path 
pluginprocess 
pluginprocess 
pluginprocess 
...
unit 
utility 
utility 
utility 
validation 
wresolver 
xerces 
xmltool 
yamlsmith 
root@Vd:~# 

To get rid of any sideeffects of my own build I decided to setup two virtual machines with OpenWrt, because these versions should be very well maintained according to marcus2330.
I have decided to set up two versions, because the current, official release still contains libelktra in version 0.8.21. The second version was a snapshot from today with libelektra-0.9.2.
I've installed all modules with "luci". This should make sure that the dependencies are ok.

Finally, I still wanted to familiarize myself with the xerces plugin ;-))

To make another long story short this problem is also existing in the OpenWrt versions:

root@OpenWrt:/etc# kdb plugin-list 
...
glob 
globbing 
globbing 
globbing 
hexcode 
hexnumber 
hidden 
highlevel 
highlevel 
highlevel 
...
root@OpenWrt:/etc#

Getting more info about such plugins leads to:

root@OpenWrt:/etc# kdb plugin-info merge 
Module does not seem to be loaded. 
Now in fallback code. Will directly load config from plugin. 
The command kdb plugin-info terminated unsuccessfully with the info: 
Was not able to load such a plugin! 

Maybe you misspelled it, there is no such plugin or the loader has problems. 
You might want to try to set LD_LIBRARY_PATH, use kdb-full or kdb-static. 
Errors/Warnings during loading were: 
1 Warning was issued: 
       Sorry, module kdb issued the warning C01100: 
       Resource: Dlsym failed. Could not get pointer to factory for module: libelektra-merge.so. Reason: Symbol not found: elektraPluginSymb
ol 
       Mountpoint:  
       Configfile:  
       At: elektra-0.9.2/src/libs/loader/dl.c:100 

Please report the issue at https://issues.libelektra.org/ 
root@OpenWrt:/etc#

I don't know if it is not allowed to call up info about such a plugin, but the plugin names appear in the output of "kdb plugin-list".

... still trying to explore the xerces plugin with a primitive "ini" file, because "toml" is not available:

Both version's output:

root@OpenWrt:~# cat example.conf 
[default]
Port = 80

root@OpenWrt:~# kdb mount example.conf user/example ini
 Sorry, 1 warning was issued ;(
        Sorry, module kdb issued the warning C01200:
        Installation: Dlopen failed. Could not load module libelektra-spec.so. Reason: Error loading shared library libelektra-spec.so: No such file or directory

Finally, not even "kdb --version" seems to work on the snapshot version:

root@OpenWrt:~# kdb --version
terminate called after throwing an instance of 'kdb::KDBException'
  what():   Sorry, 1 warning was issued ;(
        Sorry, module kdb issued the warning C01200:
        Installation: Dlopen failed. Could not load module libelektra-spec.so. Reason: Error loading shared library libelektra-spec.so: No such file or directory
        Mountpoint: system/elektra/version
        Configfile: 
        At: elektra-0.9.2/src/libs/loader/dl.c:90


Sorry, I crashed by the signal SIGABRT
This should not have happened!

Please report the issue at https://issues.libelektra.org/
Aborted
root@OpenWrt:~#

I admit that I'm getting a little tired now. I am still fascinated and convinced by the idea, but I now think I understand why libelektra has not become a standard tool until now.
... or I'm just too stupid to use it ... ;-))

@markus2330
Copy link
Contributor

First of all: You can ask questions much earlier, you do not need to try every possibility before asking a question 😉

Second: While embedded systems are an important target for Elektra, it is very unusual (or you are the first?) who tries to develop, or even learn Elektra on an embedded system. I recommend you to install Elektra on a desktop distribution (or use docker) and to experiment/develop there.

E.g. kdb plugin-info are like man pages. I would not even expect them to be present on an embedded system. kdb plugin-list ditto, it basically makes an ls libelektra-* in the folder of modules. We did not not implement it for the case where all libraries are in the same directory (which is only the case for embedded systems). E.g. merge is not a plugin but a library, so it should not be in the output of kdb plugin-list and cannot work with plugin-info.

Could not load module libelektra-spec.so

Did you install the package which contains the spec plugin? See above, better to use some desktop distribution where it is much easier to get a clean installation (but also easier to get a messed-up installation 😉) and where you can run all tests to verify which parts of the installation work as intended (kdb run_all).

I now think I understand why libelektra has not become a standard tool until now.

I think the version number 0.9.2 indicates quite well where we currently are, we are at about 92% for the ability of applications to use Elektra. So there are definitely some areas and situations, where development or at least bug fixes are needed before you can be 100% satisfied 🥇

So better ask here first before you make elaborate testing of things where it is quite clear for us that this part is not there yet.

@markus2330
Copy link
Contributor

I talked with @mpranj and it is not a good idea to use the xerces plugin for your use case (sending/receiving XML documents). It is better if you do the XML processing and construction of the KeySet directly in your application. The problem is that xerces does not do escaping in the meta-data but the comments need escaping (your requirement was that you want to send comments in the XML document). Elektra would then only be used for what it is designed to be: to get/set local configuration.

@poldi871
Copy link
Author

poldi871 commented Sep 7, 2020

I talked with @mpranj and it is not a good idea to use the xerces plugin for your use case (sending/receiving XML documents). It is better if you do the XML processing and construction of the KeySet directly in your application. The problem is that xerces does not do escaping in the meta-data but the comments need escaping (your requirement was that you want to send comments in the XML document). Elektra would then only be used for what it is designed to be: to get/set local configuration.

Thanks very much for the info. Today and eventually tomorrow I have to make a break in matters "libelektra". Then I will reconsider the situation.

This was referenced Sep 12, 2020
@kodebach kodebach closed this as completed Mar 3, 2021
@ElektraInitiative ElektraInitiative locked and limited conversation to collaborators Mar 3, 2021

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants