Toml fixes

[bauhaus93]

Fixing TOML issues mentioned in #3490.

• Make error messages less technical
• Fix floats in exponential form (didn't allow leading zero after exponent, but is now allowed)
• Allow empty keynames, which are valid but discouraged
• Write info in README about which TOML standard is supported / what the current gaps are
• Preserve whitespace in front of key names / around assignments
• Proper string quoting on writing, don't assume just basic. First tries of that in TOML as default storage plugin #3478.

Additional fixes:

• Fix multiline string reading. There was a problem, when a string contained non-terminating quotation characters. Also fixed function to not loop endlessly when encountering EOF during reading.
• Check, if non-decimal conversion functions are handling overflows properly (They didn't, so added checks if a base 2/8/10/16 integer over/underflows).
• More strict handling of truth values on writing.
• Fix parser so that some invalid TOML files don't get parsed as valid. (Happens on invalid simple table/tablearray combinations or redefintions).
• Add metakey preservation capabilties.

Metakey preservation problems:

• Metakeys assigned to a table array root key. The table array root key (which holds the array and tomltype metakey of an table array) doesn't have any anchors in a TOML file.
• Metakeys assigned to inline table elements (since no comments are allowed in direct inline table elements by the TOML standard). This is a general comment problem. Either we change an existing inlinetable to a simpletable or we may throw an error in that case.

Basics

These points need to be fulfilled for every PR:

• Short descriptions of your changes are in the release notes
  (added as entry in doc/news/_preparation_next_release.md which
  contains _(my name)_)
  Please always add something to the release notes.
• Details of what you changed are in commit messages
  (first line should have module: short statement syntax)
• References to issues, e.g. close #X, are in the commit messages.
• The buildservers are happy.
• The PR is rebased with current master.

If you have any troubles fulfilling these criteria, please write
about the trouble as comment in the PR. We will help you.
But we cannot accept PRs that do not fulfill the basics.

Checklist

Check relevant points but please do not remove entries.
For docu fixes, spell checking, and similar none of these points below
need to be checked.

• I added unit tests for my code
• I fully described what my PR does in the documentation
  (not in the PR description)
• I fixed all affected documentation
• I added code comments, logging, and assertions as appropriate (see Coding Guidelines)
• I updated all meta data (e.g. README.md of plugins and METADATA.ini)
• I mentioned every code not directly written by me in THIRD-PARTY-LICENSES

Review

Reviewers will usually check the following:

• Documentation is introductory, concise, good to read and describes everything what the PR does
• Examples are well chosen and understandable
• Code is conforming to our Coding Guidelines
• APIs are conforming to our Design Guidelines
• Code is consistent to our Design Decisions

Labels

If you are already Elektra developer:

• Add the "work in progress" label if you do not want the PR to be reviewed yet.
• Add the "ready to merge" label if the basics are fulfilled and you also
  say that everything is ready to be merged.

[markus2330]

Looks very good, thank you for the improvements of the error messages. The unit tests are very nicely written, too.

[bauhaus93]

@markus2330
I've added a for now very basic implementation of metakey preservation in this PR (Metakey names assumed to have no spaces - I have to find a proper way/format to differentiate between metakey name and value in the comment string; Also, special characters/escape sequences are not yet handled).

But at least the failing example from #3491 with the base64 metamode should work now as intended (without any need of knowing that metamode is on).

[markus2330]

Thank you so much, great work 🎆

[mpranj]

@bauhaus93 thank you for implementing so many improvements to the TOML plugin. This is even more important than #3098. 😉

[kodebach]

@bauhaus93 @markus2330 Currently toml uses the tomltype metadata. I think it would be a good idea to generalize this into type/[plugin], where [plugin] is a storage plugin. Very few storage formats have a type system that exactly matches Elektra's, so storing the storage format's type separately makes sense. For example a JSON plugin would use one of these:

type/json = string
type/json = boolean
type/json = number
type/json = null

(array and object aren't needed, since the are inferred from KeySet structure and JSON doesn't have things like TOML's inline tables)
The corresponding type values would be:

type/json = string -> type = any
type/json = boolean -> type = boolean
type/json = number -> type = long_long [OR] type = double
type/json = null -> type = any

(string corresponds to any because in Elektra string means non-empty)

[markus2330]

Please avoid making new proposals in PRs, this could be easily misunderstood. Status-quo is:

• In the cases the types match (which is not soo rare) type should be used.
• In cases where types cannot be mapped, internal/<plugin>/* (in this case internal/toml/type should be used), see:

libelektra/doc/METADATA.ini

Lines 1072 to 1074 in f370cc5

	[internal/<plugin>/*]
	status= implemented
	description= Internal metadata to be ignored by other plugins.

and here also the meta-spec for type:

libelektra/doc/METADATA.ini

Lines 223 to 244 in f370cc5

	[type]
	type= enum
	short
	unsigned_short
	long
	unsigned_long
	long_long
	unsigned_long_long
	float
	double
	long_double
	char
	boolean
	octet
	any
	empty
	string
	status= implemented
	usedby/plugin= range type
	description= defines the type of the value, as specified in CORBA
	Unlike check/type, type is also used for code generation and within the APIs
	example= any

If unsupported types are encountered during kdbSet, please give the user an error.

[kodebach]

Yes, internal/toml/type would also be a possibility. But even in this case, we should probably define in METADATA.ini that internal/<plugin>/type should be used exclusively for this purpose to avoid possible confusion.

[markus2330]

Thank you, I clarified in d198a7e

[kodebach]

@markus2330 I think this PR is now in a working state. My suggestion would be to merge this and then I'll use new PR(s) to fix the remaining TOML issues (see #3910). If we do a release soon, we should wait until after the release with the merge, or remove the semi-functional comment-based metadata handling from the PR.

[markus2330]

Yes, the idea is to release asap so that Harald can continue with his work, see #3908. I think its best if we remove the "semi-functional comment-based metadata handling from the PR" as the specs in TOML is not urgent and having a otherwise working TOML is important&urgent.

[kodebach]

I think this is now ready to be merged (assuming the build servers are happy).

I removed the metadata handling for now (new handling will be added in a new PR). I did not yet copy the string quoting stuff from #3478, because I found some issues with the existing code. I will add a new version in another PR.

[mpranj]

Thank you so much @bauhaus93 and @kodebach! ❤️