New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make SpecElektra easier and safer to use #1683

Closed
e1528532 opened this Issue Oct 31, 2017 · 3 comments

Comments

Projects
None yet
2 participants
@e1528532
Contributor

e1528532 commented Oct 31, 2017

SpecElectra Integration

Some Ideas how the upcoming type system can be integrated in Elektra in various
places, following up @markus2330's suggestion to think about it in #1668. I think
an open discussion about this topic can be beneficial, so everyone can add further
thoughts to this, suggest better integration possibilities for a smoother Elektra
experience, or correct false assumptions of mine.

I think there are basically two target groups, called users and writers. I tried to categorize my integration ideas to target either one or both of these groups.

Writers are people who develop specifications for the use with Elektra.
They will probably make use of the type inference and type checking in order to
write correct specifications more efficiently (e.g. autocompletion for types).

Users use specifications to validate their configuration when mounting new
backends by specification. The type system can check if the specification is
correct and will issue a warning in cases it is not so users will already be
warned about possible upcoming issues beforehand.

This can also make use of type inference, as it would allow the writer of a
specification to actually leave away parts of the specification which would be
inferred to keep the specifications more compact. The downside of this possibility
is that users would have to include the type system in their elektra build as
otherwise the inference can't happen upon specification-mounting thus the runtime
typechecks wouldn't happen as the required metakeys are missing.

General considerations

It should be kept in mind that the typesystem should be optional and thus can be
excluded via a build variable. Furthermore we may want to allow different
implementations of the type system, so we should consider having some kind of
interface for that which defines the necessary operations and data structures.

(Users) libtools integration / specmount

One way is to integrate the type system in libtools, specifically in the smount
process. Upon issuing a specification mount, the type system can check if the
specification to mount is correct and show a warning otherwise.

To integrate this for all kinds of tools, we can add these kinds of checks to
libtools to specreader.cpp.

(Writers) kdb editor integration

An easy way to integrate the type system in a specification writer's procedure
is to integrate it into the kdb editor command. It may enable via a parameter,
or automatically if the user edits something in the spec namespace. Upon saving,
it can validate the changes and give feedback about the correctness of the
specification.

Furthermore we can add a parameter to automatically add inferred types to the
result before saving, so the result is a complete specification while the
writer can leave extra type info away during editing.

A downside is that there is not really a direct feedback possible as usually
experienced when working on a typed language with an IDE. Instead it would only
happen when the user finishes editing. To have such a direct feedback, the editor
being used has to provide this feature (e.g. for SublimeText one could write a
plugin which provides such functionality).

(Writers) libtools integration / merging

kdb editor, suggested above, uses the merging framework of libtools. So instead of
integration type system functionality in the kdb editor command, we could
implement it in the underlying merging API instead. Therefore, other tools that use
the merging APIs can also benefit.

A little downside is that kdb editor also offers custom conflict strategies which don't
make use of the merging APIs, currently the validate strategy. Those would have to be
treated separately.

(Both) global plugin

Another way is to write a global plugin that watches for changes in the spec
namespace, and if there is a change, it can run the type check procedure and
give feedback about changes. A downside this involves is that
if we treat type checking issues as "hard" errors, it would be hard to edit
a configuration partially, e.g. by manually setting appropriate metadata with the
setmeta command in the spec namespace "one by one", leaving the specification in
an incorrect state in-between the single setmeta commands, even if it'd be correct
again in the end.

This would work for both authors and readers in a general way, though its not
really tailored to their specific needs. Performance may also be an issue, as it would
react to all changes of the spec namespace and revalidate it every time, which
might be quite costly depending on the type system implementation.

(Both) Standalone tools

We can offer standalone tools (or expand the kdb tools with something like
kdb typecheck or kdb typeinfer). Those take a specification as an argument
and either checks the specification for correctness or embeds inferred types
directly in the specification. Downside is that they'd have to be used explicitly.

@e1528532 e1528532 added this to the 1.0.0 milestone Oct 31, 2017

@markus2330

This comment has been minimized.

Show comment
Hide comment
@markus2330

markus2330 Oct 31, 2017

Contributor

Thanks for these detailed thoughts. I'll have a look after the release.

Contributor

markus2330 commented Oct 31, 2017

Thanks for these detailed thoughts. I'll have a look after the release.

@markus2330

This comment has been minimized.

Show comment
Hide comment
@markus2330

markus2330 Nov 1, 2017

Contributor

These are quite nifty ideas!

I like your idea of having the specification validation as plugin best. Remarks on this idea:

  1. Your haskell plugin fits quite well this approach. We would not have the problem of any hard dependency between the type checker and tooling.

  2. It does not have to be global plugin. Users can also decide to mount it to specific placed in the spec namespaces, as they can do for other validation plugins.

  3. A downside this involves is that if we treat type checking issues as "hard" errors

    Actually, we could encode intermediate validation errors via special errors and/or warnings. This issue is not specific to specification validation but also applies to other forms of validation.

    Editors that (want) to understand intermediate validation errors, could guide users through the process based on info they got from the errors. Your work would be to define such errors, and describe a general process how tooling (or users) can make best use of the errors you emit. Maybe we even need an intermediate error state per key? A simpler protocol, however, would be most advantageous.

    More intelligent editors, like the WebUI of @omnidan, already consider the types internally. They then prepare the whole corrected key set, and already do a correct kdbSet.

Your idea can also applied on a more general level. For example, we could write plugins (maybe they even use your type system), to protect system/elektra/mountpoints from invalid mountpoints or even individual plugins from wrong plugin configuration.

@petermax2 actually had a quite similar approach with checkconf. checkconf also communicates intermediate errors (other GPG key needed) to the users.

we can add a parameter to automatically add inferred types to the
result before saving

We should not keep such behavior undecided. An important goal of your type system is to unify current ad-hoc solutions #1092. Users rely on getting deterministic results.

merging API

I like this option but I think we should opt against it. While it might be very interesting to use such advanced knowledge to find better merging solutions (@fberlakovich might even have anticipated such ideas and kept the merging strategies very modular for this very reason?) we are at a terrain with little knowledge what works well. It would be a topic quite different to your ideas discussed so far.

To move forward it might be best to start writing decisions of how the errors and plugin interface needs to be.

Contributor

markus2330 commented Nov 1, 2017

These are quite nifty ideas!

I like your idea of having the specification validation as plugin best. Remarks on this idea:

  1. Your haskell plugin fits quite well this approach. We would not have the problem of any hard dependency between the type checker and tooling.

  2. It does not have to be global plugin. Users can also decide to mount it to specific placed in the spec namespaces, as they can do for other validation plugins.

  3. A downside this involves is that if we treat type checking issues as "hard" errors

    Actually, we could encode intermediate validation errors via special errors and/or warnings. This issue is not specific to specification validation but also applies to other forms of validation.

    Editors that (want) to understand intermediate validation errors, could guide users through the process based on info they got from the errors. Your work would be to define such errors, and describe a general process how tooling (or users) can make best use of the errors you emit. Maybe we even need an intermediate error state per key? A simpler protocol, however, would be most advantageous.

    More intelligent editors, like the WebUI of @omnidan, already consider the types internally. They then prepare the whole corrected key set, and already do a correct kdbSet.

Your idea can also applied on a more general level. For example, we could write plugins (maybe they even use your type system), to protect system/elektra/mountpoints from invalid mountpoints or even individual plugins from wrong plugin configuration.

@petermax2 actually had a quite similar approach with checkconf. checkconf also communicates intermediate errors (other GPG key needed) to the users.

we can add a parameter to automatically add inferred types to the
result before saving

We should not keep such behavior undecided. An important goal of your type system is to unify current ad-hoc solutions #1092. Users rely on getting deterministic results.

merging API

I like this option but I think we should opt against it. While it might be very interesting to use such advanced knowledge to find better merging solutions (@fberlakovich might even have anticipated such ideas and kept the merging strategies very modular for this very reason?) we are at a terrain with little knowledge what works well. It would be a topic quite different to your ideas discussed so far.

To move forward it might be best to start writing decisions of how the errors and plugin interface needs to be.

@markus2330 markus2330 changed the title from SpecElektra Typesystem Integration to SpecElektra Typesystem Mar 15, 2018

@markus2330 markus2330 changed the title from SpecElektra Typesystem to Make SpecElektra easier and safer to use Apr 1, 2018

@e1528532

This comment has been minimized.

Show comment
Hide comment
@e1528532

e1528532 Jun 11, 2018

Contributor

Ultimately this is implemented as a plugin now, thus closing this issue. We still have the closed issue around in order not to loose the other ideas should we decide to also use one of them at some point.

Contributor

e1528532 commented Jun 11, 2018

Ultimately this is implemented as a plugin now, thus closing this issue. We still have the closed issue around in order not to loose the other ideas should we decide to also use one of them at some point.

@e1528532 e1528532 closed this Jun 11, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment