Plugin and Hooks arquitechiture

[svallory]

Looks like you already have a good modularization in Comtrya. It would be nice if it supported plugins. Actually, since I don't know Rust and had some issues with using Comtrya to run the manifests I almost created a clone of it in Typescript hahaha. I could help with the plugin architecture design if you give me some pointers on the current architecture.

Here are some extension points I think could be created if you decide to add plugin support:

• Custom Actions
• Custom Contexts
• Custom Providers
• Custom top-level "things" (not sure what you would call depends and actions at the top of the yml file)

If you want to go further and turn Comtrya into a framework, you could also add extension points to:

• Manifest parser
  which would allow creating parsers for other languages. I think JSON can be useful in certain contexts
• Cli commands
  I created a script called pack_stuff that could became a plugin. It exports PPAs, repositories and user installed packages list that can then be read by the move_in.yml manifest in the new system
• Hooks for many points of the process, e.g.:
  • on_start - that would allow setting up logging
  • before_action - could allow me to define a rule that asks for confirmation before running and cancel execution
  • on_dependency_require - could enable custom dependency processing like downloading a yml from github and returning the path to the local file

Anyways, these are just examples. The point is that with plugins and hooks you don't have to implement most of the feature requests

[rawkode]

Hi @svallory,

Thanks for your great suggestion. Apologies for the huge delay in me getting back to you, but I had some personal things kicking my ass over the last few months.

I love your ideas, and we can certainly implement a few of them.

Side note: I almost implemented Comtrya in TypeScript before attempting it in Rust 🤣

Manifest Parser

100%. It's actually very simple for us to support new formats, JSON would be a very small change and I'd love to see support for CUE (but there's no Rust library atm).

CLI Commands & Plugins

This also shouldn't be too difficult due to how Comtrya works. Actions actually emit "plans" which are really small data structures. These could be serialized and sent to/from plugins, over RPC or even files.

Hooks

This is something I started dropping into Comtrya with initializers and finalizers on actions. We could definitely provide more extension points.

---

I don't know if you're still using Comtrya, but if you wanna continue this discussion and get some of these moving, I'm happy to chat. Please let me know.

Thanks again

[rawkode]

Link #161

[icepuma]

As soon I have more time I'll have a look into https://wasmer.io/ for our plugin system.

[svallory]

@icepuma I was looking at Wasmer to see if I could take a shot at implementing the plugin system but the whole process looks cumbersome to me.

I think you were on the right track with #161. At least for now, plugin executables by defining them in a manifest would go a long way. We would be able to define custom actions and even hooks written in any language.

I think we can get that very cheap by implementing a plugin parser very similar to the actions one. Let's say we can define a .comtrya-plugins.yml somewhere and if comtrya finds it, it loads that file first, before any manifest.

plugins:
  - hook: secrets
    desc: Creates a new variable in the `env` context and sets it's value
    interactive: true
    trigger: context_loaded
    # Either a command, a koto script or a shell script must be defined
    command: create-env-var-for-comtrya.sh
    script.koto: |
      io.print("Variable name:")
      name = io.stdin().read_to_string()
      io.print("Variable value:")
      value = io.stdin().read_to_string()
      comtrya.env[name] = value
    script.shell: |
      read -p "Variable name:" name
      read -p "Variable value:" value
      # the function below would be made available by comtrya
      # as part of the plugins and hooks API
      comtrya set_env $name $value

  - action: configure
    group: transcript
    description: |
      Configure transparent encryption of
      sensitive files stored in a Git repository

      Learn more: https://github.com/elasticdog/transcrypt
    command: transcrypt
    interactive: true         # not sure if comtrya needs to know this
    args:
      - arg: cipher
        desc: the symmetric cipher to utilize for encryption;
        default: aes-256-cbc
        required: false
        type: string
        formatStr:  "-c {value}"

      - arg: password
        desc: the password to derive the key from
        required: true
        type: string
        formatStr:  "-p '{value}'"

Reading the yaml above, comtrya would automatically run the hook and allow actions in manifests with the following structure:

  - action: transcrypt.configure
    cipher: aes-256-cbc
    # suppose I created the env variable ENC_PASS when the hook was called
    password: {{ env.ENC_PASS }}

[svallory]

Btw, I just found out that deno can be embedded in a Rust app. How about allowing commands to be written in Typescript/javascript?

https://deno.land/manual@v1.26.2/embedding_deno

[icepuma]

I will think a bit about the idea with deno. If you want, you can join our Discord @ https://rawkode.chat/ in the forum channel oss-projects / #comtrya and we can discuss it there with more people :)

[svallory]

@icepuma That invite is invalid :/

[rawkode]

@svallory https://rawkode.chat works on my end.

Here's another link, just in-case:

https://discord.gg/8pZsbTbj

[svallory]

@rawkode sorry, it was actually a weird issue with my Discord account. rawkode.chat works just fine

[icepuma]

Will be continued @ #263