Allow mapping cloud-init syntax to custom stages

[davidcassany]

This is a feature I have been wondering about for a while. Would be nice having some sort of mechanism to allow mapping cloud-init syntax to specific stages instead of being stuck to boot and initramfs.

One option would be adding an extra field like yip_stage or something similar, we could eventually write something like:

#cloud-config
write_files:
  - content: |
      created once connected to the network
run_cmds:
  - echo "I am online" 
yip_stage: network

The limitation of this is that it would not be possible to set multiple stages in a single file.

Other more sophisticated (and probably a bit convoluted) approach could be providing a map directly:

#cloud-config
write_files:
  - content: |
      created once connected inside the initramfs env
run_cmds:
  - echo "I am online" 
yip_stages:
  run_cmds: network
  write_files: initramfs

I was curious to know what you think about these approaches @mudler @Itxaka
Probably there are better options I did not think about

[Itxaka]

Ah I see, like the pure cloud-init syntax.

I dont know to be honest, If you write cloud-init syntax you kind of expect to be used like the original cloud-init does no? Just at boot/initramfs and thats it.

Once you get into the need for stages, then its much nicer to use yip syntax directly so you control all the stages and such...

IMHO, cloud-init syntax is onyl there to make a nice bridge when you come into yip and have an ease of entry into writing yip stages, but its not really useful, when you can do the whole thing much better with more fine grained control under yip.

Is there an use case for that?

Not that Im against it, just trying to understand how this would fit yip properly :)

[davidcassany]

Indeed you have great points

Is there an use case for that?

Not really, is that I'd like to think how could we manage yip to actually turn it into a some sort drop-in replacement without having to code and compromise the integration within yip itself. Also to showcase the flexibility and potential of yip.

Not that Im against it, just trying to understand how this would fit yip properly :)

Yes me too, I don't see it clear that's why I asked for feedback as I was not sure too. I also see your points and I think I have now way better understanding of the use case I'd like to cover, but I still doubt, on how this could be implemented.

I agree with you the mapped stages for cloud-init within the cloud-init file itself sucks, this is a bad approach. This should not be in the hands of the cloud-init file writer. This should be in the hands of the system integrator (whoever set a system to honor certain yip stages).

So I think we should distinguish this two users:

1. An integrator that builds an appliance or OS and wants to use yip to manage cloud-init style actions
   For instance, I'd love this user has the flexibility to control when run_cmds and/or write_files are executed.
2. A sysadmin of a system that is managed in a cloud-native way where yip is used under the hood.
   For those users when and how run_cmds or write_files actions are executed should be given and have no control over it.

One thing I love from yip is the fact that in simply collects actions in to stages, what those stages mean and how and when they are executed is not yip's business. I think this is really good as it provides a lot of flexibility to anyone wanting to integrate yip into a system (1st type of users).

It is in that context that I dislike the idea of grouping cloud-init actions into potentially unrelated stages. From an integrator point of view quite some flexibility is lost in that context, the integrator is no longer in control of the stages.

One idea that comes to my mind is simply mapping each cloud-init item to a cloud-init devoted stage, imagine something like mapping run_cmds to a run_cmds_stage stage and so on. This way the integrator could actually decide when and how to execute them. Then assuming the integrator had some predefined stages such as boot, network etc. it is in its hands to configure, for instance, to run boot and run_cmds_stage stages together.