Convert the PE console into a foreman plugin

[bastelfreak]

Use Case

Foreman is an established open source tool for life cycle management. It has a working open source community, different companies offer commercial support/training/consulting for it. There are also commercial downstream products like RedHat Satellite or Orcharhino. Foreman is used for many on premise setups, it has a proper CI setup and a plugin setup. It already has support to view puppet reports and facts. With HDM it has also an option to view, edit and diff Hiera data. Foreman can be used as an ENC and as hiera backend. And it has support for run remote jobs.

Foreman and the PE console overlap a lot in their feature set. I think Foreman is the better implementation because it looks more healthy. There are more docs, in general more activity and people can contribute to it. Contribute to the PE console is impossible and feature requests are mostly ignored.

I think it makes sense to replace the standalone PE console with a set of foreman plugins. Even if those are commercial/closed source people can at least contribute fixes to the rest of foreman and I assume that maintenance of the whole PE console codebase requires more time than it takes to maintain a small set of foreman plugins.

And since many setups already use Satellite/Orcharhino/Foreman they are already used to the UI and they don't need a second Web UI.

Describe the Solution You Would Like

Replace the PE Console with a set of foreman plugins to run PE Tasks/plans via foreman.

Describe Alternatives You've Considered

I would like to implement this on my own but your documentation about the orchestrator internals/bolt-server API is quite thin.

Additional Context

I pitched this idea around half a year ago at slack and I heard that different people within perforce are talking about it. I think it makes sense to have this issue here as an official request (and I would love to have the discussion about it in public, not private).

[ekohl]

I'd be very interested to know how we could make this happen.

First of all I'll lay out some crucial components we already have in place:

• foreman is the core. For historical reasons, it has some components still in there. Most notably, Puppet CA integration but also fact and report processors.
• foreman_puppet is the plugin providing the front end integration. Historically this was part of Foreman core, but has been extracted
• smart-proxy is also known as Foreman Proxy and these represent services. Each Smart Proxy has some modules and for this particular case the puppet and puppetca modules are most relevant.
  • These days the puppet module (thankfully) talks to Puppetserver's API to get lists of environments and classes, but in the distant past it parsed modules before there was a Puppetserver API at all.
  • puppetca also talks to Puppetserver's API for CA related tasks. It also contains two modes for approval: autosign.conf management (through direct files) and a callback to validate the CSR (where Foreman can embed a signed JWT).
• puppet-puppetserver_foreman contains all the puppetserver integration bits (ENC, fact upload, report processor).
• puppetdb_foreman is the PuppetDB integration. Notably, this is a 1-to-1 relation with Foreman where the Smart Proxy is a 1-to-many relationship.

As someone working full time on Foreman and generally fond of Puppet, I'd like to see this happen.

I think the big question is: which features would be missing for it to be a PE replacement.

Some thing that come to mind:

• Implement a bolt backend in Foreman's Remote Execution
• Improve PuppetDB integration in Foreman
• Completely move over PuppetCA bits from foreman core to foreman_puppet

[bastelfreak]

I made some thoughts about this in the past. I think the biggest point is to run bolt/Puppet tasks/plans via foreman.

• make bolt server useable (Provide documentation for bolt-server bolt#3285)
• write a foreman-proxy/remote-execution plugin to interact with bolt-server / pe-orchestration-services
  • should run on a puppetserver to get code and to use fileserver
• write a foreman plugin to support tasks/plans in the UI
• optionally write a bolt plugin to use choria as transport protocol (or the zmq stuff in foreman?)

Improve PuppetDB integration in Foreman

What do you have in mind? Filtering in the new view pased on PQL queries would be nice.

[ekohl]

Foreman's pull provider uses MQTT, not ZeroMQ

[bastelfreak]

oh derp, then this!

[ekohl]

Improve PuppetDB integration in Foreman

What do you have in mind? Filtering in the new view pased on PQL queries would be nice.

Today what it can do is limited and hasn't really seen a change in years. I'm sure PE is more capable

[tuxmea]

possible "PE" Components in Foreman:

• Event viewer - which types/classes/hosts have received changes (events) from which classes or types
• Node graph - was removed from PE: provide a clickable node graph which shows resource dependencies
• PQL queries - Filter hosts or create host groups based on PQL queries