Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invoker extensibility #1093

Open
scothis opened this Issue Feb 5, 2019 · 6 comments

Comments

Projects
None yet
3 participants
@scothis
Copy link
Member

commented Feb 5, 2019

In the past riff had an Invoker CRD that allowed users to create riff functions with a new language. With the transition to buildpacks, we scaled back the number of languages we support and lost the extensibility to add new languages.

There are three key components to adding a new language

With an appropriate buildpack and invoker, adding a new language/runtime support to riff should be as easy as it was previously. Before exposing a model to users, we should experiment with making all current languages and invokers pluggable. Eliminating, or isolating, imperative logic and coupling between invokers.

@scothis scothis self-assigned this Feb 12, 2019

scothis added a commit to scothis/libfnbuildpack that referenced this issue Feb 12, 2019

Make invokers more pluggable
In the future, we want to allow for new function invokers to easily be
added to the riff buildpack. In order to make it easier to add (or
remove) invokers in a consistent way, we're normalizing the interaction
between the detect and build commands with the invokers. There is no
longer and language or runtime specific logic in the mainline code.

There is still further work needed in order to support dynamic loading
of invokers.

This PR does (there is no change in functionality):
- introduces a new types for RiffBuildpackContribution and
  RiffInvoker, and applies them to each language
- moves all language and runtime specifc code into that language's
  package
- moves non-main packages under ./pkg
- moves invoker packages under ./pkg/invoker
- moves build logic into build.go within the invoker package

Refs projectriff/riff#1093
@scothis

This comment has been minimized.

Copy link
Member Author

commented Feb 12, 2019

Our current thinking is that we will create a buildpack per language that is responsible for contributing the invoker and managing any build time configuration or steps required. The invoker manages the runtime lifecycle of the function, the buildpack managed the compile time aspects of the function.

This means that invoker authors will also be responsible for authoring a buildpack in order for the invoker to be consumed. In riff 0.0.x this role was managed by a Dockerfile provided by the invoker that knew how to compile the function atop a base image.

In theory, a buildpack can be implemented in any language. In practice, using a library is much more convenient. We are looking at creating a function buildpack library in the model of libbuildpack and libcfbuildpack.

@kingdonb

This comment has been minimized.

Copy link

commented Feb 13, 2019

I'll sign me up for adding the Ruby invoker/buildpack support

Is there any language maintainer documentation planned? That seems like a realistic idea, given our last invoker was ~100-200 LOC only

@scothis

This comment has been minimized.

Copy link
Member Author

commented Feb 13, 2019

Thanks @kingdonb. I really appreciate your sticking with us through these changes.

We'll absolutely need to create a guide and reference documentation for creating new invokers and registering them within the system. I'll let you know when we're ready to test the model.

@scothis

This comment has been minimized.

Copy link
Member Author

commented Feb 14, 2019

I spiked splitting out each language from riff-buildpack into independent function buildpacks. The individual buildpacks are reassembled in riff-buildpack-group.

riff-buildpack is acting as:

  • a "finalizer" buildpack that ensures exactly one function buildpack is detected. Ambiguous detection requires the user to specify which function buildpack to consume.
  • a library for other function buildpacks to consume to normalize the detection behavior. We may split this out, minimally it needs additional iteration

With this change, it is possible to implement a custom invokers and a function buildpack that are pulled into a builder image that contains that buildpack/invoker.

Next steps:

  • ensure riff-buildpack is exposing a library abstraction we are comfortable with
  • create a mechanism to create a builder image from a declarative set of function buildpacks (builder inception). Perhaps bringing back the Invoker CRD
  • have riff consume the custom builder image without manually updating the ClusterBuildTemplate or recompiling the riff cli
  • docs, more tests, all the good stuff

@ericbottard @kingdonb @nebhale WDYT?

@scothis scothis changed the title Easy invoker extensibility Invoker extensibility Feb 14, 2019

@kingdonb

This comment has been minimized.

Copy link

commented Feb 14, 2019

Interesting! I will take a look at the links you posted this weekend, and give some feedback.

@jldec jldec referenced this issue Feb 25, 2019

Open

Invoker Roadmap #675

5 of 18 tasks complete

This was referenced Feb 25, 2019

@fbiville fbiville added the roadmap label Feb 25, 2019

@scothis

This comment has been minimized.

Copy link
Member Author

commented Feb 27, 2019

Function buildpack decomposition has landed with riff-buildpack acting as both a finalizer and a library for other function buildpacks. Next up is to make the riff-buildpack-group more configurable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.