New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Support multiple-revisions of functions #1075

Open
alexellis opened this Issue Feb 1, 2019 · 9 comments

Comments

Projects
None yet
6 participants
@alexellis
Copy link
Member

alexellis commented Feb 1, 2019

Support multiple-revisions of functions

Description

We should consider how to enable / support multiple-revisions of OpenFaaS functions. This is currently done by changing the name of the function before deployment.

If possible we could use a new feature-flag (for backwards compatibility) and conventions on the Docker image tag to support multiple concurrent versions of functions.

Imagine three different git commits and image tags defined in your stack.yml file:

functions:
  # version allows multiple slash commands in one comment
  name: derek
  image: derek:ef5ja
 
functions:
  # version has rebase command enabled
  name: derek
  image: derek:ffa3j

functions:  
  # version represents latest stable release, indicated via sem-ver
  name: derek
  image: derek:0.6.0 

This would produce 3 endpoints when deployed:

/function/derek-dev
/function/derek-ffa3j
/function/derek-0.6.0

We may want to then add an additional route like /release or /revision which can make use of "smart" aliases:

/release/derek/dev/    latest SHA   (ef5ja)
/release/derek/ffa3j/  other  SHA   (ffa3j)
/release/derek/0.6.0/  specific semver
/release/derek/0.6.1/   specific semver
/release/derek/  latest semver (0.6.1)

When we have two semver versions we can order these using a semver sort so that /release/derek/ points to 0.6.1.

Ideas for feature: Alex / Stefan

@burtonr

This comment has been minimized.

Copy link
Member

burtonr commented Feb 4, 2019

I think it's important to keep the URL path the same for current/latest function versions.

gateway.openfaas.com/function/derek should always point to the most recent version available. I'm not certain how to accomplish this exactly. It could be an additional label latest_function_version: true and the label would need to be removed once another function of the same name is deployed.

For routing to a specific version, we could add the version to the route:
gateway.openfaas.com/function/derek/ffa3j in order to "lock" some outside application to a specific version.

The proxy should remove the version from the route when forwarding to the function to avoid forcing the individual functions to know if there is a version attached so that we don't break the path inside the functions.

Behind the scenes, we could use labels to identify the version of the function. That could be created automatically when the function is deployed by taking the label from the container image and creating a label.

There is also the question of how does the deploy work with this in place? Would this mean that the users would need to explicitly remove all the containers themselves? Could we implement some sort of prune command that would clear out all but the latest version of the function?
I worry a little about having a lot of containers being deployed while testing/development that may cause issues...

@noppen

This comment has been minimized.

Copy link

noppen commented Feb 4, 2019

Very much in favour of managed versioning of functions. Ideally the latest on the same endpoint or at least a softlink endpoint that lands you on the last deployed iteration of a function. Under ideal circumstances you would also have a rollback mechanism that allows you to determine a fixed endpoint of a previous version you know your dependent application worked well with. For interoperability as well as migration in our environment this would be extremely useful.

@LucasRoesler

This comment has been minimized.

Copy link
Member

LucasRoesler commented Feb 5, 2019

I like this idea.

With respect to automating smart aliases, i see how the default/latest path could be implemented for semver but not with shas. Wouldn't it be easier to add an alias property to allow the user to manager these aliases. The "default" path would be either the last or the first found in the stack file. So,

functions: 
    # version allows multiple slash commands in one comment 
    name: derek 
    image: derek:ef5ja 
    
functions: 
    # version has rebase command enabled 
    name: derek 
    image: derek:ffa3j 
    
functions: 
    # version represents latest stable release, indicated via sem-ver 
    name: derek 
    image: derek:0.6.0

you would get this by default (assuming last in takes the default)

/function/derek-ef5ja 
/function/derek-ffa3j 
/function/derek-0.6.0
/function/derek # aliases /function/derek-0.6.0

but the user could override this using

functions: 
    # version allows multiple slash commands in one comment 
    name: derek 
    image: derek:ef5ja 
    alias:
        - derek-latest
    
functions: 
    # version has rebase command enabled 
    name: derek 
    image: derek:ffa3j 
    alias:
        - derek-feature-branch
    
functions: 
    # version represents latest stable release, indicated via sem-ver 
    name: derek 
    image: derek:0.6.0
    alias:
         - derek-stable
/function/derek-ef5ja 
/function/derek-ffa3j 
/function/derek-0.6.0
/function/derek-stable # aliases /function/derek-0.6.0
/function/derek-feature-branch # aliases /function/derek-ffa3j
/function/derek # aliases /function/derek-ef5ja
@alexellis

This comment has been minimized.

Copy link
Member Author

alexellis commented Feb 5, 2019

Stefan's idea for the priority order for SHA would be the date deployed.

@alexellis

This comment has been minimized.

Copy link
Member Author

alexellis commented Feb 5, 2019

The aliases idea gets us too far into "configure everything" Vs our style or convention over configuration. When building from a feature branch I think you'd tag the image with the branch name.

@stefanprodan could probably add more of how he does this with customers on Kubernetes with flux.

Alex

@LucasRoesler

This comment has been minimized.

Copy link
Member

LucasRoesler commented Feb 5, 2019

Not adding aliases makes sense

I think I would prefer not automating aliases for shas, because it might be too difficult for someone to consistently understand why and which alias is set. Automting based on "latest tag" and then "semver order" for the default path and then unique names for anything else (shas or branch names) would be pretty clear. You can easily read this directly from the file and predict what will happen.

Using datetime created would mean that the resulting urls would depend on how they were deployed. If you deploy the entire stack file it would be "order of the file". But you could deploy one at a time in a random order.

Note: One downside to semver order is that it does not play well it CI/CD git tag based versions because strict semver does not have a concept of a post-release, i.e. "number of commits after a version". It only has pre-releases, ie "number of versions prior to the next release. See semver/semver#200 and semver/semver#394 This is not an issue, but just something to be aware of. Any strict semver ordering will not be able to properly order nightly or CI/CD builds that use git describe to generate the tag

@stefanprodan

This comment has been minimized.

Copy link
Member

stefanprodan commented Feb 5, 2019

Why would anyone use sem ver for nightly builds? Usually you would use the short SHA for nightly builds or pre releases and strict sem ver for releases. It's easy to follow this patter and from a CI perspective it's 10 lines of bash: https://github.com/stefanprodan/flagger/blob/master/.travis.yml#L22

@LucasRoesler

This comment has been minimized.

Copy link
Member

LucasRoesler commented Feb 5, 2019

I only brought it up because I have been on multiple teams where the consensus was "we use semver" but then there were unexpected differences in how tools behave because people thought that git describe --tags would produce semver, but this is not a valid semver: v3.2.2-17-g98f8c58. Or it is a semver but is has a different meaning in git and in semver, so the order is not what you expect. This discussion probably isn't so helpful in this thread, we could move it to slack, but it might be something that we need to address in documentation or FAQ at some point.

@imishravmw

This comment has been minimized.

Copy link

imishravmw commented Feb 6, 2019

Hi @alexellis ,
I have some thought about function versioning.
Right now, when we do "faas-cli new" we get a yml file where image tag is latest.
I suggest to add a new flag "-v " to following CLI
faas-cli new
faas-cli build
faas-cli push
faas-cli deploy

Let me explain how this new version flag can behave in all above CLI
faas-cli new -v
Currently above CLI generates a yml file where image is tagged with :latest label, so when someone gives i.e. "faas-cli new -v v2" the yml file will contain image with label ":v2" , default could remain same as ":latest"

faas-cli build -v
This will build docker image having version tag, even if yml file contains image with ":latest" label, so version flag should be given precedence over already existing label of image.

faas-cli push -v
This should push docker image with version

faas-cli deploy -v
This should deploy docker images with version label, irrespective of what is mentioned in yml file. also we should be adding label to Pod and service for function ( in case of K8) with version , so if any routing or selection needs to be done based on version labels, it can be done.

Using version labels on pod can help in rolling updates or rollback to previous version (this can be discussed later)

Let me know if above proposal suits the purpose.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment