Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python Dynamic Providers #2900

Merged
merged 14 commits into from
Jul 19, 2019
Merged

Python Dynamic Providers #2900

merged 14 commits into from
Jul 19, 2019

Conversation

lukehoban
Copy link
Member

@lukehoban lukehoban commented Jul 3, 2019

Dynamic providers in Python.

This PR uses dill for code serialization, along with a customization to help ensure deterministic serialization results.

One notable limitation - which I believe is a general requirement of Python - is that any serialization of Python functions must serialize byte code, and byte code is not safely versioned across Python versions. So any resource created with Python 3.x.y can only be updated by exactly the same version of Python. This is very constraining, but it's not clear there is any other option within the realm of what "dynamic providers" are as a feature. It is plausible that we could ensure that updates which only update the serialized provider can avoid calling the dynamic provider operations, so that version updates could still be accomplished. We can explore this separately.

from pulumi import ComponentResource, export, Input, Output
from pulumi.dynamic import Resource, ResourceProvider, CreateResult, UpdateResult
from typing import Optional
from github import Github, GithubObject

auth = "<auth token>"
g = Github(auth)

class GithubLabelArgs(object):
    owner: Input[str]
    repo: Input[str]
    name: Input[str]
    color: Input[str]
    description: Optional[Input[str]]
    def __init__(self, owner, repo, name, color, description=None):
        self.owner = owner
        self.repo = repo
        self.name = name
        self.color = color
        self.description = description

class GithubLabelProvider(ResourceProvider):
    def create(self, props):
        l = g.get_user(props["owner"]).get_repo(props["repo"]).create_label(
            name=props["name"],
            color=props["color"],
            description=props.get("description", GithubObject.NotSet))
        return CreateResult(l.name, {**props, **l.raw_data}) 
    def update(self, id, _olds, props):
        l = g.get_user(props["owner"]).get_repo(props["repo"]).get_label(id)
        l.edit(name=props["name"],
               color=props["color"],
               description=props.get("description", GithubObject.NotSet))
        return UpdateResult({**props, **l.raw_data})
    def delete(self, id, props):
        l = g.get_user(props["owner"]).get_repo(props["repo"]).get_label(id)
        l.delete()

class GithubLabel(Resource):
    name: Output[str]
    color: Output[str]
    url: Output[str]
    description: Output[str]
    def __init__(self, name, args: GithubLabelArgs, opts = None):
        full_args = {'url':None, 'description':None, 'name':None, 'color':None, **vars(args)}
        super().__init__(GithubLabelProvider(), name, full_args, opts)

label = GithubLabel("foo", GithubLabelArgs("lukehoban", "todo", "mylabel", "d94f0b"))

export("label_color", label.color)
export("label_url", label.url)

Fixes #2902.

@lukehoban lukehoban marked this pull request as ready for review July 5, 2019 21:32
We updated our pylint version from 2.1.1 to 2.3.1 and this started surfacing a few new classes of lint warnings.
Dill by default does not recurisvely capture global state, which leads to dangling name references.  For many dynamic provider use cases it is necessary to be able to correctly capture globals.

Also, customize the pickler to ensure dicts are sorted prior to serializing so that we get a more deterministic serialization.
@lukehoban lukehoban merged commit 3768e5c into master Jul 19, 2019
@pulumi-bot pulumi-bot deleted the lukehoban/pythondynamicproviders branch July 19, 2019 17:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support Dynamic Providers in Python
3 participants