-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support community language plugins #11882
Comments
Here are some of the issues that we have encountered while implementing all of the JVM-based Pulumi SDKs (Java, Kotlin, Scala). Most of them are related to provider package schemas but some apply to general architectural decisions and feature support.
Should this really result in generation of an empty class? Some usages of such types seem to suggest something different, e.g. (inside the definition of
The intent of using
Is there an exhaustive list of such extensions so that we can figure out which of them we could reuse in our code generation? E.g. currently we reuse the mapping of package names from java.
In general, remote resources as a feature seem quite complex and hard to understand. For instance, is it possible, in case of import of a foreign stack, that a serialized resource will be received that has a resource type that is not available in the currently executing Pulumi program (i.e. an AWS EC2 instance resource will arrive in a Pulumi program using Java or Scala SDK that has no direct dependency on AWS package (so no jar added to classpath and subsequently no Instance class available). If not, how so?
|
Regarding integration with Pulumi itself: for besom (pulumi-scala) we have found out that by implementing a language host and manually installing language plugin we are able to use besom with pulumi cli without any issues. We haven't really gotten to the point where our codegen would require integration with pulumi tooling (cli, crd2pulumi, tf2pulumi) but we strongly hope that's something that is going to be possible in the future. There's also the question of integration with docs and examples. We assume that snippets in javadoc/godoc/*doc (mentioned in point 7 above) and/or proper pulumi.com website docs are not written by hand and are also code generated. It would be massively helpful to know the following: a) from what are these docs generated from? We'd love to generate them for our SDKs too. |
I suspect this is just missing validation, I don't think it's actaully a valid schema if the same name is used for a Type and a Resource. "package:module:member" is probably sufficent everywhere and we should just unify and validate towards that.
This is just missing validation. We shouldn't be publishing schemas like this.
I think the current rule is that pulumi is case-sensitive. I would like to change this to instead be case sensitive but with restrictions on what casing is allowed in a name. The above would then have to be written as "IP_allocation_method", mixed case in words would be disallowed.
I think an empty type should just be an empty object. I suspect the use of this type here in aws-native is probably a bug?
I don't think there's any way to exclude types from generation. If a language can't generate a type it should probably just make that decision itself and print a warning that the SDK hasn't been fully generated.
These are "overlays". We're probably going to remove these, they are by their nature extreamly language specific and are probably better handled by extension packages manually written per-language (e.g. we'd have "@pulumi/aws" and "@pulumi/aws-extensions" where aws-extensions would be a manually written package to supply those extensions.
This needs an overhaul. We're thinking we'll probably be sticking to markdown because it's so common but not 100% certain. The example sections will be getting replaced with dedicated "example" attributes and will have examples written in PCL (Our internal cross-code markup) or YAML.
Most of these should probably be removed. In general add what makes sense for your language, don't copy other languages options. We're also looking at a way to supply these data side-by-side rather than inline with the schema.json.
We try to stick to semver but it's probably not perfect and nothing automatically enforces it.
I think this should only really occur with remote component packges. It probably doesn't strictly need reflection because the schema should say what the expected resources to be returned are.
This is a good point. I think they we're heavily designed around a TypeScript world with dynamic types. We'll need to think about this.
Not really. A lot of these are for compat with old engines and old SDKs. We should probably document these better in the protobuf files themselves.
Again not really documented anywhere. But there are two paths of work happening to improve this. Firstly we're trying to remove the need for logic around these properties from SDKs, things like inheriting proect from the parent will be done by the engine so SDKs don't need to. Secondly we're starting up a new way to supply test programs to language plugins that will validate they behave in the way the engine expects, this will include things like merging secret and non-secret properties together and checking they are still secret.
crd2pulumi and tf2pulumi are both on the path to deprecation (tf2pulumi is in fact already deprecated) so don't worry about integration with those. We are working on new RPC interfaces to allow language plugins to supply codegen to the CLI rather than it being built in and in fact the Go and NodeJS language plugins are already working this way for most cli integrations.
They're generated by our "docgen" tool which is split over https://github.com/pulumi/pulumi-hugo/ and https://github.com/pulumi/pulumi. It's also due a workover so I wouldn't invest much time into learning it now. |
Are the links in the docs for "How can I add support for my favorite language?" still the best place to learn more about this?
|
Yes and feel free to ask questions on the #contribute channel in our community slack. |
Hello!
Issue details
We have a number of issues requesting new languages:
It is not feasible for Pulumi (the company) to build and maintain support for every language. However we would like to support engineers writing Pulumi programs in the language of their choice, and so need to support the community building their own language plugins.
Currently languages are heavily coupled to the pulumi cli itself:
All of this needs fixing to allow the community to have a good experience building and publishing their own language plugins.
The best way to support this is that our language plugins should not be privileged, that is the cli should not know about any of our languages. Everything should be done via the language plugin interface.
We made some progress cleaning up things in this space last year (a lot of language specific code in
pulumi new
for example got moved across to the language plugins, and all of it inpulumi about
was moved across).As well as all of the above that really must be done for the community to have a chance of building their own languages we should also invest in making it so language SDKs need less manual work to be complete.
This should be done in two ways:
The text was updated successfully, but these errors were encountered: