Plans for a V2

[deepmap-marcinr]

oapi-codegen was originally a quick little project to help us generate server code from OpenAPI specs at DeepMap, but it's grown quite a bit bigger than I expected.

I've started a v2-dev branch to work on breaking changes to both generated code and the runtime. My plans are as follows.

• Nillable properties of objects, such as maps and arrays, need not have an indirect in front of them, as they support the three critical values of (unset, empty, non-empty). We need the indirect for basic types because zero values due to absence and zero are indistinguishable
• The code generation is messy. I'm going to create a struct to pass into all templates, which is always the same, so it's easier to add additional templates, even at runtime. This struct will hopefully be fairly stable.
• We generate too many types, too many type definitions. I want to track the typedef references, and simply follow them to either the deepest complex type or primitive.
• In our own OpenAPI specs, we declare all types in #/components/schemas, so we never had to think about declaring types for anonymous types, such as response bodies which don't have a schema under #/components. In my opinion, it is better for this package to be dumb, and leave the choice whether to declare a type or not based on the structure of the spec - inline types are anonymous, explicit types create Go types. It appears, though, that people have been pointing this code generator at specs which they don't control, so this gets annoying really quickly. I'm not sure how to best handle this yet.
• Identifier names in parameters are fragile. All need to go through a common validation and fixup path, and test cases need to generate every kind of invalid character in each kind of identifier.
• ClientWithResponses is a mess, both in terms of implementation and generator output. I'm still thinking about this one, but I don't like what it produces.
• I'm going to refactor how references are handled, to make it easier to refer to types in embedded packages.
• The command line is getting too large, but we support config files. I'm thinking of just moving all generator settings to the config file and stripping out the command line options, it'll be one less thing to maintain.
• Once we have stable template inputs, we can start versioning template files themselves. Sometimes, we have to make breaking changes to generated code, so we'll have a way of using a new generator, but using older templates.

[davidovich]

I am in the process of implementing a Terraform provider for an OpenAPI based API. I thought that naturally we could generate the Terraform boilerplate from OpenAPI, but this implies that the generator would allow generating non-struct based schemas like the terraform schema.Ressource.

For this, I think there should be more data extracted to template functions and data and less in the generator go codepaths. In essence, a more complete abstract model that can be manipulated in templates. This would allow, in templates, iteration on the OpenAPI definition and generation of many output forms, Terraform included.

In short:

Read in the OpenAPI -> transform in intermediate form -> feed to template.

WDYT?

[deepmap-marcinr]

This is a tricky change. Yes, I think feeding an intermediate format to templates and letting them choose how to render types is conceptually a good idea, however, the logic available in templates is very primitive, so the intermediate format would have to be something that's a trivial manipulation away from its final form, or something that we can feed to a pre-canned transformer function.

I'm curious, how would that Terraform schema be expressed in OpenAPI at all? I don't see a way to do it in a generic way. AdditionalProperties lets you have a map of fields, but then, since each value in that map is a different type, your only choice really is to make each element a "OneOf" schema of all possible value types.

This may be one of those edge cases where we're trying to go so far off the beaten path, and torturing the spec into submission, that it may not be worth doing. I always consider the tradeoff between flexibility and usability, since they are at times at odds with each other.

[davidovich]

The OpenAPI document can describe server interactions (expected inputs and outputs). From this information, oapi-codegen can create a client and types for this API (with expected types generated for us). These types are not far from the Terraform schema.Resource description. These schemas allow describing keys and types of resources which map to the underlying json representation.

Here is a generated client for our internal logstack management API:

type LogstackRequest struct {

	// The plan_id(size) of the logstack.
	BillingPlanId int32 `json:"billing_plan_id"`

	// The project description to assign the logstack.
	Description string `json:"description"`

	// Fill in LDAP groups that are allowed to access your kibana ui and access the Elasticsearch API.
	LdapFields struct {
		ReadOnlyApi  *[]string `json:"read_only_api,omitempty"`
		ReadOnlyWeb  *[]string `json:"read_only_web,omitempty"`
		ReadWriteWeb *[]string `json:"read_write_web,omitempty"`
	} `json:"ldap_fields"`

	// The project name to assign the logstack.
	Name          string                        `json:"name"`
	RoutingFields LogstackRequest_RoutingFields `json:"routing_fields"`

	// The plan_id(size) of the logstack.
	SizingPlanId int32 `json:"sizing_plan_id"`
}

And here is the (draft) terraform boilerplate:

	return &schema.Resource{
		// This description is used by the documentation generator and the language server.
		Description: "Logstack stack resource.",
		Schema: map[string]*schema.Schema{
			"name": {
				Type:     schema.TypeString,
				Required: true,
			},
			"description": {
				Type:     schema.TypeString,
				Required: true,
			},
			"sizing_plan_id": {
				Type: schema.TypeInt,
				Computed: true,
			},
			"billing_plan_id": {
				Type:     schema.TypeInt,
				Required: true,
				ForceNew: true,
			},
			"region": {
				Type:     schema.TypeString,
				Required: true,
				ForceNew: true,
			},
			"size": {
				Type:     schema.TypeString,
				Optional: true,
				ForceNew: true,
			},
		},
	}

oapi-codegen is doing all the types and interaction heavy-lifting, and we could somehow use that information in the templates. Of course this would need some custom extensions in the OpenAPI spec. I am very aware that this may be an outlier use-case.