Source Positions for Pulumi

[pgavlin]

The Pulumi resource model does not currently provide any first-class way to associate source positions with resources, invokes, etc. Enabling source position information in the resource model can provide substantial benefits, including but not limited to:

• Better errors from the Pulumi CLI
• Go-to-defintion for resources in state
• Editor integration for pulumi preview

This document describes an approach to packaging and persisting source position information such that it can be consumed within the context of an active update or a stack's statefile.

Design

Source position representation

Source positions are (file, line) or (file, line, column) tuples represented as URIs. The line and column are stored in the fragment portion of the URI as "line(,column)?". The scheme of the URI and the form of its path component depends on the context in which it is generated or used:

• During an active update, the URI's scheme is file and paths are absolute filesystem paths. This allows consumers to easily access arbitrary files that are available on the host.
• In a statefile, the URI's scheme is project and paths are relative to the project root. This allows consumers to resolve source positions relative to the project file in different contexts irrespective of the location of the project itself (e.g. given a project-relative path and the URL of the project's root on GitHub, one can build a GitHub URL for the source position).

Paths may refer to files outside of the project. In a statefile, such paths will begin with a .. component.

For example, given the following project:

/home/pulumi/hello-world/
    Pulumi.yaml
    index.ts

import * as random from "@pulumi/random";
new random.RandomPet("pet");

The source position of new random.RandomPet would be passed to the engine as file:///home/pulumi/hello-world/index.ts#2,1, and would be stored in the state file as project:///index.ts#2,1.

Communicating source position information during an update

During an update, source position information may be attached to gRPC calls as "source-position" metadata. This allows arbitrary calls to be associated with source positions without changes to their protobuf payloads. Modifying the protobuf payloads is also a viable approach, but is somewhat more invasive than attaching metadata, and requires changes to every call signature.

Source positions should reflect the position in user code that initiated a resource model operation (e.g. the source position passed with RegisterResource for pet in the example above should be the source position in index.ts, not the source position in the Pulumi SDK). In general, the Pulumi SDK should be able to infer the source position of the resource registration, as the relationship between a resource registration and its corresponding user code should be static per SDK.

Storing source position information in state files

Source positions in state files will be stored as a new registeredAt property on each resource. This property is optional.

Implementation tasks

• Add engine support for extracting source position information from gRPC messages
• Add engine support for resolving absolute filesystem paths to project-relative paths
• Add statefile support for storing resource registration positions
• Add SDK support for inferring source positions