perf(filestate): DoesProjectExist should not list all

[abhinav]

Hello!

• Vote on this issue by adding a 👍 reaction
• If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

There's room for a small performance improvement to the DoesProjectExist method
in the filestate backend:
Instead of listing all projects and searching through them, it could search for
the prefix $projectName/, and report success if that doesn't fail with
NotFound.

Credit to @Frassle for the suggestion:
#12437 (comment)

Affected area/feature

filestate backend

[rafiramadhana]

Hi @abhinav , I am interested to work on this issue.

Here is my proposed solution:

• Update projectReferenceStore.ListProject (store.go#L207) to accept $projectName as arg (or we can put the value in ctx to preserve backward compatibility)
• Update prefix in store.go#L208 from path := StacksDir to path := StacksDir + "/" + $projectName

Btw, I am assuming the behavior of listBucket (bucket.go#L66) to support filtering by having $projectName in the prefix (sry, haven't tried to explore the repo that much)

Please LMK if you have concern or anything. Thanks!

[abhinav]

Thanks, @neverbeenthisweeb! A PR would absolutely be welcome.

Adding projectName to ListProjects is a valid option.
However, it may be easier to add a new method to projectReferenceStore.
For example, if we added a ProjectExists(ctx context.Context, projectName string) (bool, error) method to projectReferenceStore, it could implement the specialized behavior, while still keeping the layout details isolated to the projectReferenceStore.

The implementation that you're planning itself sounds reasonable: see if listing path.Join(StacksDir, projectName) returns an empty result.

[rafiramadhana]

@abhinav I see, agree. Would you mind to assign this to me?

[abhinav]

Done!

[rafiramadhana]

@abhinav Hi, questions

Let's say we have this bucket structure

.
└── pulumi/
    └── stacks/
        ├── a
        └── b/
            └── foo.json

I'm quite sure that we consider project b as exist here, CMIIW.

However, do we consider project a as exist too?

As a context, I am going to use listBucket (https://github.com/pulumi/pulumi/blob/master/pkg/backend/filestate/bucket.go#L66) for checking if the project exist. But It looks like listBucket is not able to differ between non-exist project and empty project.

PS. I've created a draft PR for this case (see https://github.com/pulumi/pulumi/pull/12798/files#r1181096721)

[Frassle]

I would say "a" should return as not existing. A project exists if it has any stacks.