yaml-url test has broken names #478

lblackstone · 2019-03-07T18:12:12Z

At some point recently, the yaml-url test started printing the following errors for resource names:

urn=urn:pulumi:dev::guestbook::kubernetes:yaml:ConfigFile$kubernetes:apps/v1:Deployment::Calling [toString] on an [Output<T>] is not supported.

To get the value of an Output<T> as an Output<string> consider either:
1: o.apply(v => `prefix${v}suffix`)
2: pulumi.interpolate `prefix${v}suffix`

See https://pulumi.io/help/outputs for more details.
This function may throw in a future version of @pulumi/pulumi./frontend]

Full output looks like this:

$ pulumi up --skip-preview
Updating (yaml-url-step1):

     Type                           Name
 +   pulumi:pulumi:Stack            guestbook-yaml-url-step1
 +   ├─ kubernetes:yaml:ConfigFile  guestbook
 +   │  └─ kubernetes:core:Service  Calling [toString] on an [Output<T>] is not supported.

 +   │  ├─ kubernetes:core:Service  Calling [toString] on an [Output<T>] is not supported.
 +   └─ kubernetes:core:Namespace   test-namespace
 +   │  └─ kubernetes:core:Service  Calling [toString] on an [Output<T>] is not supported.
     Type                              Name
 +   pulumi:pulumi:Stack               guestbook-yaml-url-step1
 +   ├─ kubernetes:yaml:ConfigFile     guestbook
 +   │  ├─ kubernetes:core:Service     Calling [toString] on an [Output<T>] is not supported.
 +   │  ├─ kubernetes:core:Service     Calling [toString] on an [Output<T>] is not supported.
 +   │  ├─ kubernetes:core:Service     Calling [toString] on an [Output<T>] is not supported.
To get the value of an Output<T> as an Output<string> consider either:
 +   │  └─ kubernetes:apps:Deployment  Calling [toString] on an [Output<T>] is not supported.
1: o.apply(v
 +   │  ├─ kubernetes:apps:Deployment  Calling [toString] on an [Output<T>] is not supported.
 +   └─ kubernetes:core:Namespace      test-namespace
 +   │  └─ kubernetes:apps:Deployment  Calling [toString] on an [Output<T>] is not supported.
1: o.apply(v
 +   │  ├─ kubernetes:apps:Deployment  Calling [toString] on an [Output<T>] is not supported.
 +   └─ kubernetes:core:Namespace      test-namespace
 +   │  └─ kubernetes:apps:Deployment  Calling [toString] on an [Output<T>] is not supported.
1: o.apply(v
To get the value of an Output<T> as an Output<string> consider either:
 +   └─ kubernetes:core:Namespace      test-namespace

Resources:
    + 9 created

Duration: 13s

Here's the code for the test that shows the error'd name output:

import * as k8s from "@pulumi/kubernetes";

// Create a test namespace to allow test parallelism.
const namespace = new k8s.core.v1.Namespace("test-namespace");

function addNamespace(o: any) {
  if (o !== undefined) {
    if (o.metadata !== undefined) {
      o.metadata.namespace = namespace.metadata.name;
    } else {
      o.metadata = {namespace: namespace.metadata.name}
    }
  }
}

// Create resources from standard Kubernetes guestbook YAML example in the test namespace.
new k8s.yaml.ConfigFile("guestbook", {
  file: "https://raw.githubusercontent.com/pulumi/pulumi-kubernetes/master/tests/examples/yaml-guestbook/yaml/guestbook.yaml",
  transformations: [addNamespace]
});

but removing the namespace transformation makes it work as expected

import * as k8s from "@pulumi/kubernetes";

// Create a test namespace to allow test parallelism.
const namespace = new k8s.core.v1.Namespace("test-namespace");

// Create resources from standard Kubernetes guestbook YAML example in the test namespace.
new k8s.yaml.ConfigFile("guestbook", {
  file: "https://raw.githubusercontent.com/pulumi/pulumi-kubernetes/master/tests/examples/yaml-guestbook/yaml/guestbook.yaml",
});

Here's the package.json:

{
  "name": "guestbook",
  "version": "0.1.0",
  "dependencies": {
    "@pulumi/pulumi": "dev"
  },
  "devDependencies": {
    "@types/node": "^9.3.0",
    "typescript": "^3.0.0"
  },
  "peerDependencies": {
    "@pulumi/kubernetes": "latest"
  },
  "license": "MIT"
}

The text was updated successfully, but these errors were encountered:

metral · 2019-03-07T18:15:00Z

I'm hitting this on v0.16.16 w/ no dev packages in my package-lock.json and not using transformation actions. I'm getting them on pulumi & AWS resources:

aws:ec2:SecurityGroup
cloud:service:Service
pulumi:pulumi:Stack

lukehoban · 2019-03-07T18:50:16Z

Looks like this shipped in almost all recent versions of @pulumi/pulumi:

luke:~/dd/pulumidemos/tostringtest
$ curl -s $(npm view @pulumi/pulumi@v0.16.16 dist.tarball) | zgrep -a "is not supported"

luke:~/dd/pulumidemos/tostringtest
$ curl -s $(npm view @pulumi/pulumi@v0.16.17 dist.tarball) | zgrep -a "is not supported"

luke:~/dd/pulumidemos/tostringtest
$ curl -s $(npm view @pulumi/pulumi@v0.16.18 dist.tarball) | zgrep -a "is not supported"
            const message = `Calling [toString] on an [Output<T>] is not supported.
            const message = `Calling [toJSON] on an [Output<T>] is not supported.

luke:~/dd/pulumidemos/tostringtest
$ curl -s $(npm view @pulumi/pulumi@v0.16.19 dist.tarball) | zgrep -a "is not supported"
            const message = `Calling [toString] on an [Output<T>] is not supported.
            const message = `Calling [toJSON] on an [Output<T>] is not supported.

luke:~/dd/pulumidemos/tostringtest
$ curl -s $(npm view @pulumi/pulumi@v0.17.0 dist.tarball) | zgrep -a "is not supported"
            const message = `Calling [toString] on an [Output<T>] is not supported.
            const message = `Calling [toJSON] on an [Output<T>] is not supported.

luke:~/dd/pulumidemos/tostringtest
$ curl -s $(npm view @pulumi/pulumi@v0.17.1 dist.tarball) | zgrep -a "is not supported"
            const message = `Calling [toString] on an [Output<T>] is not supported.
            const message = `Calling [toJSON] on an [Output<T>] is not supported.

CyrusNajmabadi · 2019-03-07T20:28:02Z

Looks like this shipped in almost all recent versions of @pulumi/pulumi:

We return that string from toString/toJSON. But we don't log that message. Logging was only in 16.18 and was rolled back in 16.19.

CyrusNajmabadi · 2019-03-07T20:44:54Z

Reopening since someone is still calling toString

CyrusNajmabadi · 2019-03-07T20:45:20Z

removing my assignment. if you still get htis after moving to 16.19, i can help you determine who is doing this.

One thing you can do to help track this down is change the code inside Output.toString to instead call throw new Error(message);. Then you'll get a stack with the offending logic.

hausdorff · 2019-03-07T20:47:42Z

Sorry, just to clarify my understanding:

pulumi-kubernetes seems like it's calling toString
We should not be, therefore this issue should remain open
Kube team can investigate, and we should definitely solve this for M21; this does not seem hard to track down.

CyrusNajmabadi · 2019-03-07T20:48:45Z

pulumi-kubernetes seems like it's calling toString

It could be anyone doing this. including @pulumi/pulumi.

hausdorff · 2019-03-07T20:51:04Z

Do you think that @metral running into this problem is evidence that this happening somewhere other than pulumi-kubernetes?

metral · 2019-03-07T20:52:32Z

Can confirm that the message goes away after blowing away package-lock.json & node_modules, and re-doing an npm install. This now pins @pulumi/pulumi to v0.16.19 (instead of v0.16.18 before) and resolves the issue.

lblackstone · 2019-03-08T04:24:48Z

So I'm not quite sure how to fix it, but I tracked down the offending line by throwing an error in the toString method.

Here's the stack trace:

    error: Running program '/Users/levi/go/src/github.com/pulumi/pulumi-kubernetes/tests/integration/yaml-url/step1' failed with an unhandled exception:
    error: ReferenceError: BaseError is not defined
        at Proxy.OutputImpl.toString (/Users/levi/go/src/github.com/pulumi/pulumi-kubernetes/tests/integration/yaml-url/step1/node_modules/@pulumi/kubernetes/node_modules/@pulumi/pulumi/output.js:59:23)
        at parseYamlObject (/Users/levi/go/src/github.com/pulumi/pulumi-kubernetes/tests/integration/yaml-url/step1/node_modules/@pulumi/kubernetes/provider.js:256:21)
        at parseYamlDocument (/Users/levi/go/src/github.com/pulumi/pulumi-kubernetes/tests/integration/yaml-url/step1/node_modules/@pulumi/kubernetes/provider.js:133:33)
        at ConfigFile.resources.pulumi.output.apply.t (/Users/levi/go/src/github.com/pulumi/pulumi-kubernetes/tests/integration/yaml-url/step1/node_modules/@pulumi/kubernetes/provider.js:123:61)
        at OutputImpl.<anonymous> (/Users/levi/go/src/github.com/pulumi/pulumi-kubernetes/tests/integration/yaml-url/step1/node_modules/@pulumi/kubernetes/node_modules/@pulumi/pulumi/output.js:103:47)
        at next (native)
        at /Users/levi/go/src/github.com/pulumi/pulumi-kubernetes/tests/integration/yaml-url/step1/node_modules/@pulumi/kubernetes/node_modules/@pulumi/pulumi/output.js:20:71
        at __awaiter (/Users/levi/go/src/github.com/pulumi/pulumi-kubernetes/tests/integration/yaml-url/step1/node_modules/@pulumi/kubernetes/node_modules/@pulumi/pulumi/output.js:16:12)
        at exports.Output.promise.then (/Users/levi/go/src/github.com/pulumi/pulumi-kubernetes/tests/integration/yaml-url/step1/node_modules/@pulumi/kubernetes/node_modules/@pulumi/pulumi/output.js:90:70)
        at process._tickCallback (internal/process/next_tick.js:109:7)
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

And here's the offending code:

pulumi-kubernetes/sdk/nodejs/provider.ts

Lines 2196 to 2201 in 4511a47

    
           const meta = obj["metadata"]; 
        
           let id: string = meta["name"]; 
        
           const namespace = meta["namespace"] || undefined; 
        
           if (namespace !== undefined) { 
        
               id = `${namespace}/${id}`; 
        
           }

(Line 2200)

CyrusNajmabadi · 2019-03-08T06:01:58Z

It sounds like this has found an actual bug. One, or both, of 'id' or 'namespace' are Outputs. So doing this concatenation is just not valid. If these can be Outputs, then this should be written as id = pulumi.all([namespace, id]).apply(([namespace, id]) => ${namespace}/${id}).

However, this will lead to further problems based on how 'id' is used below.

lblackstone · 2019-03-08T17:49:39Z

Yeah, confirmed that namespace is an Output in this case. Trying to figure out the best way to fix this, as it has wide-ranging effects on the surrounding code.

lblackstone · 2019-03-08T21:26:12Z

Given the wider impact caused by fixing the buggy line, this won't be fixed in M21, but will be addressed early in M22

lukehoban · 2019-03-08T21:28:00Z

Does this affect users? Or is it specific to this test? Also - was this always broken (as in did it previously emit [object Object] in names? I can’t quite follow how it could have worked correctly before but not anymore.

lblackstone · 2019-03-08T21:32:10Z

@lukehoban It looks like it's been broken since the feature was introduced last August, but it wasn't on the common path. #409 is caused by the same underlying bug.

In the common case, namespace was a string; this bug only manifests in cases where namespace was a computed value, like using a transformation on a yaml file.

hausdorff · 2019-03-12T00:24:56Z

This specific problem happens when you do something like programmatically set a value (such as namespace) in transforms -- if that value is Output<T>, this cause us to be sad. Looking at the code, it looks like if we change parseYamlObject to return Output, this will cause parseYamlDocument to return Output, which should be fine because CollectionComponentResource#resources is already an Output.

We should be able to fix this quickly, I'm guessing...

lukehoban · 2019-03-12T05:04:26Z

#409 is caused by the same underlying bug.

Indeed - this is precisely a dupe of #409 then - just the details of the text that get rendered changed.

lblackstone added kind/bug Some behavior is incorrect or out of spec priority/P1 labels Mar 7, 2019

lukehoban assigned CyrusNajmabadi Mar 7, 2019

lukehoban added this to the 0.21 milestone Mar 7, 2019

hausdorff changed the title ~~yarl-url test has broken names~~ yaml-url test has broken names Mar 7, 2019

CyrusNajmabadi closed this as completed Mar 7, 2019

CyrusNajmabadi removed their assignment Mar 7, 2019

hausdorff reopened this Mar 7, 2019

lukehoban assigned lblackstone Mar 7, 2019

lblackstone modified the milestones: 0.21, 0.22 Mar 8, 2019

lblackstone mentioned this issue Mar 13, 2019

Fix YAML parsing for computed namespaces #483

Merged

lblackstone closed this as completed in #483 Mar 15, 2019

lblackstone mentioned this issue May 6, 2020

Add YAML support for Go SDK #1093

Merged

2 tasks

infin8x added the p1 A bug severe enough to be the next item assigned to an engineer label Jul 10, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

yaml-url test has broken names #478

yaml-url test has broken names #478

lblackstone commented Mar 7, 2019

metral commented Mar 7, 2019

lukehoban commented Mar 7, 2019 •

edited

Loading

CyrusNajmabadi commented Mar 7, 2019

CyrusNajmabadi commented Mar 7, 2019

CyrusNajmabadi commented Mar 7, 2019 •

edited

Loading

hausdorff commented Mar 7, 2019

CyrusNajmabadi commented Mar 7, 2019

hausdorff commented Mar 7, 2019

metral commented Mar 7, 2019

lblackstone commented Mar 8, 2019

CyrusNajmabadi commented Mar 8, 2019

lblackstone commented Mar 8, 2019

lblackstone commented Mar 8, 2019

lukehoban commented Mar 8, 2019

lblackstone commented Mar 8, 2019

hausdorff commented Mar 12, 2019

lukehoban commented Mar 12, 2019

yaml-url test has broken names #478

yaml-url test has broken names #478

Comments

lblackstone commented Mar 7, 2019

metral commented Mar 7, 2019

lukehoban commented Mar 7, 2019 • edited Loading

CyrusNajmabadi commented Mar 7, 2019

CyrusNajmabadi commented Mar 7, 2019

CyrusNajmabadi commented Mar 7, 2019 • edited Loading

hausdorff commented Mar 7, 2019

CyrusNajmabadi commented Mar 7, 2019

hausdorff commented Mar 7, 2019

metral commented Mar 7, 2019

lblackstone commented Mar 8, 2019

CyrusNajmabadi commented Mar 8, 2019

lblackstone commented Mar 8, 2019

lblackstone commented Mar 8, 2019

lukehoban commented Mar 8, 2019

lblackstone commented Mar 8, 2019

hausdorff commented Mar 12, 2019

lukehoban commented Mar 12, 2019

lukehoban commented Mar 7, 2019 •

edited

Loading

CyrusNajmabadi commented Mar 7, 2019 •

edited

Loading