Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model validation failed (#: required key [Code] not found) [has workaround] #906

Closed
jakobo opened this issue May 8, 2023 · 7 comments · Fixed by #1395
Closed

Model validation failed (#: required key [Code] not found) [has workaround] #906

jakobo opened this issue May 8, 2023 · 7 comments · Fixed by #1395
Assignees
Labels
area/providers good-first-issue Start here if you'd like to start contributing to Pulumi impact/usability Something that impacts users' ability to use the product easily and intuitively kind/bug Some behavior is incorrect or out of spec resolution/fixed This issue was fixed size/S Estimated effort to complete (1-2 days).

Comments

@jakobo
Copy link

jakobo commented May 8, 2023

What happened?

I have an existing lambda funnction using the TypeScript awsnative library. I attempted to add the official AWS layer for KMS support (arn:aws:lambda:us-west-2:345057560386:layer:AWS-Parameters-and-Secrets-Lambda-Extension-Arm64:4).

Expected Behavior

pulumi up runs without issue

actual
error: operation error CloudControl: UpdateResource, https response error StatusCode: 400, RequestID: zzzzz, api error ValidationException: Model validation failed (#: required key [Code] not found)

With debug output, the request payload sent to CloudControl is

{"ClientToken":"zzzzzz","Identifier":"dev-jakob-graphql-public","PatchDocument":"[{\"op\":\"add\",\"path\":\"/Layers\",\"value\":[\"arn:aws:lambda:us-west-2:345057560386:layer:AWS-Parameters-and-Secrets-Lambda-Extension-Arm64:4\"]}]","TypeName":"AWS::Lambda::Function"}

With the response:

{"__type":"com.amazon.coral.validate#ValidationException","message":"Model validation failed (#: required key [Code] not found)"}

Steps to reproduce

The following lambda configuration will work on the first deploy.

export const lambdaStore = new awsnative.s3.Bucket("ls", {
  // note: do not rename a bucket
  // https://github.com/pulumi/pulumi-aws-native/issues/584
  bucketName,
  accessControl: "Private",
  versioningConfiguration: {
    status: "Enabled",
  },
  lifecycleConfiguration: {
    rules: [
      {
        id: "keepLastX",
        status: "Enabled",
        noncurrentVersionExpiration: {
          // keep last 10, even if a service isn't touched for a while
          newerNoncurrentVersions: 10,
          // keep anything from last 30 days
          noncurrentDays: 30,
        },
      },
    ],
  },
});

// file is a placeholder, overwritten by external deploys in the lambda
export const graphqlPublicZip = new aws.s3.BucketObject(
  lambdaName,
  {
    bucket: lambdaStore.id,
    key: "samplefn",
    source: new pulumi.asset.AssetArchive({
      "index.js": new pulumi.asset.StringAsset(
        `module.exports = () => { console.log("placeholder"); return Promise.resolve(); }`
      ),
    }),
  },
  { ignoreChanges: ["source"] }
);

const graphqlPublicRole = new aws.iam.Role(lambdaName, {
  assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal(
    aws.iam.Principals.LambdaPrincipal
  ),
  managedPolicyArns: [
    /** Can execute lambdas and send logs to cloudwatch */
    aws.iam.ManagedPolicy.AWSLambdaBasicExecutionRole,
  ],
});

export const graphqlPublic = new awsnative.lambda.Function(
  "sample",
  {
    functionName: "fnname",
    role: graphqlPublicRole.arn,
    memorySize: 128,
    architectures: ["arm64"],
    runtime: "nodejs16.x",
    handler: "index.handler",
    code: {
      s3Bucket: lambdaStore.id,
      s3Key: graphqlPublicZip.key,
      s3ObjectVersion: latest,
    },
  }
);

Before a second pulumi up, add the layers object to the lambda config

{
  // ...
  layers: [
    // enable KMS communication
    "arn:aws:lambda:us-west-2:345057560386:layer:AWS-Parameters-and-Secrets-Lambda-Extension-Arm64:4",
  ],
}

Then, running pulumi up generates the 400 Bad Request errors above.

Output of pulumi about

CLI
Version      3.66.0
Go Version   go1.20.3
Go Compiler  gc

Plugins
NAME            VERSION
aws             5.35.0
aws-apigateway  1.0.1
aws-native      0.57.0
command         0.7.2
mongodbatlas    3.7.2
nodejs          unknown
purrl           0.3.1
random          4.12.1

Host
OS       ubuntu
Version  20.04
Arch     x86_64

This project is written in nodejs: executable='/home/jakobo/.asdf/shims/node' version='v18.12.1'

Current Stack: taskless/infra/development

TYPE                                                        URN
pulumi:pulumi:Stack                                         urn:pulumi:development::infra::pulumi:pulumi:Stack::infra-development
pulumi:providers:aws                                        urn:pulumi:development::infra::pulumi:providers:aws::default_5_35_0
aws:iam/role:Role                                           urn:pulumi:development::infra::aws:iam/role:Role::dev-jakob-graphql-public
aws:apigatewayv2/api:Api                                    urn:pulumi:development::infra::aws:apigatewayv2/api:Api::dev-jakob-api
pulumi:providers:aws-native                                 urn:pulumi:development::infra::pulumi:providers:aws-native::default_0_57_0
pkg:index:MongoDataAPI                                      urn:pulumi:development::infra::pkg:index:MongoDataAPI::dev-jakob
aws:iam/role:Role                                           urn:pulumi:development::infra::aws:iam/role:Role::cloudwatch
aws:acm/certificate:Certificate                             urn:pulumi:development::infra::aws:acm/certificate:Certificate::dev-jakob-api-4960dcd6-cert
aws-native:sqs:Queue                                        urn:pulumi:development::infra::aws-native:sqs:Queue::dev-jakob-intake-dlq
aws-native:s3:Bucket                                        urn:pulumi:development::infra::aws-native:s3:Bucket::dev-jakob-lambda-artifacts
pulumi:providers:mongodbatlas                               urn:pulumi:development::infra::pulumi:providers:mongodbatlas::default_3_7_2
pulumi:providers:random                                     urn:pulumi:development::infra::pulumi:providers:random::default_4_12_1
mongodbatlas:index/project:Project                          urn:pulumi:development::infra::mongodbatlas:index/project:Project::dev-jakob
aws:apigateway/account:Account                              urn:pulumi:development::infra::aws:apigateway/account:Account::account
random:index/randomPassword:RandomPassword                  urn:pulumi:development::infra::random:index/randomPassword:RandomPassword::password
aws-native:sqs:Queue                                        urn:pulumi:development::infra::aws-native:sqs:Queue::dev-jakob-intake
aws:s3/bucketObject:BucketObject                            urn:pulumi:development::infra::aws:s3/bucketObject:BucketObject::dev-jakob-graphql-public
mongodbatlas:index/serverlessInstance:ServerlessInstance    urn:pulumi:development::infra::mongodbatlas:index/serverlessInstance:ServerlessInstance::dev-jakob
mongodbatlas:index/projectIpAccessList:ProjectIpAccessList  urn:pulumi:development::infra::mongodbatlas:index/projectIpAccessList:ProjectIpAccessList::all
pulumi:providers:pulumi-nodejs                              urn:pulumi:development::infra::pulumi:providers:pulumi-nodejs::default
mongodbatlas:index/databaseUser:DatabaseUser                urn:pulumi:development::infra::mongodbatlas:index/databaseUser:DatabaseUser::dev-jakob
pulumi-nodejs:dynamic:Resource                              urn:pulumi:development::infra::pkg:index:MongoDataAPI$pulumi-nodejs:dynamic:Resource::dev-jakob-application
aws:iam/policy:Policy                                       urn:pulumi:development::infra::aws:iam/policy:Policy::dev-jakob-graphql-public-policy
pulumi-nodejs:dynamic:Resource                              urn:pulumi:development::infra::pkg:index:MongoDataAPI$pulumi-nodejs:dynamic:Resource::dev-jakob-config
pulumi-nodejs:dynamic:Resource                              urn:pulumi:development::infra::pkg:index:MongoDataAPI$pulumi-nodejs:dynamic:Resource::dev-jakob-data-source
pulumi-nodejs:dynamic:Resource                              urn:pulumi:development::infra::pkg:index:MongoDataAPI$pulumi-nodejs:dynamic:Resource::dev-jakob-auth-provider
aws:iam/rolePolicyAttachment:RolePolicyAttachment           urn:pulumi:development::infra::aws:iam/rolePolicyAttachment:RolePolicyAttachment::dev-jakob-graphql-public-rpa
pulumi-nodejs:dynamic:Resource                              urn:pulumi:development::infra::pkg:index:MongoDataAPI$pulumi-nodejs:dynamic:Resource::dev-jakob-default-rule
pulumi-nodejs:dynamic:Resource                              urn:pulumi:development::infra::pkg:index:MongoDataAPI$pulumi-nodejs:dynamic:Resource::dev-jakob-api-key
aws:route53/record:Record                                   urn:pulumi:development::infra::aws:route53/record:Record::dev-jakob-api-4960dcd6-validator
aws:acm/certificateValidation:CertificateValidation         urn:pulumi:development::infra::aws:acm/certificateValidation:CertificateValidation::dev-jakob-api-4960dcd6-cert-validation
aws:apigatewayv2/domainName:DomainName                      urn:pulumi:development::infra::aws:apigatewayv2/domainName:DomainName::dev-jakob-api-domain
aws:route53/record:Record                                   urn:pulumi:development::infra::aws:route53/record:Record::dev-jakob-api-dns
aws-native:lambda:Function                                  urn:pulumi:development::infra::aws-native:lambda:Function::dev-jakob-graphql-public
aws:apigatewayv2/integration:Integration                    urn:pulumi:development::infra::aws:apigatewayv2/integration:Integration::dev-jakob-api-graphql-eb6263ba-ext
aws:apigatewayv2/integration:Integration                    urn:pulumi:development::infra::aws:apigatewayv2/integration:Integration::dev-jakob-api-graphql-eb6263ba
aws:apigatewayv2/route:Route                                urn:pulumi:development::infra::aws:apigatewayv2/route:Route::dev-jakob-api-graphql-eb6263ba-ext
aws:lambda/permission:Permission                            urn:pulumi:development::infra::aws:lambda/permission:Permission::dev-jakob-api-graphql-eb6263ba-ext
aws:apigatewayv2/route:Route                                urn:pulumi:development::infra::aws:apigatewayv2/route:Route::dev-jakob-api-graphql-eb6263ba
aws:lambda/permission:Permission                            urn:pulumi:development::infra::aws:lambda/permission:Permission::dev-jakob-api-graphql-eb6263ba
aws:apigatewayv2/deployment:Deployment                      urn:pulumi:development::infra::aws:apigatewayv2/deployment:Deployment::dev-jakob-api-deployment
aws:apigatewayv2/stage:Stage                                urn:pulumi:development::infra::aws:apigatewayv2/stage:Stage::dev-jakob-api-stage
aws:apigatewayv2/apiMapping:ApiMapping                      urn:pulumi:development::infra::aws:apigatewayv2/apiMapping:ApiMapping::dev-jakob-api-map


Found no pending operations associated with taskless/development

Backend
Name           pulumi.com
URL            https://app.pulumi.com/jakobo
User           jakobo
Organizations  jakobo, taskless

Pulumi locates its logs in /tmp by default
warning: Failed to get information about the Pulumi program's dependencies: could not find either /home/jakobo/code/t2/infra/yarn.lock or /home/jakobo/code/t2/infra/package-lock.json

Additional context

Slack Thread: https://pulumi-community.slack.com/archives/C84L4E3N1/p1683565128664669
Related: hashicorp/terraform-provider-awscc#185 (comment)

keywords: awsnative, lambda, CloudControl, "[Code] not found"

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

@jakobo jakobo added kind/bug Some behavior is incorrect or out of spec needs-triage Needs attention from the triage team labels May 8, 2023
@jakobo
Copy link
Author

jakobo commented May 8, 2023

Workaround

I was able to work around this issue by requiring all changes to layers to result in a replacement using the replaceOnChange resource option. Because the lambda can be updated outside of pulumi, the replacement needs to be aware of the latest versionId when recreating the resource.

const latest = pulumi
  .all([graphqlPublicZip.bucket, graphqlPublicZip.key])
  .apply(async ([bucket, key]) => {
    const result = await aws.s3.getObject({
      bucket,
      key,
    });

    return result.versionId;
  });
export const graphqlPublic = new awsnative.lambda.Function(
  lambdaName,
  {
    functionName: lambdaName,
    role: graphqlPublicRole.arn,
    memorySize: 128,
    architectures: ["arm64"],
    runtime: "nodejs16.x",
    handler: "index.handler",
    code: {
      s3Bucket: lambdaStore.id,
      s3Key: graphqlPublicZip.key,
      s3ObjectVersion: latest,
    },
    layers: [
      // enable KMS communication
      "arn:aws:lambda:us-west-2:345057560386:layer:AWS-Parameters-and-Secrets-Lambda-Extension-Arm64:4",
    ],
  },
  { replaceOnChanges: ["code", "layers"] }
);

It's hardly elegant, but it does ensure that the object is being recreated correctly.

@justinvp
Copy link
Member

This seems to be specific to awsnative, so I'm going to transfer the issue over to that repo.

@justinvp justinvp transferred this issue from pulumi/pulumi May 12, 2023
@jakobo jakobo changed the title awsnative Model validation failed (#: required key [Code] not found) [has workaround] Model validation failed (#: required key [Code] not found) [has workaround] May 12, 2023
@kpitzen kpitzen added impact/usability Something that impacts users' ability to use the product easily and intuitively good-first-issue Start here if you'd like to start contributing to Pulumi size/S Estimated effort to complete (1-2 days). area/providers and removed needs-triage Needs attention from the triage team labels May 15, 2023
@kpitzen
Copy link
Contributor

kpitzen commented May 15, 2023

Hi @jakobo - thank you for opening this issue! We really appreciate your diligence in not only providing a reproduction, but also the potential workaround - it seems like a relatively straightforward fix, so we'll try to get this queued up soon.

Thanks again!

@mjeffryes mjeffryes self-assigned this Jul 27, 2023
@mikhailshilkov mikhailshilkov added this to the 0.93 milestone Aug 11, 2023
@mikhailshilkov mikhailshilkov modified the milestones: 0.93, 0.94 Sep 5, 2023
@ebr
Copy link

ebr commented Nov 8, 2023

This causes the function to fail deployment when function code is updated outside of Pulumi (for example, when code is managed by a separate CI workflow). The workaround replaces function code, which is not an option for a production deployment. I also tried pulumi.ResourceOptions(ignore_changes=["code"]), but that doesn't solve the issue.

@ghferrari
Copy link

I am seeing this same error when simply attempting to update the description on an existing lambda function that includes no additional layers.

To reproduce:

# create a simple lambda function
my_lambda_function = aws_native.lambda_.Function(
    "MyLambdaFunction",
    description="Simple lambda function",
    code=aws_native.lambda_.FunctionCodeArgs(
        zip_file="return {'message': 'success'}"
    ),
    runtime="python3.12",
    handler="index.handler"
)

Deploying the above as normal with pulumi up should work fine. Then making a change to the text in the description and re-deploy produces this error:

error: operation error CloudControl: UpdateResource, https response error StatusCode: 400, RequestID: <redacted>, api error ValidationException: Model validation failed (#: required key [Code] not found)

I would hazard a guess that the cloudcontrol api is expected pulumi to resend the code i.e. the value of zip_file but that this isn't happening, though I don't know how to verify explanation.

Many thanks for your help.

@misterorion
Copy link

misterorion commented Mar 4, 2024

This is still happening, in my case when code is set to the URI of a container image.

My current workaorund is to update the image tag of the URI to a random string using the Pulumi random package (which also triggers a new image push). Not great.

@paritosh16
Copy link

This is still happening for me, when code is set to a S3 objcet key. I updated the handler and pulumi up refuses to deploy the lambda function (with updated configuration!).

Is there any workaround other than deleting and recreating?

danielrbradley added a commit that referenced this issue Mar 8, 2024
Write-only properties can't even be read internally within the
CloudControl service so they must be included in PATCH requests to
ensure the updated model validates.
- Refactor where we unmarshal inputs to avoid duplicate work.

Fixes #906

Might also fix #1243
but there's no code to reproduce the issue.
@pulumi-bot pulumi-bot added the resolution/fixed This issue was fixed label Mar 8, 2024
@mjeffryes mjeffryes removed their assignment Mar 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/providers good-first-issue Start here if you'd like to start contributing to Pulumi impact/usability Something that impacts users' ability to use the product easily and intuitively kind/bug Some behavior is incorrect or out of spec resolution/fixed This issue was fixed size/S Estimated effort to complete (1-2 days).
Projects
None yet
Development

Successfully merging a pull request may close this issue.