Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cloud Function is not updated when BucketObject changes #370

Open
vflopes opened this issue Mar 6, 2022 · 10 comments
Open

Cloud Function is not updated when BucketObject changes #370

vflopes opened this issue Mar 6, 2022 · 10 comments
Labels
kind/bug Some behavior is incorrect or out of spec needs-repro Needs repro steps before it can be triaged or fixed

Comments

@vflopes
Copy link

vflopes commented Mar 6, 2022

Hello!

  • Vote on this issue by adding a 馃憤 reaction
  • To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already)

Issue details

Changing the code triggers the update for the BucketObject, but the function is not updated as it should be.

Steps to reproduce

My code (bundled):

import { ComponentResource, ComponentResourceOptions, asset, interpolate } from "@pulumi/pulumi";
import * as fs from 'fs/promises';
import * as functionsv1 from '@pulumi/google-native/cloudfunctions/v1/index.js';
import * as storagev1 from '@pulumi/google-native/storage/v1/index.js';

export interface PackageJson {
    type?: 'module' | 'commonjs';
}

export async function producePackageJson(excludedPackages: string[], externalPackageJson?: PackageJson): Promise<string> {
    const packageJson = JSON.parse(await fs.readFile('package.json', 'utf-8'));

    return JSON.stringify({
        dependencies: {
            ...Object.keys(packageJson.dependencies)
            .filter(pkg => !excludedPackages.includes(pkg) && !pkg.startsWith('@pulumi'))
            .map((pkg) => ({[pkg]: packageJson.dependencies[pkg]}))
            .reduce((prev, curr) => ({...prev, ...curr})),
        },
        ...(externalPackageJson || {}),
    });
}

export interface FunctionV1Args extends functionsv1.FunctionArgs {

    bucket?: storagev1.Bucket;
    location?: string;
    assets: asset.AssetMap | Promise <asset.AssetMap>;
    packageJson?: PackageJson;

}
export class FunctionV1 extends ComponentResource {

    public readonly function: functionsv1.Function;
    public readonly bucket: storagev1.Bucket;
    public readonly bucketObject: storagev1.BucketObject;

    constructor(name: string, args: FunctionV1Args, opts?: ComponentResourceOptions) {
        
        opts = opts || {};

        super("google-native:cloudfunctions/v1:Function", name, undefined, opts);

        const parentOpts = { parent: this };

        const bucketName = name.toLowerCase().replace(/[^-_a-z0-9]/gi, "-");
        
        this.bucket = args.bucket || new storagev1.Bucket(bucketName, {
            project: args.project,
            location: args.location || "US",
            storageClass: 'STANDARD',
            iamConfiguration: {
                publicAccessPrevention: 'enforced',
            },
        }, parentOpts);


        this.bucketObject = new storagev1.BucketObject(`${name}`, {
            bucket: this.bucket.name,
            source: new asset.AssetArchive(args.assets),
        }, parentOpts);

        const argsCopy = {...args};

        delete argsCopy.assets;

        const functionName = name.replace(/[^-_a-z0-9]/gi, "-");

        this.function = new functionsv1.Function(
            functionName,
            {
                runtime: 'nodejs16',
                sourceArchiveUrl: interpolate`gs://${this.bucket.name}/${this.bucketObject.name}`,
                ...argsCopy,
            },
            parentOpts,
        );

    }
}

const assetFunction = new FunctionV1(
    'asset-function',
    {
        entryPoint: 'handler',
        runtime: 'nodejs16',
        availableMemoryMb: 128,
        environmentVariables: {},
        maxInstances: 4,
        serviceAccountEmail: serviceAccount.email,
        timeout: '30s',
        eventTrigger: {
            eventType: 'providers/cloud.storage/eventTypes/object.change',
            resource: interpolate`projects/${gcpProjectId}/buckets/${assetBucket.name}`,
        },
        assets: (async (): Promise<asset.AssetMap> => {

            return {
                'package.json': new asset.StringAsset(await producePackageJson(
                    [
                    'services-libs',
                    ],
                    {
                        type: 'module',
                    },
                )),
                'index.js': new asset.FileAsset('bundle/asset-callback/index.js'),
            };

        })(),
    },
);
  1. Any code changes to bundle/asset-callback/index.js triggers changes on BucketObject but the function don't catch the update:
    image

  2. Versions

pulumi cli: v3.25.1

NodeJS dependencies:

    "@pulumi/google-native": "^0.15.0"
    "@pulumi/pulumi": "^3.25.0"

PS: With the GCP (Classic) provider, I'm getting the correct behavior, and this issue is somehow related to #177

Expected: When the code is updated, the function must be deployed with it's new content from bucket
Actual: The BucketObjcet is updated, but the function keeps the same previous code.

@vflopes vflopes added the kind/bug Some behavior is incorrect or out of spec label Mar 6, 2022
@vflopes
Copy link
Author

vflopes commented Mar 6, 2022

Using the Bucket/BucketObject from GCP (Classic) provider does trigger the function update:

image

@viveklak viveklak added the needs-repro Needs repro steps before it can be triaged or fixed label Mar 10, 2022
@antstanley
Copy link

I had the same issue. I ended up changing the name of the of the Bucket Object on each deployment. The downside is it deploys a new object with every deployment, whether the function code has changed or not.

@lephuongbg
Copy link

lephuongbg commented Apr 13, 2022

This looks like a bug of the BucketObject class itself. On the cloud console, even though "Last modified" timestamp was updated upon deploy, if you download the object and check the content, the content was still from the first time it was uploaded and did not change on subsequent deployment.

Found out that the classic provider has behavior of replacing the object when content changes, but the native provider does not.

Workaround: add replace_on_changes: true option on the bucket object's resource options should make the deployment work as expected, like in classic provider.

@pcgilday
Copy link

I am able to repro this and seeing the same thing @lephuongbg mentioned. What's the best way to help resolve this issue? I'm new to the pulumi codebase, but am open to contributing if someone can point me in the right direction. Thanks!

@banool
Copy link

banool commented Feb 24, 2023

Not sure if related but I'm seeing a similar with the classic provider. Unlike with this issue, the BucketObject changing does trigger an update of the Function, but after the update the Function is still using the old code. I have been able to fix it by setting replaceOnChanges: ["*"] for both resources.

@maticortesr
Copy link

This looks like a bug of the BucketObject class itself. On the cloud console, even though "Last modified" timestamp was updated upon deploy, if you download the object and check the content, the content was still from the first time it was uploaded and did not change on subsequent deployment.

Found out that the classic provider has behavior of replacing the object when content changes, but the native provider does not.

Workaround: add replace_on_changes: true option on the bucket object's resource options should make the deployment work as expected, like in classic provider.

Also seeing this but that option is not available on Python

@maticortesr
Copy link

temporary workaround: run pulumi up --replace urn_of_resource. You can find the urn running pulumi stack export. I have a fixed Resource Name so I had to also add delete_before_replace=True to the cloud function, which means some downtime.

@avetkhov
Copy link

Additional information. Do not set up the name parameter of a bucket object due to replaceOnChanges will not effect in this case.

@ecmonsen
Copy link

replace_on_changes = ["*"] on the BucketObject and on the Function object does not fix this for me. I'm using Python.

@bunert
Copy link

bunert commented Nov 16, 2023

Any update on this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Some behavior is incorrect or out of spec needs-repro Needs repro steps before it can be triaged or fixed
Projects
None yet
Development

No branches or pull requests

10 participants