Allow options like metrics, logs, tags, and tracing to work with externally defined Rest APIs for the AWS provider

[dls314]

Is there an existing issue for this?

• I have searched existing issues, it hasn't been reported yet

Use case description

I would like to use some of the AWS provider's features for Rest APIs that are managed through updateStage for externally defined Rest APIs.

The protection in updateStage that guards against any of these features being used when the Rest API is externally defined seems very blunt given that some (perhaps all or most) of the features are stage-specific.

I would like to be able to enable all these options for an externally defined Rest API.

I am creating a basic Rest API and logging role externally so that the Rest API id remains stable even if the serverless project's stack is torn down, so that consumers can use the Rest API id, for example to create IAM policies. However, all other management of the Rest API is done through the serverless project.

Proposed solution (optional)

Make everything work normally for externally defined Rest APIs and add a note to the documentation about items that impact the Rest API as a whole versus those that impact just the stage

[medikoo]

The protection in updateStage that guards against any of these features being used when the Rest API is externally defined seems very blunt given that some (perhaps all or most) of the features are stage-specific.

When external Rest API is referenced, it doesn't mean that the service owns the stage, it means that the service just owns the endpoints it defines. For that reason, we do not allow updating any stage-wide settings. Those should be set where the stage is deployed.

There is a pattern of users relying on external Rest API defined in root service and having different services for different endpoint sets.

Having that, all stage settings are managed in the root service.

[dls314]

Those should be set where the stage is deployed.

I'm not sure I understand, my Rest API and logging role are deployed by a CDK application, and the stage is defined and deployed by the serverless project (there's only one)

[medikoo]

and the stage is defined and deployed by the serverless project (there's only one)

Indeed, I missed that stage is actually deployed with the endpoints, and it's perfectly ok to deploy different endpoints to the same stage by different services (stage deployments won't remove endpoints deployed by another service with the previous stage deployment)

If I remember correctly, deploying stage settings were turned off for such cases, as such deployments were overriding the settings set by prior service. It was quite bad, as e.g. if service A had tracing turned on, and service B didn't have any configuration for it, deployment of service B was turning tracing off.

Besides turning off such a possibility, such logic was improved as currently, we do not attempt to touch settings if there's no configuration. So it may seem safe to re-enable it. Still, a problem remains if we decide to override settings as set by the other service you won't get any warning (while if you try to override endpoint managed by the other service deployment will crash - that's well guarded)

Other important note: such settings are updated via direct AWS SDK calls (it's not part of CF template), so probably the best approach would be if you update it manually in AWS Console

[dls314]

Thank you for looking! :-)

Still, a problem remains if we decide to override settings as set by the other service you won't get any warning

I can't see a way to test for this, and I think I understand the concern (but I'm not sure which settings, or whether all of them are potentially in conflict with other serverless services).

At the moment, I've had to abandon using an external Rest API definition, because having logs and metrics is more important than the maintainability wins that I was trying to achieve there.

I'm still hopeful that there's a way forward here, my alternative is waiting until I can afford the maintenance cycles for moving the entire API Gateway definition out of the serverless framework, and I had hoped to avoid that.