Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support APIGW as ingress for Backend Services #2050

Open
rmarapp-dio opened this issue Mar 12, 2021 · 32 comments
Open

Support APIGW as ingress for Backend Services #2050

rmarapp-dio opened this issue Mar 12, 2021 · 32 comments
Labels
type/feature Issues that are new feature requests.

Comments

@rmarapp-dio
Copy link

Hello,

Is there any guidance or support to implement API gateway to - 'Backend Service' integration. Does copilot have any support to integrate with API Gateway for backend services?
@rmarapp-dio rmarapp-dio mentioned this issue Mar 15, 2021
Closed
@efekarakus efekarakus added the type/feature Issues that are new feature requests. label Mar 15, 2021
@efekarakus
Copy link
Contributor

Hi @rmarapp-prisidio ! We don't have an integration with APIGW available at the moment.

API Gateway for backend services?

Do you want the APIGW to be private? so that the APIGW is accessible only within the VPC or do you want it to be a public endpoint? At the moment the APIGW would need to be created outside of copilot unfortunately

@rmarapp-dio
Copy link
Author

rmarapp-dio commented Mar 15, 2021 via email

@efekarakus efekarakus changed the title Integration with API Gateway and Backend Services Support APIGW as ingress for Backend Services Mar 15, 2021
@efekarakus
Copy link
Contributor

Please let me know if you would like me to put in a feature request.

I updated this issue to be a feature request (renamed it!)

Would you mind sharing with us what features of APIGW you'd like to use with Copilot? 🙏
Any reason why you'd prefer to use APIGW over the existing "Load Balanced Web Service" pattern?

@rmarapp-dio
Copy link
Author

rmarapp-dio commented Mar 15, 2021 via email

@efekarakus
Copy link
Contributor

I think so, thanks! Is there a particular feature of APIGW that allows the RESTful service not to be accessible to the internet? Something like API Keys?

I'm trying to understand which particular features of APIGW we could expose in our manifest files to make it easier for you to integrate with it.

@rmarapp-dio
Copy link
Author

rmarapp-dio commented Mar 15, 2021 via email

@rmarapp-dio
Copy link
Author

rmarapp-dio commented Mar 17, 2021 via email •
edited

@efekarakus
Copy link
Contributor

Thanks Ram! This is great :D

@rmarapp-dio
Copy link
Author

rmarapp-dio commented Mar 17, 2021 via email •
edited

@ceagan
Copy link

ceagan commented Mar 26, 2021

I think the key is https://aws.amazon.com/blogs/architecture/field-notes-integrating-http-apis-with-aws-cloud-map-and-amazon-ecs-services/ with routes secured by Cognito.

@rmarapp-dio
Copy link
Author

rmarapp-dio commented Mar 27, 2021 via email •
edited

@efekarakus
Copy link
Contributor

I think the key is https://aws.amazon.com/blogs/architecture/field-notes-integrating-http-apis-with-aws-cloud-map-and-amazon-ecs-services/ with routes secured by Cognito.

Application Load Balancers also support integration with Cognito would that be something you'd be interested in to configure for existing "Load Balanced Web Services"?

How do y'all use Cognito today? I wonder if we can add a copilot auth init command to create an Amazon Cognito user pool via CloudFormation and attach it to your existing ALB 🤔

Any chance that this would get implemented near-term. We were going to start to implement it, but wanted check with the co-pilot team and see whether there are any plans for co-pilot support.

We really want to integrate with APIGW just like in the post from @ceagan but there is no short term plan for it.
My first thought was that APIGW would instead be an option for "Load Balanced Web Services" if the APIGW is going to be publicly accessible:

type: "Load Balanced Web Service"
api:
   openapi: ./api-definition.json

How would you like this field to be surfaced in the manifest? What options would you like to see?

@rmarapp-dio
Copy link
Author

rmarapp-dio commented Mar 30, 2021 via email •
edited

@rmarapp-dio
Copy link
Author

rmarapp-dio commented Mar 30, 2021 via email •
edited

@efekarakus
Copy link
Contributor

Yup that's right! Copilot never creates security groups that allow ingress from the internet. For Load Balanced Web Services, only through the ALB or other containers in the cluster and it would be similar with the APIGW integration.

@rmarapp-dio
Copy link
Author

rmarapp-dio commented Mar 30, 2021 via email •
edited

@efekarakus
Copy link
Contributor

Right now the ALB is visible publicly for "Load Balanced Web Services", however if we do enable APIGW or ALB for "Backend Services" then they wouldn't be publicly visible.

@rmarapp-dio
Copy link
Author

rmarapp-dio commented Mar 31, 2021 via email •
edited

@ceagan
Copy link

ceagan commented Mar 31, 2021

I think it is important to support authenticated routes with the deployed API Gateway too. They could be configured to use Amazon Cognito for authentication.

It would be great if there was an option to automatically deploy a Swagger-UI for the Copilot-deployed API as well. The Swagger-UI would be configured automatically to support in-browser testing of authenticated routes and allow developers/users of the API to access documentation of the deployed APIs.

@rmarapp-dio
Copy link
Author

@efekarakus Any timeframe on when you think the team might be able to implement this feature, we have a need, and if it takes longer we might have to look at other options. Please let me know your thoughts.

@efekarakus
Copy link
Contributor

Hi @rmarapp-prisidio !

Any timeframe on when you think the team might be able to implement this feature

We don't have currently bandwidth to start on the implementation for this feature any time soon unfortunately. Our near-term roadmap is available here: https://github.com/aws/copilot-cli/projects/2

We were going to start to implement it

If y'all are still available to do it, we would love to have it as a contribution! But before implementation it would great to have a UX design discussion first such as #1154

However, in the mean time, I think it's possible to build this integration outside of Copilot with CloudFormation today although it won't be managed by Copilot.

In the service stack's we are exporting DiscoveryServiceARN and we also export the subnets and security groups created in the environment stack. So I think it should be possible to deploy a stack with APIGatewayV2:

PParameters:
  AppName:
    Type: String
  EnvName:
    Type: String
  SvcName:
    Type: String
Resources:
  CloudMapHttpApi:
    Type: AWS::ApiGatewayV2::Api
    Properties:
      ProtocolType: HTTP

  APIRoute:
    Type: AWS::ApiGatewayV2::Route
    Properties:
      ApiId: !Ref CloudMapHttpApi
      RouteKey: 'ANY /'
      Target: !Join
        - /
        - - integrations
          - !Ref APIIntegration

  PrivateAPIGWvpcLink:  
    Type: AWS::ApiGatewayV2::VpcLink
    Properties: 
      Name: private-apigw-vpclink
      SubnetIds:
      - Fn::Select:
        - 0
        - Fn::Split:
          - ','
          - Fn::ImportValue: !Sub '${AppName}-${EnvName}-PublicSubnets'
      - Fn::Select:
        - 1
        - Fn::Split:
          - ','
          - Fn::ImportValue: !Sub '${AppName}-${EnvName}-PublicSubnets'
      SecurityGroupIds:
      - Fn::ImportValue: !Sub '${AppName}-${EnvName}-EnvironmentSecurityGroup'

  APIIntegration:
    Type: AWS::ApiGatewayV2::Integration
    Properties:
      ApiId: !Ref CloudMapHttpApi
      IntegrationType: HTTP_PROXY
      IntegrationMethod: ANY
      ConnectionType: VPC_LINK
      ConnectionId: !Ref PrivateAPIGWvpcLink
      IntegrationUri: 
      - Fn::ImportValue: !Sub '${AppName}-${EnvName}-${SvcName}-DiscoveryServiceARN'
      PayloadFormatVersion: '1.0'  
  
  APIStage:
    Type: AWS::ApiGatewayV2::Stage
    Properties:
      StageName: $default
      AutoDeploy: true
      ApiId: !Ref CloudMapHttpApi

Outputs:
  APIURL:
    Description: Invoke URL
    Value: !Sub https://${CloudMapHttpApi}.execute-api.${AWS::Region}.amazonaws.com/

Hope this helps!

@sekamaneka
Copy link

I have actually implemented this as an addon.

Parameters:
  App:
    Type: String
    Description: Your application's name.
  Env:
    Type: String
    Description: The environment name your service, job, or workflow is being deployed to.
  Name:
    Type: String
    Description: The name of the service, job, or workflow being deployed.

Resources:
  ApiGatewayV2Api:
    Type: "AWS::ApiGatewayV2::Api"
    Properties:
      Name: !Sub "${Name}.${Env}.${App}.example.com"
      ApiKeySelectionExpression: "$request.header.x-api-key"
      ProtocolType: "HTTP"
      RouteSelectionExpression: "$request.method $request.path"
      CorsConfiguration:
        AllowCredentials: false
        AllowHeaders:
          - "*"
        AllowMethods:
          - "*"
        AllowOrigins:
          - "*"
        ExposeHeaders:
          - "*"
        MaxAge: 0
      DisableExecuteApiEndpoint: false

  ApiGatewayV2Stage:
    Type: "AWS::ApiGatewayV2::Stage"
    Properties:
      StageName: "$default"
      ApiId: !Ref ApiGatewayV2Api
      AutoDeploy: true

  ApiGatewayV2Authorizer:
    Type: "AWS::ApiGatewayV2::Authorizer"
    Properties:
      ApiId: !Ref ApiGatewayV2Api
      AuthorizerType: "JWT"
      IdentitySource:
        - "$request.header.AuthorizationJWT"
      Name: "cognito"
      JwtConfiguration:
        Audience:
          - !Ref UserPoolClient
        Issuer: !Sub
          - "https://cognito-idp.${AWS::Region}.amazonaws.com/${PoolID}"
          - { PoolID: !Ref MyUserPool }

  MyCert:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName: !Sub "${Name}.${Env}.${App}.example.com"
      DomainValidationOptions:
        - DomainName: !Sub "${Name}.${Env}.${App}.example.com"
          HostedZoneId:
            Fn::ImportValue: !Sub "${App}-${Env}-HostedZone"
      ValidationMethod: DNS

  MyUserPool:
    Type: AWS::Cognito::UserPool
    Properties:
      UserPoolName: !Sub "${Name}.${Env}.${App}"

  UserPoolClient:
    Type: "AWS::Cognito::UserPoolClient"
    DependsOn: UserPoolResourceServer
    Properties:
      ClientName: !Sub "${Name}.${Env}.${App}"
      GenerateSecret: true
      UserPoolId: !Ref MyUserPool
      AllowedOAuthFlows:
        - client_credentials
      AllowedOAuthFlowsUserPoolClient: true
      SupportedIdentityProviders:
        - COGNITO
      AllowedOAuthScopes:
        - !Sub "${Name}.${Env}.${App}/api.readwrite"

  UserPoolDomain:
    Type: AWS::Cognito::UserPoolDomain
    Properties:
      Domain: !Sub "${Name}-${Env}-${App}"
      UserPoolId: !Ref MyUserPool

  UserPoolResourceServer:
    Type: AWS::Cognito::UserPoolResourceServer
    Properties:
      UserPoolId: !Ref MyUserPool
      Identifier: !Sub "${Name}.${Env}.${App}"
      Name: !Sub "${Name}.${Env}.${App}"
      Scopes:
        - ScopeName: "api.readwrite"
          ScopeDescription: "All access"

  MyApiMapping:
    Type: "AWS::ApiGatewayV2::ApiMapping"
    DependsOn: MyDomainName
    Properties:
      DomainName: !Sub "${Name}.${Env}.${App}.example.com"
      ApiId: !Ref ApiGatewayV2Api
      Stage: !Ref ApiGatewayV2Stage

  MyDomainName:
    Type: "AWS::ApiGatewayV2::DomainName"
    Properties:
      DomainName: !Sub "${Name}.${Env}.${App}.example.com"
      DomainNameConfigurations:
        - EndpointType: REGIONAL
          CertificateArn: !Ref MyCert
          CertificateName: !Sub "${Name}.${Env}.${App}.example.com"

  myDNSRecord:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId:
        Fn::ImportValue: !Sub "${App}-${Env}-HostedZone"
      Name: !Sub "${Name}.${Env}.${App}.example.com"
      AliasTarget:
        HostedZoneId: !GetAtt MyDomainName.RegionalHostedZoneId
        DNSName: !GetAtt MyDomainName.RegionalDomainName
        EvaluateTargetHealth: true
      Type: A

  MyVpcLink:
    Type: AWS::ApiGatewayV2::VpcLink
    Properties:
      Name: !Sub "${App}-${Env}-${Name}"
      # Get exported ids from the parent stack that creates the VPC
      SubnetIds:
        !Split [",", Fn::ImportValue: !Sub "${App}-${Env}-PublicSubnets"]
      SecurityGroupIds:
        - Fn::ImportValue: !Sub "${App}-${Env}-EnvironmentSecurityGroup"

  ApiGatewayV2Integration:
    Type: "AWS::ApiGatewayV2::Integration"
    Properties:
      ApiId: !Ref ApiGatewayV2Api
      ConnectionId: !Ref MyVpcLink
      ConnectionType: "VPC_LINK"
      IntegrationMethod: "ANY"
      IntegrationType: "HTTP_PROXY"
      IntegrationUri: 
        Fn::ImportValue: !Sub "${App}-${Env}-${Name}-DiscoveryServiceARN"
      TimeoutInMillis: 30000
      PayloadFormatVersion: "1.0"

  ApiGatewayV2Route:
    Type: "AWS::ApiGatewayV2::Route"
    Properties:
      ApiId: !Ref ApiGatewayV2Api
      AuthorizerId: !Ref ApiGatewayV2Authorizer
      AuthorizationType: "JWT"
      # allow to all endpoints behind it
      RouteKey: "$default"
      Target: !Sub "integrations/${ApiGatewayV2Integration}"

@efekarakus
Copy link
Contributor

Wow this is amazing thanks for sharing it @swilso793 ! 🤩

@rmarapp-dio
Copy link
Author

rmarapp-dio commented Apr 3, 2021 via email

@rmarapp-dio
Copy link
Author

I did the following

  1. Changed my service from 'Loadbalanced web service' to 'Backend Service'
  2. Made minor modifications to the template file provided by @swilso793 and dropped it into the addons folder I keep getting the following error

No export named vault-rmar-dev-jasper-svc-DiscoveryServiceARN found. Rollback requested by user.

Any idea what I'm doing wrong? Any help would be greatly appreciated.

Thanks
image

@sekamaneka
Copy link

sekamaneka commented Apr 4, 2021 •
edited

I have patched copilot to make the namespaces differ based on environment.. You can check the issue out here.

#1929

Also:

copilot/api/addons/api-gateway.yaml:ApiGatewayV2Integration
copilot/api/addons/api-gateway.yaml:ApiGatewayV2Route

Depend on the output of the parent stack. so they have to be commented out when first creating a service
Then run again with them enabled

@rmarapp-dio
Copy link
Author

Thanks @swilso793. I've got most of it working.
In terms of the use-case, I'm using a custom lambda authorizer and from the authorizer, I needed to call a co-pilot backend service. The lambda is not within the VPC so what is dns entry that I must use to invoke the back-end service. Or I need to set up an internal network loadbalancer and add fargate target groups to it etc.

@efekarakus, @swilso793 Is there an easy way to do this, or do I have to create an NLB.
Thank you fo all the assitance. Super helpful.

I have patched copilot to make the namespaces differ based on environment.. You can check the issue out here.

#1929

Also:

copilot/api/addons/api-gateway.yaml:ApiGatewayV2Integration
copilot/api/addons/api-gateway.yaml:ApiGatewayV2Route

Depend on the output of the parent stack. so they have to be commented out when first creating a service
Then run again with them enabled

@efekarakus
Copy link
Contributor

Sharing a tutorial here for how to setup http apis and ECS services: https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-private-integration.html

@rmarapp-dio
Copy link
Author

rmarapp-dio commented May 13, 2021 via email

@efekarakus efekarakus mentioned this issue Jul 12, 2021
Open
@huanjani huanjani mentioned this issue Mar 16, 2022
Closed
@dave-moser
Copy link
Contributor

dave-moser commented Jul 19, 2022 •
edited

Creating an API Gateway using the addon template works great. However, when I attempt to delete the copilot service that contains the addon, I get a Cloudformation error and the stack deletion cancels.

Export todo-dev-api-DiscoveryServiceARN cannot be deleted as it is in use by todo-dev-api-AddonsStack-1VK70SO7IGC4B

Shouldn't the copilot svc delete command also delete the addons nested stack?

Has anyone encountered this? The only workaround I have so far is to remove the addon folder, redeploy the copilot service - which removes the API Gateway, then delete the copilot service.

I'm curious if anyone else who added an API Gateway via an addon was able to delete the parent service.

Thanks!

@efekarakus
Copy link
Contributor

Hi @dave-moser ! Apologies for the late response 🙇

Shouldn't the copilot svc delete command also delete the addons nested stack?

It should have! I think the reason is that the Addons stack is using the IntegrationURI using an Fn::Import in the template above:

      IntegrationUri: 
        Fn::ImportValue: !Sub "${App}-${Env}-${Name}-DiscoveryServiceARN"

This means that the addons was created in two steps:

  1. Deploy the service without an addons/ template.
  2. Deploy with addons so that it can use the import.

So deletion would also require the reverse two-steps.

I believe the above template was written before Copilot introduced the addons.parameters.yml feature: https://aws.github.io/copilot-cli/docs/developing/additional-aws-resources/#customizing-the-parameters-section

So instead of using ImportValue on a service stack output, if we re-write it in the addons.parameters.yml as:

Parameters:
   DiscoveryServiceARN:  !GetAtt DiscoveryService.Arn

Then I believe that you would need this 2-step process for deletion. CloudFormation will delete the addons stack first and then the Service Discovery Namespace.

@dave-moser
Copy link
Contributor

dave-moser commented Jul 22, 2022 •
edited

Thanks! This worked perfectly. I can now delete the Copilot app in one step.

@takashi-uchida takashi-uchida mentioned this issue Apr 14, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/feature Issues that are new feature requests.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@efekarakus @ceagan @sekamaneka @dave-moser @rmarapp-dio