ScriptLocation is not updated for Glue::Job when building

[wnkz]

Description

When running sam build ; AWS::Glue::Job Command.ScriptLocation's path is not updated in .aws-sam/build/template.yaml.

My real template includes Serverless::Functions so I need to run build and cannot just use package.
This works just fine for other resources supported by sam package (e.g. AWS::CloudFormation::Stack) but not for Glue Jobs.

Steps to reproduce

template.yml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Resources:
  GlueJob:
    Type: AWS::Glue::Job
    Properties:
      GlueVersion: '1.0'
      MaxCapacity: 5
      Command:
        Name: glueetl
        PythonVersion: '3'
        ScriptLocation: glue.py
      Role: !GetAtt GlueServiceRole.Arn

  GlueServiceRole:
    Type: AWS::IAM::Role
    Properties:
      Path: /service-role/
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - glue.amazonaws.com
            Action:
              - sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole

sam build

Observed result

.aws-sam/build/template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
  GlueJob:
    Type: AWS::Glue::Job
    Properties:
      GlueVersion: '1.0'
      MaxCapacity: 5
      Command:
        Name: glueetl
        PythonVersion: '3'
        ScriptLocation: glue.py
      Role:
        Fn::GetAtt:
        - GlueServiceRole
        - Arn
  GlueServiceRole:
    Type: AWS::IAM::Role
    Properties:
      Path: /service-role/
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - glue.amazonaws.com
          Action:
          - sts:AssumeRole
      ManagedPolicyArns:
      - arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole

Command.ScriptLocation is left untouched.

Expected result

.aws-sam/build/template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
  GlueJob:
    Type: AWS::Glue::Job
    Properties:
      GlueVersion: '1.0'
      MaxCapacity: 5
      Command:
        Name: glueetl
        PythonVersion: '3'
        ScriptLocation: ../../glue.py
      Role:
        Fn::GetAtt:
        - GlueServiceRole
        - Arn
  GlueServiceRole:
    Type: AWS::IAM::Role
    Properties:
      Path: /service-role/
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - glue.amazonaws.com
          Action:
          - sts:AssumeRole
      ManagedPolicyArns:
      - arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole

Command.ScriptLocation should be updated.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: macOS 10.15.1
2. sam --version: SAM CLI, version 0.31.1

[wnkz]

Tracked it down to here https://github.com/awslabs/aws-sam-cli/blob/52708db40a140990e608606c50c3217fd04cf0cd/samcli/commands/_utils/template.py#L135 and here https://github.com/awslabs/aws-sam-cli/blob/52708db40a140990e608606c50c3217fd04cf0cd/samcli/commands/_utils/resources.py#L35

So in this case, SAM runs

path = properties.get("Command.ScriptLocation")

Which obviously isn't a valid key from properties

This has no chance to work as well https://github.com/awslabs/aws-sam-cli/blob/52708db40a140990e608606c50c3217fd04cf0cd/samcli/commands/_utils/template.py#L142

[sanathkr]

Thanks for diving deep and finding the root cause. It should be a simple bugfix to support dot notations in the path_prop_name variables under samcli/commands/_utils/template.py

[wnkz]

@wingkwong Nice changes in #1537 ; had no idea set_value_from_jmespath even existed.
@jfuss Any idea of when this fix will be released?

[awood45]

The fix for this has been released, closing.