The S3 deployment custom resource now properly handles strings containing quotes when used in JSON files, while maintaining the existing behavior for other file types including YAML files.

Issue #22661

Closes #22661 .

Reason for this change

Investigation Findings

1. At the Source.jsonData level, we only have markers, not actual values:

// User code
Source.jsonData('config.json', {
  secret_value: param.stringValue // Token containing "test"with"quotes"
})

// What actually gets written:
{
  "secret_value": "<<marker:0xbaba:0>>" // Marker, not the actual value
}

2. The actual value substitution happens in the S3 deployment Lambda (custom resource), where the markers are replaced with actual values.

BEFORE Fix (Current Broken State)

When the Lambda replaces markers:

// Lambda receives:
{"secret_value":"<<marker:0xbaba:0>>"}
// And the mapping:
markers = {
  "<<marker:0xbaba:0>>": "test"with"quotes"
}

// Simple string replacement results in:
{"secret_value":"test"with"quotes"}  // Invalid JSON

AFTER (With Fix)

The Lambda now detects JSON files and handles them specially:

1. Detects if the file is JSON
2. If JSON:
   • Parses the JSON structure
   • Properly escapes strings during marker replacement
   • Re-serializes to valid JSON
3. If not JSON (including YAML):
   • Uses simple string replacement (which works fine for YAML)

// Lambda receives same input:
{"secret_value":"<<marker:0xbaba:0>>"}
// But now properly escapes quotes in JSON context:
{"secret_value":"test\"with\"quotes"}  // Valid JSON

Key Insights

1. The issue wasn't in the CDK token system but in the marker replacement in the Lambda
2. YAML files work without special handling because YAML is more permissive with quotes
3. The fix:
   • Maintains existing behavior for non-JSON files (YAML, text, etc.)
   • Properly handles JSON string escaping
   • Potential Breaking Change: This fix might cause double-escaping for customers who implemented workarounds (e.g., manually adding escape characters in marker values to handle JSON)

Description of changes

Changes:

• Added JSON detection and special handling in the Lambda custom resource
• Added integration tests for both JSON and YAML files with quoted values
• Added bucket cleanup configuration to the test stack

Describe any new or updated permissions being added

N/A

Description of how you validated changes

Test cases show that:

• JSON files properly escape quotes: {"secret_value": "test\"with\"quotes"}
• YAML files work as-is: secret_value: test"with"quotes

Checklist

• My code adheres to the CONTRIBUTING GUIDE and DESIGN GUIDELINES

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license