Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to create V2 Replication Configuration w/ S3 Replication Time Control on S3 buckets #1910

Closed
dsciacca opened this issue Jun 2, 2021 · 1 comment

Comments

@dsciacca
Copy link

dsciacca commented Jun 2, 2021

In order to set S3 Replication Time Control (S3 RTC) on a bucket we must use V2 of the Replication Configuration which replaces the Prefix property with the Filter property (see AWS Docs on Replication Configuration - Backward Compatibility).

As troposphere.s3 is defined in v2.7.1 the ReplicationConfigurationRules class requires the Prefix property, leaving this out in lieu of the Filter property results in the following error:

Traceback (most recent call last):
  File "s3_automation.py", line 957, in <module>
    main()
  File "s3_automation.py", line 158, in main
    template_output = template.to_json()
  File "/Users/dsciac105/dev/src/dx/dl/dl-acct-automation/venv/lib/python3.6/site-packages/troposphere/__init__.py", line 795, in to_json
    return json.dumps(self.to_dict(), indent=indent,
  File "/Users/dsciac105/dev/src/dx/dl/dl-acct-automation/venv/lib/python3.6/site-packages/troposphere/__init__.py", line 746, in to_dict
    return encode_to_dict(t)
  File "/Users/dsciac105/dev/src/dx/dl/dl-acct-automation/venv/lib/python3.6/site-packages/troposphere/__init__.py", line 66, in encode_to_dict
    props[name] = encode_to_dict(prop)
  File "/Users/dsciac105/dev/src/dx/dl/dl-acct-automation/venv/lib/python3.6/site-packages/troposphere/__init__.py", line 66, in encode_to_dict
    props[name] = encode_to_dict(prop)
  File "/Users/dsciac105/dev/src/dx/dl/dl-acct-automation/venv/lib/python3.6/site-packages/troposphere/__init__.py", line 57, in encode_to_dict
    return encode_to_dict(obj.to_dict())
  File "/Users/dsciac105/dev/src/dx/dl/dl-acct-automation/venv/lib/python3.6/site-packages/troposphere/__init__.py", line 257, in to_dict
    return encode_to_dict(self.resource)
  File "/Users/dsciac105/dev/src/dx/dl/dl-acct-automation/venv/lib/python3.6/site-packages/troposphere/__init__.py", line 66, in encode_to_dict
    props[name] = encode_to_dict(prop)
  File "/Users/dsciac105/dev/src/dx/dl/dl-acct-automation/venv/lib/python3.6/site-packages/troposphere/__init__.py", line 66, in encode_to_dict
    props[name] = encode_to_dict(prop)
  File "/Users/dsciac105/dev/src/dx/dl/dl-acct-automation/venv/lib/python3.6/site-packages/troposphere/__init__.py", line 57, in encode_to_dict
    return encode_to_dict(obj.to_dict())
  File "/Users/dsciac105/dev/src/dx/dl/dl-acct-automation/venv/lib/python3.6/site-packages/troposphere/__init__.py", line 257, in to_dict
    return encode_to_dict(self.resource)
  File "/Users/dsciac105/dev/src/dx/dl/dl-acct-automation/venv/lib/python3.6/site-packages/troposphere/__init__.py", line 66, in encode_to_dict
    props[name] = encode_to_dict(prop)
  File "/Users/dsciac105/dev/src/dx/dl/dl-acct-automation/venv/lib/python3.6/site-packages/troposphere/__init__.py", line 61, in encode_to_dict
    new_lst.append(encode_to_dict(o))
  File "/Users/dsciac105/dev/src/dx/dl/dl-acct-automation/venv/lib/python3.6/site-packages/troposphere/__init__.py", line 57, in encode_to_dict
    return encode_to_dict(obj.to_dict())
  File "/Users/dsciac105/dev/src/dx/dl/dl-acct-automation/venv/lib/python3.6/site-packages/troposphere/__init__.py", line 253, in to_dict
    self._validate_props()
  File "/Users/dsciac105/dev/src/dx/dl/dl-acct-automation/venv/lib/python3.6/site-packages/troposphere/__init__.py", line 318, in _validate_props
    raise ValueError(msg)
ValueError: Resource Prefix required in type <unknown type>

However, if I include both Prefix and Filter properties, the stack gets stuck in the UPDATE_IN_PROGRESS state and never completes the update.

I was able to get around this by editing the troposphere.s3 module to set the Prefix property of the ReplicationConfigurationRules class to optional as follows and only including the Filter property in place of the Prefix property in my code base:

class ReplicationConfigurationRules(AWSProperty):
    props = {
        'DeleteMarkerReplication': (DeleteMarkerReplication, False),
        'Destination': (ReplicationConfigurationRulesDestination, True),
        'Filter': (ReplicationRuleFilter, False),
        'Id': (str, False),
        'Prefix': (str, False),
        'Priority': (integer, False),
        'SourceSelectionCriteria': (SourceSelectionCriteria, False),
        'Status': (str, True)
    }

If this approach is a satisfactory solution I'm happy to create a PR w/ the fix, but creating this in case it's better addressed through some other means.

@markpeek
Copy link
Member

markpeek commented Jul 4, 2021

Fixed. Thanks for the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants