Catch invalid regular expressions in the configuration #125

Closed
OXYAMINE opened this Issue Mar 11, 2014 · 10 comments

Projects

None yet

2 participants

@OXYAMINE

Daemon doesn't start.

Running command:

dynamic-dynamodb -c /etc/dynamic_dynamoDB.conf --log-config-file dynamic_dynamoDB_logging.conf --daemon foreground

Output is:

2014-03-11 00:54:46,691 - dynamic-dynamodb - INFO - Test_Item - Consumed read units: 12%
2014-03-11 00:54:46,714 - dynamic-dynamodb - INFO - Test_Item - Read throttle count: 0
2014-03-11 00:54:46,772 - dynamic-dynamodb - INFO - Test_Item - Consumed write units: 0%
2014-03-11 00:54:46,790 - dynamic-dynamodb - INFO - Test_Item - Write throttle count: 0
2014-03-11 00:54:46,790 - dynamic-dynamodb - INFO - Test_Item - No need to change provisioning
Traceback (most recent call last):
  File "/usr/local/bin/dynamic-dynamodb", line 27, in <module>
    dynamic_dynamodb.main()
  File "/usr/local/lib/python2.6/dist-packages/dynamic_dynamodb/__init__.py", line 128, in main
    check_interval=config['global']['check_interval'])
  File "/usr/local/lib/python2.6/dist-packages/dynamic_dynamodb/__init__.py", line 52, in run
    if re.match(gsi_key, gsi_name):
  File "/usr/lib/python2.6/re.py", line 137, in match
    return _compile(pattern, flags).match(string)
  File "/usr/lib/python2.6/re.py", line 245, in _compile
    raise error, v # invalid expression
sre_constants.error: nothing to repeat

What is this?

@sebdah
Owner
sebdah commented Mar 11, 2014

Please attach your configuration. I would assume that you have an invalid
regular expression in it, given the last rows in the exception.
On Mar 11, 2014 5:56 AM, "OXYAMINE" notifications@github.com wrote:

Daemon doesn't start.

Running command:
dynamic-dynamodb -c /etc/dynamic_dynamoDB.conf --log-config-file
dynamic_dynamoDB_logging.conf --daemon foreground

Output is:
2014-03-11 00:54:46,691 - dynamic-dynamodb - INFO - Test_Item - Consumed
read units: 12%
2014-03-11 00:54:46,714 - dynamic-dynamodb - INFO - Test_Item - Read
throttle count: 0
2014-03-11 00:54:46,772 - dynamic-dynamodb - INFO - Test_Item - Consumed
write units: 0%
2014-03-11 00:54:46,790 - dynamic-dynamodb - INFO - Test_Item - Write
throttle count: 0
2014-03-11 00:54:46,790 - dynamic-dynamodb - INFO - Test_Item - No need to
change provisioning
Traceback (most recent call last):
File "/usr/local/bin/dynamic-dynamodb", line 27, in
dynamic_dynamodb.main()
File "/usr/local/lib/python2.6/dist-packages/dynamic_dynamodb/init.py",
line 128, in main
check_interval=config['global']['check_interval'])
File "/usr/local/lib/python2.6/dist-packages/dynamic_dynamodb/init.py",
line 52, in run
if re.match(gsi_key, gsi_name):
File "/usr/lib/python2.6/re.py", line 137, in match
return

_compile(pattern, flags).match(string) File "/usr/lib/python2.6/re.py",
line 245, in _compile raise error, v # invalid expression sre_constants.error:
nothing to repeat

What is this?

Reply to this email directly or view it on GitHubhttps://github.com/sebdah/dynamic-dynamodb/issues/125
.

@OXYAMINE
[global]
# AWS access keys
aws-access-key-id: removed
aws-secret-access-key-id: removed

# AWS region to use
region: us-east-1

# How often should Dynamic DynamoDB monitor changes (in seconds)
check-interval: 300

# Circuit breaker configuration
# No provisioning updates will be made unless this URL returns
# a HTTP 200 OK status code
#circuit-breaker-url: http://my.service.com/v1/is_up
#circuit-breaker-timeout: 500

[logging]
# Log level [debug|info|warning|error]
log-level: info

# Log file (comment out to get only console output)
log-file: /var/log/dynamic-dynamodb.log

# External Python logging configuration file
# Overrides both log-level and log-file
# log-config-file: /path/to/logging.conf

[table: Test_Item]
#
# Read provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
reads-upper-threshold: 90
reads-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-reads-with: 10
decrease-reads-with: 10

# Units to increase or decrease reads with, must be either percent or units
increase-reads-unit: percent
decrease-reads-unit: percent

# Maximum and minimum read provisioning
# Dynamic DynamoDB will not provision any more or less reads than this
min-provisioned-reads: 5
max-provisioned-reads: 500

#
# Write provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
writes-upper-threshold: 90
writes-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-writes-with: 10
decrease-writes-with: 10

# Units to increase or decrease writes with, must be either percent or units
increase-writes-unit: percent
decrease-writes-unit: percent

# Maximum and minimum write provisioning
# Dynamic DynamoDB will not provision any more or less writes than this
min-provisioned-writes: 5
max-provisioned-writes: 500

#
# Maintenance windows (in UTC)
#
#maintenance-windows: 22:00-23:59,00:00-06:00

#
# Other settings
#

# Allow down scaling when at 0% consumed reads
allow-scaling-down-reads-on-0-percent: true
allow-scaling-down-writes-on-0-percent: true

# Restric scale down to only happen when BOTH reads AND writes are in need
# of scaling down. Set this to "true" to minimize down scaling.
#always-decrease-rw-together: true

[gsi: * table: Test_Item]
#
# Read provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
reads-upper-threshold: 90
reads-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-reads-with: 50
decrease-reads-with: 50

# Units to increase or decrease reads with, must be either percent or units
increase-reads-unit: percent
decrease-reads-unit: percent

# Maximum and minimum read provisioning
# Dynamic DynamoDB will not provision any more or less reads than this
min-provisioned-reads: 4
max-provisioned-reads: 500

#
# Write provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
writes-upper-threshold: 90
writes-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-writes-with: 50
decrease-writes-with: 50

# Units to increase or decrease writes with, must be either percent or units
increase-writes-unit: percent
decrease-writes-unit: percent

# Maximum and minimum write provisioning
# Dynamic DynamoDB will not provision any more or less writes than this
min-provisioned-writes: 4
max-provisioned-writes: 500

#
# Maintenance windows (in UTC)
#
#maintenance-windows: 22:00-23:59,00:00-06:00

#
# Other settings
#

# Allow down scaling when at 0% consumed reads
allow-scaling-down-reads-on-0-percent: true
allow-scaling-down-writes-on-0-percent: true

# Restric scale down to only happen when BOTH reads AND writes are in need
# of scaling down. Set this to "true" to minimize down scaling.
#always-decrease-rw-together: true
@OXYAMINE
[global]
# AWS access keys
aws-access-key-id: XXX
aws-secret-access-key-id: XXXX

# AWS region to use
region: us-east-1

# How often should Dynamic DynamoDB monitor changes (in seconds)
check-interval: 300

# Circuit breaker configuration
# No provisioning updates will be made unless this URL returns
# a HTTP 200 OK status code
#circuit-breaker-url: http://my.service.com/v1/is_up
#circuit-breaker-timeout: 500

[logging]
# Log level [debug|info|warning|error]
log-level: info

# Log file (comment out to get only console output)
log-file: /var/log/dynamic-dynamodb.log

# External Python logging configuration file
# Overrides both log-level and log-file
# log-config-file: /path/to/logging.conf

[table: Test_Item]
#
# Read provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
reads-upper-threshold: 90
reads-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-reads-with: 10
decrease-reads-with: 10

# Units to increase or decrease reads with, must be either percent or units
increase-reads-unit: percent
decrease-reads-unit: percent

# Maximum and minimum read provisioning
# Dynamic DynamoDB will not provision any more or less reads than this
min-provisioned-reads: 5
max-provisioned-reads: 500

#
# Write provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
writes-upper-threshold: 90
writes-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-writes-with: 10
decrease-writes-with: 10

# Units to increase or decrease writes with, must be either percent or units
increase-writes-unit: percent
decrease-writes-unit: percent

# Maximum and minimum write provisioning
# Dynamic DynamoDB will not provision any more or less writes than this
min-provisioned-writes: 5
max-provisioned-writes: 500

#
# Maintenance windows (in UTC)
#
#maintenance-windows: 22:00-23:59,00:00-06:00

#
# Other settings
#

# Allow down scaling when at 0% consumed reads
allow-scaling-down-reads-on-0-percent: true
allow-scaling-down-writes-on-0-percent: true

# Restric scale down to only happen when BOTH reads AND writes are in need
# of scaling down. Set this to "true" to minimize down scaling.
#always-decrease-rw-together: true

[gsi: * table: Test_Item]
#
# Read provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
reads-upper-threshold: 90
reads-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-reads-with: 50
decrease-reads-with: 50

# Units to increase or decrease reads with, must be either percent or units
increase-reads-unit: percent
decrease-reads-unit: percent

# Maximum and minimum read provisioning
# Dynamic DynamoDB will not provision any more or less reads than this
min-provisioned-reads: 4
max-provisioned-reads: 500

#
# Write provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
writes-upper-threshold: 90
writes-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-writes-with: 50
decrease-writes-with: 50

# Units to increase or decrease writes with, must be either percent or units
increase-writes-unit: percent
decrease-writes-unit: percent

# Maximum and minimum write provisioning
# Dynamic DynamoDB will not provision any more or less writes than this
min-provisioned-writes: 4
max-provisioned-writes: 500

#
# Maintenance windows (in UTC)
#
#maintenance-windows: 22:00-23:59,00:00-06:00

#
# Other settings
#

# Allow down scaling when at 0% consumed reads
allow-scaling-down-reads-on-0-percent: true
allow-scaling-down-writes-on-0-percent: true

# Restric scale down to only happen when BOTH reads AND writes are in need
# of scaling down. Set this to "true" to minimize down scaling.
#always-decrease-rw-together: true
@OXYAMINE

OMG! why the font is so large?
anyway... it's just a test config.
Table - Test_item, all indexes

@OXYAMINE

you're 100% right!
changing * to .* seems solve the issue.
Could you consider to check config before execution and output user friendly message?
I feel myself stupid a little bit :)

@sebdah
Owner
sebdah commented Mar 11, 2014

Alright :).

I'll definitely add a more human readable error message.

Happy hacking and thanks for the report!
On Mar 11, 2014 6:21 AM, "OXYAMINE" notifications@github.com wrote:

you're 100% right!
changing * to .* seems solve the issue.
Could you consider to check config before execution and output user
friendly message?
I feel myself stupid a little bit :)

Reply to this email directly or view it on GitHubhttps://github.com/sebdah/dynamic-dynamodb/issues/125#issuecomment-37264432
.

@sebdah
Owner
sebdah commented Mar 11, 2014

Forgot to say, GitHub uses Markdown for formatting the comments, that's why it looks a bit messed up. I have fixed the formatting above :).

@sebdah sebdah added this to the 1.9.x milestone Mar 11, 2014
@sebdah sebdah self-assigned this Mar 11, 2014
@OXYAMINE

Thanks a lot

@sebdah sebdah added a commit that referenced this issue Mar 17, 2014
@sebdah Updated release notes for #125 71b4a74
@sebdah sebdah closed this Mar 17, 2014
@sebdah
Owner
sebdah commented Mar 17, 2014

This has been fixed in version 1.9.1

@OXYAMINE

thanks a lot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment