New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Catch invalid regular expressions in the configuration #125

Closed
OXYAMINE opened this Issue Mar 11, 2014 · 10 comments

Comments

Projects
None yet
2 participants
@OXYAMINE

OXYAMINE commented Mar 11, 2014

Daemon doesn't start.

Running command:

dynamic-dynamodb -c /etc/dynamic_dynamoDB.conf --log-config-file dynamic_dynamoDB_logging.conf --daemon foreground

Output is:

2014-03-11 00:54:46,691 - dynamic-dynamodb - INFO - Test_Item - Consumed read units: 12%
2014-03-11 00:54:46,714 - dynamic-dynamodb - INFO - Test_Item - Read throttle count: 0
2014-03-11 00:54:46,772 - dynamic-dynamodb - INFO - Test_Item - Consumed write units: 0%
2014-03-11 00:54:46,790 - dynamic-dynamodb - INFO - Test_Item - Write throttle count: 0
2014-03-11 00:54:46,790 - dynamic-dynamodb - INFO - Test_Item - No need to change provisioning
Traceback (most recent call last):
  File "/usr/local/bin/dynamic-dynamodb", line 27, in <module>
    dynamic_dynamodb.main()
  File "/usr/local/lib/python2.6/dist-packages/dynamic_dynamodb/__init__.py", line 128, in main
    check_interval=config['global']['check_interval'])
  File "/usr/local/lib/python2.6/dist-packages/dynamic_dynamodb/__init__.py", line 52, in run
    if re.match(gsi_key, gsi_name):
  File "/usr/lib/python2.6/re.py", line 137, in match
    return _compile(pattern, flags).match(string)
  File "/usr/lib/python2.6/re.py", line 245, in _compile
    raise error, v # invalid expression
sre_constants.error: nothing to repeat

What is this?

@sebdah

This comment has been minimized.

Owner

sebdah commented Mar 11, 2014

Please attach your configuration. I would assume that you have an invalid
regular expression in it, given the last rows in the exception.
On Mar 11, 2014 5:56 AM, "OXYAMINE" notifications@github.com wrote:

Daemon doesn't start.

Running command:
dynamic-dynamodb -c /etc/dynamic_dynamoDB.conf --log-config-file
dynamic_dynamoDB_logging.conf --daemon foreground

Output is:
2014-03-11 00:54:46,691 - dynamic-dynamodb - INFO - Test_Item - Consumed
read units: 12%
2014-03-11 00:54:46,714 - dynamic-dynamodb - INFO - Test_Item - Read
throttle count: 0
2014-03-11 00:54:46,772 - dynamic-dynamodb - INFO - Test_Item - Consumed
write units: 0%
2014-03-11 00:54:46,790 - dynamic-dynamodb - INFO - Test_Item - Write
throttle count: 0
2014-03-11 00:54:46,790 - dynamic-dynamodb - INFO - Test_Item - No need to
change provisioning
Traceback (most recent call last):
File "/usr/local/bin/dynamic-dynamodb", line 27, in
dynamic_dynamodb.main()
File "/usr/local/lib/python2.6/dist-packages/dynamic_dynamodb/init.py",
line 128, in main
check_interval=config['global']['check_interval'])
File "/usr/local/lib/python2.6/dist-packages/dynamic_dynamodb/init.py",
line 52, in run
if re.match(gsi_key, gsi_name):
File "/usr/lib/python2.6/re.py", line 137, in match
return

_compile(pattern, flags).match(string) File "/usr/lib/python2.6/re.py",
line 245, in _compile raise error, v # invalid expression sre_constants.error:
nothing to repeat

What is this?

Reply to this email directly or view it on GitHubhttps://github.com//issues/125
.

@OXYAMINE

This comment has been minimized.

OXYAMINE commented Mar 11, 2014

[global]
# AWS access keys
aws-access-key-id: removed
aws-secret-access-key-id: removed

# AWS region to use
region: us-east-1

# How often should Dynamic DynamoDB monitor changes (in seconds)
check-interval: 300

# Circuit breaker configuration
# No provisioning updates will be made unless this URL returns
# a HTTP 200 OK status code
#circuit-breaker-url: http://my.service.com/v1/is_up
#circuit-breaker-timeout: 500

[logging]
# Log level [debug|info|warning|error]
log-level: info

# Log file (comment out to get only console output)
log-file: /var/log/dynamic-dynamodb.log

# External Python logging configuration file
# Overrides both log-level and log-file
# log-config-file: /path/to/logging.conf

[table: Test_Item]
#
# Read provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
reads-upper-threshold: 90
reads-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-reads-with: 10
decrease-reads-with: 10

# Units to increase or decrease reads with, must be either percent or units
increase-reads-unit: percent
decrease-reads-unit: percent

# Maximum and minimum read provisioning
# Dynamic DynamoDB will not provision any more or less reads than this
min-provisioned-reads: 5
max-provisioned-reads: 500

#
# Write provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
writes-upper-threshold: 90
writes-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-writes-with: 10
decrease-writes-with: 10

# Units to increase or decrease writes with, must be either percent or units
increase-writes-unit: percent
decrease-writes-unit: percent

# Maximum and minimum write provisioning
# Dynamic DynamoDB will not provision any more or less writes than this
min-provisioned-writes: 5
max-provisioned-writes: 500

#
# Maintenance windows (in UTC)
#
#maintenance-windows: 22:00-23:59,00:00-06:00

#
# Other settings
#

# Allow down scaling when at 0% consumed reads
allow-scaling-down-reads-on-0-percent: true
allow-scaling-down-writes-on-0-percent: true

# Restric scale down to only happen when BOTH reads AND writes are in need
# of scaling down. Set this to "true" to minimize down scaling.
#always-decrease-rw-together: true

[gsi: * table: Test_Item]
#
# Read provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
reads-upper-threshold: 90
reads-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-reads-with: 50
decrease-reads-with: 50

# Units to increase or decrease reads with, must be either percent or units
increase-reads-unit: percent
decrease-reads-unit: percent

# Maximum and minimum read provisioning
# Dynamic DynamoDB will not provision any more or less reads than this
min-provisioned-reads: 4
max-provisioned-reads: 500

#
# Write provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
writes-upper-threshold: 90
writes-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-writes-with: 50
decrease-writes-with: 50

# Units to increase or decrease writes with, must be either percent or units
increase-writes-unit: percent
decrease-writes-unit: percent

# Maximum and minimum write provisioning
# Dynamic DynamoDB will not provision any more or less writes than this
min-provisioned-writes: 4
max-provisioned-writes: 500

#
# Maintenance windows (in UTC)
#
#maintenance-windows: 22:00-23:59,00:00-06:00

#
# Other settings
#

# Allow down scaling when at 0% consumed reads
allow-scaling-down-reads-on-0-percent: true
allow-scaling-down-writes-on-0-percent: true

# Restric scale down to only happen when BOTH reads AND writes are in need
# of scaling down. Set this to "true" to minimize down scaling.
#always-decrease-rw-together: true
@OXYAMINE

This comment has been minimized.

OXYAMINE commented Mar 11, 2014

[global]
# AWS access keys
aws-access-key-id: XXX
aws-secret-access-key-id: XXXX

# AWS region to use
region: us-east-1

# How often should Dynamic DynamoDB monitor changes (in seconds)
check-interval: 300

# Circuit breaker configuration
# No provisioning updates will be made unless this URL returns
# a HTTP 200 OK status code
#circuit-breaker-url: http://my.service.com/v1/is_up
#circuit-breaker-timeout: 500

[logging]
# Log level [debug|info|warning|error]
log-level: info

# Log file (comment out to get only console output)
log-file: /var/log/dynamic-dynamodb.log

# External Python logging configuration file
# Overrides both log-level and log-file
# log-config-file: /path/to/logging.conf

[table: Test_Item]
#
# Read provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
reads-upper-threshold: 90
reads-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-reads-with: 10
decrease-reads-with: 10

# Units to increase or decrease reads with, must be either percent or units
increase-reads-unit: percent
decrease-reads-unit: percent

# Maximum and minimum read provisioning
# Dynamic DynamoDB will not provision any more or less reads than this
min-provisioned-reads: 5
max-provisioned-reads: 500

#
# Write provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
writes-upper-threshold: 90
writes-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-writes-with: 10
decrease-writes-with: 10

# Units to increase or decrease writes with, must be either percent or units
increase-writes-unit: percent
decrease-writes-unit: percent

# Maximum and minimum write provisioning
# Dynamic DynamoDB will not provision any more or less writes than this
min-provisioned-writes: 5
max-provisioned-writes: 500

#
# Maintenance windows (in UTC)
#
#maintenance-windows: 22:00-23:59,00:00-06:00

#
# Other settings
#

# Allow down scaling when at 0% consumed reads
allow-scaling-down-reads-on-0-percent: true
allow-scaling-down-writes-on-0-percent: true

# Restric scale down to only happen when BOTH reads AND writes are in need
# of scaling down. Set this to "true" to minimize down scaling.
#always-decrease-rw-together: true

[gsi: * table: Test_Item]
#
# Read provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
reads-upper-threshold: 90
reads-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-reads-with: 50
decrease-reads-with: 50

# Units to increase or decrease reads with, must be either percent or units
increase-reads-unit: percent
decrease-reads-unit: percent

# Maximum and minimum read provisioning
# Dynamic DynamoDB will not provision any more or less reads than this
min-provisioned-reads: 4
max-provisioned-reads: 500

#
# Write provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
writes-upper-threshold: 90
writes-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-writes-with: 50
decrease-writes-with: 50

# Units to increase or decrease writes with, must be either percent or units
increase-writes-unit: percent
decrease-writes-unit: percent

# Maximum and minimum write provisioning
# Dynamic DynamoDB will not provision any more or less writes than this
min-provisioned-writes: 4
max-provisioned-writes: 500

#
# Maintenance windows (in UTC)
#
#maintenance-windows: 22:00-23:59,00:00-06:00

#
# Other settings
#

# Allow down scaling when at 0% consumed reads
allow-scaling-down-reads-on-0-percent: true
allow-scaling-down-writes-on-0-percent: true

# Restric scale down to only happen when BOTH reads AND writes are in need
# of scaling down. Set this to "true" to minimize down scaling.
#always-decrease-rw-together: true
@OXYAMINE

This comment has been minimized.

OXYAMINE commented Mar 11, 2014

OMG! why the font is so large?
anyway... it's just a test config.
Table - Test_item, all indexes

@OXYAMINE

This comment has been minimized.

OXYAMINE commented Mar 11, 2014

you're 100% right!
changing * to .* seems solve the issue.
Could you consider to check config before execution and output user friendly message?
I feel myself stupid a little bit :)

@sebdah

This comment has been minimized.

Owner

sebdah commented Mar 11, 2014

Alright :).

I'll definitely add a more human readable error message.

Happy hacking and thanks for the report!
On Mar 11, 2014 6:21 AM, "OXYAMINE" notifications@github.com wrote:

you're 100% right!
changing * to .* seems solve the issue.
Could you consider to check config before execution and output user
friendly message?
I feel myself stupid a little bit :)

Reply to this email directly or view it on GitHubhttps://github.com//issues/125#issuecomment-37264432
.

@sebdah

This comment has been minimized.

Owner

sebdah commented Mar 11, 2014

Forgot to say, GitHub uses Markdown for formatting the comments, that's why it looks a bit messed up. I have fixed the formatting above :).

@sebdah sebdah added this to the 1.9.x milestone Mar 11, 2014

@sebdah sebdah self-assigned this Mar 11, 2014

@OXYAMINE

This comment has been minimized.

OXYAMINE commented Mar 11, 2014

Thanks a lot

sebdah added a commit that referenced this issue Mar 17, 2014

@sebdah sebdah closed this Mar 17, 2014

@sebdah

This comment has been minimized.

Owner

sebdah commented Mar 17, 2014

This has been fixed in version 1.9.1

@OXYAMINE

This comment has been minimized.

OXYAMINE commented Mar 18, 2014

thanks a lot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment