New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Auto reload configuration files when modified #297

Closed
derks opened this Issue Feb 21, 2015 · 4 comments

Comments

Projects
None yet
3 participants
@derks
Member

derks commented Feb 21, 2015

If any configuration files under CementApp.Meta.config_files are modified after the initial startup, they should/could be reloaded dynamically. This would probably be best to put into its own extension as it would not be specific to any particular config handler, but also may make sense to put into the base class.

@derks derks self-assigned this Feb 21, 2015

@derks derks added this to the 2.5.2 Development milestone Feb 21, 2015

@fxstein

This comment has been minimized.

Show comment
Hide comment
@fxstein

fxstein Mar 2, 2015

Contributor

Would recommend to add some basic logic as to how often to check for changes (e.g. no more than once every x seconds) and could then be called within an App's sleep(). Once in a method of the base class, it could also be called from a task within an asyncio app.

Contributor

fxstein commented Mar 2, 2015

Would recommend to add some basic logic as to how often to check for changes (e.g. no more than once every x seconds) and could then be called within an App's sleep(). Once in a method of the base class, it could also be called from a task within an asyncio app.

@gau1991

This comment has been minimized.

Show comment
Hide comment
@gau1991

gau1991 Mar 2, 2015

We were trying to use https://github.com/seb-m/pyinotify with our Python App, and its wonderful, I hope this will be helpful to fire event when file is modified

gau1991 commented Mar 2, 2015

We were trying to use https://github.com/seb-m/pyinotify with our Python App, and its wonderful, I hope this will be helpful to fire event when file is modified

derks added a commit that referenced this issue May 1, 2015

Proof of Concept on Issue #297
Actually Add The Source

Delete Old Source
@derks

This comment has been minimized.

Show comment
Hide comment
@derks

derks May 1, 2015

Member

I've put a little time into doing a proof of concept for this and pushed that branch ext_reload_config. This only includes watching the configuration files listed in CementApp.Meta.config_files, but will need to be expanded to handle plugin configuration files as well. I don't believe that will be much more work.

This implementation uses pyinotify as suggested by @gau1991 (thanks!). Using the branch ext_reload_config, the following example app demonstrates its use:

from time import sleep
from cement.core.exc import CaughtSignal
from cement.core import hook
from cement.core.foundation import CementApp
from cement.core.controller import CementBaseController, expose

def print_foo(app):
    print "Foo => %s" % app.config.get('myapp', 'foo')

class Base(CementBaseController):
    class Meta:
        label = 'base'

    @expose(hide=True)
    def default(self):
        print('Inside Base.default()')

        # simulate a long running process
        while True:
            sleep(30)

class MyApp(CementApp):
    class Meta:
        label = 'myapp'
        base_controller = Base
        extensions = ['reload_config']


with MyApp() as app:
    # run this anytime the configuration has changed
    hook.register('post_reload_config', print_foo)

    try:
        app.run()
    except CaughtSignal as e:
        # maybe do something... but catch it regardless so app.close() is
        # called when exiting `with` cleanly.
        print(e)

Running the long running process (app) in one terminal, and then modifying the foo setting in my ~/.myapp.conf in another terminal you can see everything in action (spaces added for clarity):

Inside Base.default()


2015-05-01 23:41:10,950 (DEBUG) myapp : Config path modified: mask=IN_CLOSE_WRITE, path=/home/vagrant/.myapp.conf
2015-05-01 23:41:10,950 (INFO) myapp : Reloading configuration...
2015-05-01 23:41:10,950 (DEBUG) cement.core.foundation : setting up myapp.config handler
2015-05-01 23:41:10,950 (DEBUG) cement.core.config : config file '/etc/myapp/myapp.conf' does not exist, skipping...
2015-05-01 23:41:10,950 (DEBUG) cement.core.config : config file '/home/vagrant/.myapp.conf' exists, loading settings...
2015-05-01 23:41:10,950 (DEBUG) cement.core.config : config file '/home/vagrant/.myapp/config' does not exist, skipping...
2015-05-01 23:41:10,950 (DEBUG) cement.core.hook : running hook 'post_reload_config' (<function print_foo at 0x7f8a28462050>) from __main__
Foo => bar


2015-05-01 23:41:23,815 (DEBUG) myapp : Config path modified: mask=IN_CLOSE_WRITE, path=/home/vagrant/.myapp.conf
2015-05-01 23:41:23,815 (INFO) myapp : Reloading configuration...
2015-05-01 23:41:23,815 (DEBUG) cement.core.foundation : setting up myapp.config handler
2015-05-01 23:41:23,816 (DEBUG) cement.core.config : config file '/etc/myapp/myapp.conf' does not exist, skipping...
2015-05-01 23:41:23,816 (DEBUG) cement.core.config : config file '/home/vagrant/.myapp.conf' exists, loading settings...
2015-05-01 23:41:23,816 (DEBUG) cement.core.config : config file '/home/vagrant/.myapp/config' does not exist, skipping...
2015-05-01 23:41:23,816 (DEBUG) cement.core.hook : running hook 'post_reload_config' (<function print_foo at 0x7f8a28462050>) from __main__
Foo => bar2


2015-05-01 23:41:34,834 (DEBUG) myapp : Config path modified: mask=IN_CLOSE_WRITE, path=/home/vagrant/.myapp.conf
2015-05-01 23:41:34,834 (INFO) myapp : Reloading configuration...
2015-05-01 23:41:34,834 (DEBUG) cement.core.foundation : setting up myapp.config handler
2015-05-01 23:41:34,834 (DEBUG) cement.core.config : config file '/etc/myapp/myapp.conf' does not exist, skipping...
2015-05-01 23:41:34,834 (DEBUG) cement.core.config : config file '/home/vagrant/.myapp.conf' exists, loading settings...
2015-05-01 23:41:34,834 (DEBUG) cement.core.config : config file '/home/vagrant/.myapp/config' does not exist, skipping...
2015-05-01 23:41:34,835 (DEBUG) cement.core.hook : running hook 'post_reload_config' (<function print_foo at 0x7f8a28462050>) from __main__
Foo => bar3
Member

derks commented May 1, 2015

I've put a little time into doing a proof of concept for this and pushed that branch ext_reload_config. This only includes watching the configuration files listed in CementApp.Meta.config_files, but will need to be expanded to handle plugin configuration files as well. I don't believe that will be much more work.

This implementation uses pyinotify as suggested by @gau1991 (thanks!). Using the branch ext_reload_config, the following example app demonstrates its use:

from time import sleep
from cement.core.exc import CaughtSignal
from cement.core import hook
from cement.core.foundation import CementApp
from cement.core.controller import CementBaseController, expose

def print_foo(app):
    print "Foo => %s" % app.config.get('myapp', 'foo')

class Base(CementBaseController):
    class Meta:
        label = 'base'

    @expose(hide=True)
    def default(self):
        print('Inside Base.default()')

        # simulate a long running process
        while True:
            sleep(30)

class MyApp(CementApp):
    class Meta:
        label = 'myapp'
        base_controller = Base
        extensions = ['reload_config']


with MyApp() as app:
    # run this anytime the configuration has changed
    hook.register('post_reload_config', print_foo)

    try:
        app.run()
    except CaughtSignal as e:
        # maybe do something... but catch it regardless so app.close() is
        # called when exiting `with` cleanly.
        print(e)

Running the long running process (app) in one terminal, and then modifying the foo setting in my ~/.myapp.conf in another terminal you can see everything in action (spaces added for clarity):

Inside Base.default()


2015-05-01 23:41:10,950 (DEBUG) myapp : Config path modified: mask=IN_CLOSE_WRITE, path=/home/vagrant/.myapp.conf
2015-05-01 23:41:10,950 (INFO) myapp : Reloading configuration...
2015-05-01 23:41:10,950 (DEBUG) cement.core.foundation : setting up myapp.config handler
2015-05-01 23:41:10,950 (DEBUG) cement.core.config : config file '/etc/myapp/myapp.conf' does not exist, skipping...
2015-05-01 23:41:10,950 (DEBUG) cement.core.config : config file '/home/vagrant/.myapp.conf' exists, loading settings...
2015-05-01 23:41:10,950 (DEBUG) cement.core.config : config file '/home/vagrant/.myapp/config' does not exist, skipping...
2015-05-01 23:41:10,950 (DEBUG) cement.core.hook : running hook 'post_reload_config' (<function print_foo at 0x7f8a28462050>) from __main__
Foo => bar


2015-05-01 23:41:23,815 (DEBUG) myapp : Config path modified: mask=IN_CLOSE_WRITE, path=/home/vagrant/.myapp.conf
2015-05-01 23:41:23,815 (INFO) myapp : Reloading configuration...
2015-05-01 23:41:23,815 (DEBUG) cement.core.foundation : setting up myapp.config handler
2015-05-01 23:41:23,816 (DEBUG) cement.core.config : config file '/etc/myapp/myapp.conf' does not exist, skipping...
2015-05-01 23:41:23,816 (DEBUG) cement.core.config : config file '/home/vagrant/.myapp.conf' exists, loading settings...
2015-05-01 23:41:23,816 (DEBUG) cement.core.config : config file '/home/vagrant/.myapp/config' does not exist, skipping...
2015-05-01 23:41:23,816 (DEBUG) cement.core.hook : running hook 'post_reload_config' (<function print_foo at 0x7f8a28462050>) from __main__
Foo => bar2


2015-05-01 23:41:34,834 (DEBUG) myapp : Config path modified: mask=IN_CLOSE_WRITE, path=/home/vagrant/.myapp.conf
2015-05-01 23:41:34,834 (INFO) myapp : Reloading configuration...
2015-05-01 23:41:34,834 (DEBUG) cement.core.foundation : setting up myapp.config handler
2015-05-01 23:41:34,834 (DEBUG) cement.core.config : config file '/etc/myapp/myapp.conf' does not exist, skipping...
2015-05-01 23:41:34,834 (DEBUG) cement.core.config : config file '/home/vagrant/.myapp.conf' exists, loading settings...
2015-05-01 23:41:34,834 (DEBUG) cement.core.config : config file '/home/vagrant/.myapp/config' does not exist, skipping...
2015-05-01 23:41:34,835 (DEBUG) cement.core.hook : running hook 'post_reload_config' (<function print_foo at 0x7f8a28462050>) from __main__
Foo => bar3

derks added a commit that referenced this issue May 5, 2015

derks added a commit that referenced this issue May 5, 2015

derks added a commit that referenced this issue May 5, 2015

derks added a commit that referenced this issue May 5, 2015

derks added a commit that referenced this issue May 5, 2015

@derks

This comment has been minimized.

Show comment
Hide comment
@derks

derks May 5, 2015

Member

This has been merge into cement/master as an experimental feature.

Member

derks commented May 5, 2015

This has been merge into cement/master as an experimental feature.

@derks derks closed this May 5, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment