Config framework refactor #373

mtomwing · 2013-09-14T07:52:37Z

This new config framework makes it easier to make persistent configuration files with Pythonic access.

from freeseer import settings

profile, storage = settings.setup_profile_and_storage('default', 'freeseer.conf')
config = storage.load(settings.FreeseerConfig(), 'Global')
storage.store(config, 'Global')

This creates a profile @ $HOME/.freeseer/profiles/default and creates a default config in freeseer.conf with the defaults of any options defined inside the FreeseerConfig class.

class FreeseerConfig(Config):
    # TODO: This should probably be in a constants file somewhere
    resmap = {
        'default': '0x0',
        '240p': '320x240',
        '360p': '480x360',
        '480p': '640x480',
        '720p': '1280x720',
        '1080p': '1920x1080'
    }

    videodir = options.FolderOption('~/Videos', auto_create=True)
    auto_hide = options.BooleanOption(True)
    resolution = options.ChoiceOption(resmap.keys(), 'default')
    enable_audio_recording = options.BooleanOption(True)
    enable_video_recording = options.BooleanOption(True)
    videomixer = options.StringOption('Video Passthrough')
    audiomixer = options.StringOption('Audio Passthrough')
    record_to_file = options.BooleanOption(True)
    record_to_file_plugin = options.StringOption('Ogg Output')
    record_to_stream = options.BooleanOption(False)
    record_to_stream_plugin = options.StringOption('RTMP Streaming')
    audio_feedback = options.BooleanOption(False)
    video_preview = options.BooleanOption(True)
    default_language = options.StringOption('tr_en_US.qm')

And the file it created:

[Global]
audio_feedback = false
audiomixer = Audio Passthrough
auto_hide = true
default_language = tr_en_US.qm
enable_audio_recording = true
enable_video_recording = true
record_to_file = true
record_to_file_plugin = Ogg Output
record_to_stream = false
record_to_stream_plugin = RTMP Streaming
resolution = default
video_preview = true
videodir = /home/mtomwing/Videos
videomixer = Video Passthrough

The config object can be modified further and saved to disk whenever:

config.view_preview = False
config.auto_hide = False
config.videodir = '/tmp/videos'

storage.store(config, 'Global')

[Global]
audio_feedback = false
audiomixer = Audio Passthrough
auto_hide = false
default_language = tr_en_US.qm
enable_audio_recording = true
enable_video_recording = true
record_to_file = true
record_to_file_plugin = Ogg Output
record_to_stream = false
record_to_stream_plugin = RTMP Streaming
resolution = default
video_preview = true
videodir = /tmp/videos
videomixer = Video Passthrough

mtomwing · 2013-09-14T08:27:14Z

There are still many things to do.

freeseer-server has not been updated to use the new config/profile stuff.
Unittests need to be updated and new ones added for the config framework.
Add the Freeseer license headers to the new files.
Create a new PluginManager that fully utilizes the config framework. Plugins should be able to specify their own config classes in the same manner as FreeseerConfig.
Finish the transition to Python 3.x+. This would allow me to use the __prepare__ metaclass method to save the config file options in the same order as they are defined in the Config class.

zxiiro · 2013-09-14T12:07:54Z

src/freeseer/settings.py

+    }
+
+    videodir = options.FolderOption('/home/mtomwing/Videos', auto_create=True)


Should use something like ~/Videos or similar so that when someone else tests this it works with their home.

Whoops, forgot to change that back.

Fixed in b68e52f

mtomwing · 2013-09-20T07:17:44Z

Here's a better overview of the config architecture:

And some better example code:

from freeseer.framework.config.core import Config
from freeseer.framework.config.profile import ProfileManager
from freeseer.framework.config.persist import ConfigParserStorage
import freeseer.framework.config.options as options


class MyConfig(Config):
    foo = options.BooleanOption(True)
    bar = options.IntegerOption(10)


if __name__ == '__main__':
    # Create a ProfileManager for our main profiles folder
    profile_manager = ProfileManager('/tmp/freeseer/profiles')

    # Get a specific profile (and create it if it does not exist) from the 
    # manager. i.e. /tmp/freeseer/profiles/default
    profile = profile_manager.get('default')

    # Setup a ConfigStorage instance using the ConfigParser module as a 
    # persistence mechanism. i.e. /tmp/freeseer/profiles/default/freeseer.conf
    storage = ConfigParserStorage(profile, 'freeseer.conf')

    # Create config with default values (if specified)
    config = MyConfig()

    # Options can be accessed as if they were proper attributes
    assert(config.foo == True)
    assert(config.bar == 10)

    # A config instance can be saved to disk (or somewhere) using the
    # ConfigStorage instance. 'Global' is a section heading required by
    # ConfigParser.
    storage.store(config, 'Global')

    # Changes are only persisted when using ConfigStorage.store(...)
    config.foo = False
    config.bar = 9001

    # The config instance can be 'reset' to its previously saved state.
    storage.load(config, 'Global')

    # Verified
    assert(config.foo == True)
    assert(config.bar == 10)

dideler · 2013-09-20T17:58:38Z

src/freeseer/settings.py

+    # TODO: This should probably be in a constants file somewhere
+    resmap = {
+        'default': '0x0',


Can you add a comment to explain what 0x0 means (e.g. no scaling)?

Fixed in db5698b. Although I think @zxiiro was saying that resmap is not used anywhere atm, so maybe it's better to just remove this.

zxiiro · 2013-09-20T18:25:28Z

we need to reimplemente scaling. I meant the scaling map is not used at the moment because it's not been reimplemented yet since the rewrite of the backend. so moving it around should be easy.

mtomwing · 2013-11-26T06:02:01Z

As far as I can tell, my refactor is complete. I've also added more than 100 new tests that should cover all of my new Config-related code.

dideler · 2013-11-27T04:51:46Z

Awesome, thanks! It might take a while until I get to look at the changes though, still going through some other reviews.

zxiiro · 2013-12-01T14:44:18Z

I did some tests using the config tool and recording tool and as far as I can tell the main functionality still works. Thanks for your contribution. Can you rebase when you get a chance?

mtomwing · 2013-12-01T21:00:42Z

@zxiiro I will try and get to that today.

… refactor New concepts: - ProfileManager, Profile - ConfigStorage, Config, Option Profiles and Profile Management =============================== `~/.freeseer/profiles` is now managed by a `ProfileManager`. It handles creating profiles. Each profile is now modeled by a `Profile` object. It can be used to get files from within its folder in a pythonic-manner. See `freeseer.framework.config.profile`. Configuration ============== `FreeseerConfig` and future "configuration classes" have been reworked to remove needless boilerplate, provide stricter type checking, and be easily persisted to a file or some other backend. Configs written in a similar declarative manner like Django or SqlAlchemy models: from freeseer.framework.config import Config from freeseer.framework.config import options class MyConfig(Config): name = options.StringOption('foo') age = options.IntegerOption() Upon instantiation, those options can be accessed as if they were regular class attributes. Any changes to their values will be stored internally. As you can imagine, values assigned are validated by the various options (e.g. age only accepts integers). Persistance of configs is handled by ConfigStorages. I've only implemented two so far inside `freeseer.framework.config.persist`: one that uses `ConfigParser` for INI-style files, and one that uses a JSON file. from freeseer.framework.config.persist import ConfigParserStorage # load the config, if the file doesn't exist it'll use defaults storage = ConfigParserStorage('/tmp/path/to/file.conf') config = storage.load(MyConfig(), 'section') # make your changes config.name = 'michael' config.age = 22 # save them storage.store(config, 'section') This will create a file that looks like: [section] name = michael age = 22 In conjunction with the `Profile` abstraction, you have a nice way of storing config files inside a profile (e.g. freeseer.conf and plugins.conf).

dideler · 2013-12-04T20:33:27Z

src/freeseer/framework/config/core.py

+        dct['options'] = options
+        cls = super(ConfigBase, meta).__new__(meta, name, bases, dct)
+        for opt_name, option in options.iteritems():
+            opt_get = functools.partial(cls.get_value, name=opt_name,


I think this line and L113 should be able to fit on a single line.

Fixed in 3a836d9.

dideler · 2013-12-04T21:45:12Z

src/freeseer/frontend/configtool/configtool.py

@@ -80,8 +80,8 @@ def __init__(self, recordapp=None):
        self.avWidget = AVWidget()
        self.pluginloaderWidget = PluginLoaderWidget()

-        self.config = Config(settings.configdir)
-        self.plugman = PluginManager(settings.configdir)
+        # TODO: Make a better PluginManager


Do we have an issue open for this already? It would be a good place to provide more info.

I just created issue #416 for this and will remove this comment.

Fixed in 7bcd211.

dideler · 2013-12-06T19:59:41Z

Thanks for making the changes, looks good to me! 👍

mtomwing · 2013-12-06T20:16:21Z

I still need to fix up the docstring format as per the Freeseer guidelines. Same thing for any lines I cut to 80 chars instead of 120.

dideler · 2013-12-06T21:06:21Z

When that's done and you squash the commits, I think it'll be ready to be merged.

dideler · 2013-12-06T21:08:46Z

Do you think the diagram you made and the example code in #373 (comment) would be useful in the docs?

mtomwing · 2013-12-06T21:18:10Z

@dideler not in it's current state. I made some slight changes to the architecture.

… refactor New concepts: - ProfileManager, Profile - ConfigStorage, Config, Option Profiles and Profile Management =============================== `~/.freeseer/profiles` is now managed by a `ProfileManager`. It handles creating profiles. Each profile is now modeled by a `Profile` object. It can be used to get files from within its folder in a pythonic-manner. See `freeseer.framework.config.profile`. Configuration ============== `FreeseerConfig` and future "configuration classes" have been reworked to remove needless boilerplate, provide stricter type checking, and be easily persisted to a file or some other backend. Configs written in a similar declarative manner like Django or SqlAlchemy models: from freeseer.framework.config import Config from freeseer.framework.config import options class MyConfig(Config): name = options.StringOption('foo') age = options.IntegerOption() Upon instantiation, those options can be accessed as if they were regular class attributes. Any changes to their values will be stored internally. As you can imagine, values assigned are validated by the various options (e.g. age only accepts integers). Persistance of configs is handled by ConfigStorages. I've only implemented two so far inside `freeseer.framework.config.persist`: one that uses `ConfigParser` for INI-style files, and one that uses a JSON file. from freeseer.framework.config.persist import ConfigParserStorage # load the config, if the file doesn't exist it'll use defaults storage = ConfigParserStorage('/tmp/path/to/file.conf') config = storage.load(MyConfig(), 'section') # make your changes config.name = 'michael' config.age = 22 # save them storage.store(config, 'section') This will create a file that looks like: [section] name = michael age = 22 In conjunction with the `Profile` abstraction, you have a nice way of storing config files inside a profile (e.g. freeseer.conf and plugins.conf).

mtomwing · 2013-12-17T09:05:11Z

Rebased against master again. Rio's stuff is now included.

zxiiro · 2013-12-20T01:35:35Z

Merged. Thanks for your contribution!

zxiiro reviewed Sep 14, 2013
View reviewed changes

dideler reviewed Sep 20, 2013
View reviewed changes

dideler reviewed Dec 4, 2013
View reviewed changes

mtomwing mentioned this pull request Dec 6, 2013

Make a better PluginManager #416

Open

zxiiro merged commit efe900a into Freeseer:master Dec 20, 2013

zxiiro mentioned this pull request Dec 20, 2013

Config and DB initialized multiple times when running recordtool #406

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Config framework refactor #373

Config framework refactor #373

mtomwing commented Sep 14, 2013

mtomwing commented Sep 14, 2013

zxiiro Sep 14, 2013

mtomwing Sep 14, 2013

mtomwing commented Sep 20, 2013

dideler Sep 20, 2013

mtomwing Sep 20, 2013

zxiiro commented Sep 20, 2013

mtomwing commented Nov 26, 2013

dideler commented Nov 27, 2013

zxiiro commented Dec 1, 2013

mtomwing commented Dec 1, 2013

dideler Dec 4, 2013

mtomwing Dec 6, 2013

dideler Dec 4, 2013

mtomwing Dec 6, 2013

dideler commented Dec 6, 2013

mtomwing commented Dec 6, 2013

dideler commented Dec 6, 2013

dideler commented Dec 6, 2013

mtomwing commented Dec 6, 2013

mtomwing commented Dec 17, 2013

zxiiro commented Dec 20, 2013

		}

		videodir = options.FolderOption('/home/mtomwing/Videos', auto_create=True)

Config framework refactor #373

Config framework refactor #373

Conversation

mtomwing commented Sep 14, 2013

mtomwing commented Sep 14, 2013

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mtomwing commented Sep 20, 2013

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zxiiro commented Sep 20, 2013

mtomwing commented Nov 26, 2013

dideler commented Nov 27, 2013

zxiiro commented Dec 1, 2013

mtomwing commented Dec 1, 2013

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dideler commented Dec 6, 2013

mtomwing commented Dec 6, 2013

dideler commented Dec 6, 2013

dideler commented Dec 6, 2013

mtomwing commented Dec 6, 2013

mtomwing commented Dec 17, 2013

zxiiro commented Dec 20, 2013