Permalink
Browse files

default config path env. var, r=twobraids

  • Loading branch information...
1 parent 96bc910 commit 806e88ea394177b4102ebff70ebf1145c0d1fb03 @peterbe peterbe committed Nov 20, 2012
Showing with 123 additions and 14 deletions.
  1. +41 −13 docs/generic_app.rst
  2. +13 −1 socorro/app/generic_app.py
  3. +69 −0 socorro/unittest/app/test_generic_app.py
View
@@ -15,31 +15,31 @@ uses ``generic_app`` to leverage ``configman`` to run. Let's look at `weeklyRepo
As you can see, it's a subclass of the `socorro.app.generic_app.App
<https://github.com/mozilla/socorro/blob/master/socorro/app/generic_app.py>`_
class which is a the-least-you-need wrapper for a minimal app. As you
-can see, it takes care of logging and executing your ``main`` function.
+can see, it takes care of logging and executing your ``main`` function.
Connecting and handling transactions
------------------------------------
Let's go back to the ``weeklyReportsPartitions.py`` cron script and take
-a look at what it does.
+a look at what it does.
It only really has one ``configman`` option and that's the
-``transaction_executor_class``. The default value is
+``transaction_executor_class``. The default value is
`TransactionExecutorWithBackoff
<https://github.com/mozilla/socorro/blob/master/socorro/database/transaction_executor.py#L59>`_
which is the class that's going to take care of two things:
1. execute a callable that accepts an opened database connection as
first and only parameter
-
+
2. committing the transaction if there are no errors and rolling back
the transaction if an exception is raised
-
+
3. NB: if an ``OperationalError`` or ``InterfaceError`` exception is
raised, ``TransactionExecutorWithBackoff`` will log that and retry
after configurable delay
-
+
Note that ``TransactionExecutorWithBackoff`` is the default
``transaction_executor_class`` but if you override it, for example by the command
line, with ``TransactionExecutor`` no exceptions are swallowed and it
@@ -57,15 +57,15 @@ The idea is that any external module (e.g. HBase, PostgreSQL, etc)
can define a ``ConnectionContext`` class as per this model. What its job
is is to create and close connections and it has to do so in a
contextmanager. What that means is that you can do this::
-
+
connector = ConnectionContext()
with connector() as connection: # opens a connection
do_something(connection)
# closes the connection
-
+
And if errors are raised within the ``do_something`` function it
-doesn't matter. The connection will be closed.
-
+doesn't matter. The connection will be closed.
+
What was the point of that?!
----------------------------
@@ -75,15 +75,43 @@ configuration settings are as flexible as ``configman`` is. You can supply
different values for any of the options either by the command line
(try running ``--help`` on the ``./weeklyReportsPartitions.py`` script)
and you can control them with various configuration files as per your
-liking.
+liking.
The other thing to notice is that when writing another similar cron
script, all you need to do is to worry about exactly what to execute
and let the framework take care of transactions and opening and
closing connections. Each class is supposed to do one job and one job
-only.
+only.
``configman`` uses not only basic options such as ``database_password``
but also more complex options such as aggregators. These are basically
invariant options that depend on each other and uses functions in
-there to get its stuff together.
+there to get its stuff together.
+
+
+How to override where config files are read
+-------------------------------------------
+
+``generic_app`` supports multiple ways of picking up config files.
+The most basic option is the `--admin.conf=` option. E.g.::
+
+ python myapp.py --admin.conf=/path/to/my.ini
+
+The default if you don't specify a ``--admin.conf`` is that it will
+look for a ``.ini`` file with the same name as the app. So if
+``app_name='myapp'`` and you start it like this::
+
+ python myapp.py
+
+it will automatically try to read ``config/myapp.ini`` and if you want
+to override the directory it searches in you have to set an
+environment variable called ``DEFAULT_SOCORRO_CONFIG_PATH`` like this::
+
+ export DEFAULT_SOCORRO_CONFIG_PATH=/etc/socorro
+ python myapp.py
+
+Which means it will try to read ``/etc/socorro/myapp.ini``.
+
+**NOTE:** If you specify a ``DEFAULT_SOCORRO_CONFIG_PATH`` that
+directory must exist and be readable or else you will get an
+``IOError`` when you try to start your app.
View
@@ -3,6 +3,7 @@
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
+import os
import re
import inspect
import logging
@@ -122,10 +123,21 @@ def _convert_format_string(s):
#------------------------------------------------------------------------------
# This main function will load an application object, initialize it and then
# call its 'main' function
-def main(initial_app, values_source_list=None, config_path='./config'):
+def main(initial_app, values_source_list=None, config_path=None):
if isinstance(initial_app, basestring):
initial_app = class_converter(initial_app)
+ if config_path is None:
+ default = './config'
+ config_path = os.environ.get(
+ 'DEFAULT_SOCORRO_CONFIG_PATH',
+ default
+ )
+ if config_path != default:
+ # you tried to set it, then it must be a valid directory
+ if not os.path.isdir(config_path):
+ raise IOError('%s is not a valid directory' % config_path)
+
# the only config parameter is a special one that refers to a class or
# module that defines an application. In order to qualify, a class must
# have a constructor that accepts a DotDict derivative as the sole
@@ -0,0 +1,69 @@
+# This Source Code Form is subject to the terms of the Mozilla Public
+# License, v. 2.0. If a copy of the MPL was not distributed with this
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
+
+import tempfile
+import shutil
+import os
+import unittest
+import mock
+from configman import Namespace
+from configman.config_file_future_proxy import ConfigFileFutureProxy
+from socorro.app.generic_app import App, main
+
+
+class MyApp(App):
+ app_name = 'myapp'
+ app_version = '1.0'
+ app_description = 'bla bla'
+
+ required_config = Namespace()
+ required_config.add_option(
+ name='color_or_colour',
+ default='colour',
+ doc='How do you spell it?',
+ )
+
+ def main(self):
+ self.config.logger.error(self.config.color_or_colour)
+
+
+class TestGenericAppConfigPathLoading(unittest.TestCase):
+ """
+ Test that it's possible to override the default directory from where
+ generic_app tries to read default settings from.
+
+ This is depending on there being an environment variable called
+ DEFAULT_SOCORRO_CONFIG_PATH which must exist.
+ """
+
+ def setUp(self):
+ self.tempdir = tempfile.mkdtemp()
+
+ def tearDown(self):
+ if os.path.isdir(self.tempdir):
+ shutil.rmtree(self.tempdir)
+
+ @mock.patch('socorro.app.generic_app.logging')
+ def test_overriding_config_path(self, logging):
+ vsl = (ConfigFileFutureProxy,)
+ exit_code = main(MyApp, values_source_list=vsl)
+ self.assertEqual(exit_code, 0)
+
+ os.environ['DEFAULT_SOCORRO_CONFIG_PATH'] = '/foo/bar'
+ self.assertRaises(IOError, main, (MyApp,), values_source_list=vsl)
+
+ os.environ['DEFAULT_SOCORRO_CONFIG_PATH'] = self.tempdir
+ exit_code = main(MyApp, values_source_list=vsl)
+ self.assertEqual(exit_code, 0)
+
+ logging.getLogger().error.assert_called_with('colour')
+
+ _ini_file = os.path.join(self.tempdir, 'myapp.ini')
+ with open(_ini_file, 'w') as f:
+ f.write('color_or_colour=color\n')
+
+ exit_code = main(MyApp, values_source_list=vsl)
+ self.assertEqual(exit_code, 0)
+
+ logging.getLogger().error.assert_called_with('color')

0 comments on commit 806e88e

Please sign in to comment.