Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Fetching contributors…

Cannot retrieve contributors at this time

file 89 lines (67 sloc) 3.773 kb

How generic app and an example works using configman

The minimum app

To illustrate the example, let's look at an example of an app that uses generic_app to leverage configman to run. Let's look at weeklyReportsPartitions.py

As you can see, it's a subclass of the socorro.app.generic_app.App class which is a the-least-you-need wrapper for a minimal app. As you can see, it takes care of logging and executing your main function.

Connecting and handling transactions

Let's go back to the weeklyReportsPartitions.py cron script and take a look at what it does.

It only really has one configman option and that's the transaction_executor_class. The default value is TransactionExecutorWithBackoff which is the class that's going to take care of two things:

  1. execute a callable that accepts an opened database connection as first and only parameter
  2. committing the transaction if there are no errors and rolling back the transaction if an exception is raised
  3. NB: if an OperationalError or InterfaceError exception is raised, TransactionExecutorWithBackoff will log that and retry after configurable delay

Note that TransactionExecutorWithBackoff is the default transaction_executor_class but if you override it, for example by the command line, with TransactionExecutor no exceptions are swallowed and it doesn't retry.

Now, connections are created and closed by the ConnectionContext class. As you might have noticed, the default database_class defined in the TransactionExecutor is socorro.external.postgresql.connection_context.ConnectionContext as you can see here

The idea is that any external module (e.g. HBase, PostgreSQL, etc) can define a ConnectionContext class as per this model. What its job is is to create and close connections and it has to do so in a contextmanager. What that means is that you can do this:

connector = ConnectionContext()
with connector() as connection:  # opens a connection
    do_something(connection)
# closes the connection

And if errors are raised within the do_something function it doesn't matter. The connection will be closed.

What was the point of that?!

For one thing, this app being a configman derived app means that all configuration settings are as flexible as configman is. You can supply different values for any of the options either by the command line (try running --help on the ./weeklyReportsPartitions.py script) and you can control them with various configuration files as per your liking.

The other thing to notice is that when writing another similar cron script, all you need to do is to worry about exactly what to execute and let the framework take care of transactions and opening and closing connections. Each class is supposed to do one job and one job only.

configman uses not only basic options such as database_password but also more complex options such as aggregators. These are basically invariant options that depend on each other and uses functions in there to get its stuff together.

Something went wrong with that request. Please try again.