Merge branch 'develop' into 03-2017-tutorial-update

NEWS.md

@@ -1,4 +1,47 @@
 
+v0.28 (2017-03-03)
+------------------
+
+- The YAML model format now allows users to specify compartment information and
+  compartment boundaries in the `model.yaml` file. See the file format
+  documentation for more information.
+- The `media` key in the `model.yaml` has changed name to `exchange` to
+  reflect the fact that not only uptake exchange must be defined here. The
+  `media` key is still supported but has been deprecated.
+- The gap-filling command `gapfill` and `fastgapfill` now use the compartment
+  information to determine which artificial transport and exchange reactions
+  to add. This means that a model *must* specify compartments and compartment
+  boundaries when using gap-filling commands.
+- The `gapcheck` command now has two new methods for detecting blocked
+  compounds. The new `prodcheck` is a more robust version of the GapFind check
+  which was previously used. The new `sinkcheck` method will find compounds
+  that cannot be produced in excess. This can find some additional blocked
+  compounds that were not detected by the other methods.
+- The `gapcheck` command now reports blocked compounds in the extracellular
+  space. Previously, these compounds were excluded. An option is available to
+  switch back the old behavior of excluding these from the final output.
+- The `gapcheck` command now has an option to run the check with unrestricted
+  exchange reactions.
+- The `gapfill` command can now be run without implicit sinks. This makes it
+  possible to use this command to solve additional model gaps. It is still
+  recommended to first solve gaps using implicit sinks, then later disable
+  implicit sinks when all other gaps have been closed.
+- The `gapfill` command now has an option to enable the bounds expansion
+  proposals (e.g. making irreversible reactions reversible). By default this
+  option is now off.
+- The `fastgapfill` has improved output that contains less superfluous
+  information. The output format is now identical to the `gapfill` command.
+  The `fastgapfill` also no longer runs an FBA on the induced model since this
+  caused some confusion.
+- Added new command `checkduplicates` which detects whether the model has
+  multiple reactions with the same (or similar) reaction equation.
+- The `sbmlexport` command now allows the user to specify a file path. The
+  command can also optionally output the SBML file in a more readable format
+  with an option.
+- Fixed support for the latest CPLEX release 12.7. A change in their API made
+  PSAMM incompatible with the 12.7 release. This is now fixed.
+- We now officially support Python 3.5 and Python 3.6.
+
 v0.27 (2016-12-23)
 ------------------
 

docs/api/gapfilling.rst

@@ -0,0 +1,6 @@
+
+``psamm.gapfilling`` -- Gap-filling functions
+=============================================
+
+.. automodule:: psamm.gapfilling
+   :members:

docs/commands.rst

@@ -381,7 +381,7 @@ to perform the gap check without implicit sinks.
 
 The gap check is performed with the medium that is defined in the model. It
 may be useful to run the gap check with every compound in the medium available.
-This can easily be done by specifying the ``--unrestricted-medium`` option
+This can easily be done by specifying the ``--unrestricted-exchange`` option
 which removes all limits on the exchange reactions during the check.
 
 There are some additional gap checking methods that can be enabled with the
@@ -460,7 +460,9 @@ minimal solution.
 SBML Export (``sbmlexport``)
 ----------------------------
 
-Exports the model to the SBML file format.
+Exports the model to the SBML file format. This command exports the model as
+an `SBML level 3`_ file with flux bounds, objective and gene information
+encoded with `Flux Balance Constraints version 2`_.
 
 .. code-block:: shell
 
@@ -470,6 +472,9 @@ If the file name is omitted, the file contents will be output directly to the
 screen. Using the ``--pretty`` option makes the output formatted for
 readability.
 
+.. _`SBML level 3`: http://sbml.org/Documents/Specifications
+.. _`Flux Balance Constraints version 2`: http://sbml.org/Documents/Specifications/SBML_Level_3/Packages/fbc
+
 Excel Export (``excelexport``)
 ------------------------------
 

docs/file_format.rst

@@ -31,8 +31,8 @@ used as a template:
       - include: reactions/reactions.tsv
       - include: reactions/biomass.yaml
 
-    media:
-      - include: medium.yaml
+    exchange:
+      - include: exchange.yaml
     limits:
       - include: limits.yaml
 
@@ -188,13 +188,21 @@ group of genes or when multiple genes can independently enable a reaction:
       equation: '|amp| + |atp| <=> (2) |adp|'
       genes: gene_0001 or (gene_0002 and gene_0003)
 
-Media
------
+Exchange compounds
+------------------
 
-The optional ``media`` key provides a way of defining the medium (boundary
-conditions) for the model. The medium is defined by a set of compounds that are
-able enter or leave the model system. The following fragment is an example of
-the ``medium.yaml`` file:
+The ``exchange`` key provides a way of defining the compounds that can
+enter and exit the model system (the boundary conditions). This includes
+compounds that can enter the system (*the medium*) and compounds that are
+allowed to exit the system, like metabolic byproducts. In most cases, all
+compounds that occur in the extracellular space should also be defined in the
+exchange compounds (with lower limit of zero) so that they are allowed to
+leave the model system, and PSAMM will generate a warning if this is not the
+case for some compounds. Compounds that are allowed to be taken up
+(*the medium*) should in addition be specified with a negative lower limit
+indicating the maximum allowed uptake.
+
+The following fragment is an example of the ``exchange.yaml`` file:
 
 .. code-block:: yaml
 
@@ -205,21 +213,18 @@ the ``medium.yaml`` file:
       - id: o2
       - id: glcD    # D-Glucose with uptake limit of 10
         lower: -10
-      - id: compound_x
-        compartment: c
-        lower: 0    # Provide a sink for compound_x
       # ...
 
-When a medium file is specified, the corresponding exchange reactions are
+When an exchange file is specified, the corresponding exchange reactions are
 automatically added. For example, if the compounds ``o2`` in compartment ``e``
-is in the medium, the exchange reaction ``EX_o2_e`` is added to the model. The
-desired ID for the exchange reaction can be set explicitly using the
+is in the exchange file, the exchange reaction ``EX_o2_e`` is added to the
+model. The desired ID for the exchange reaction can be set explicitly using the
 ``reaction`` attribute.
 
-The medium can also be specified using a TSV-file as the following fragment
-shows. The second column specifies the compartment while third and fourth
-columns specify the lower and upper bounds, respectively. Both can be omitted
-or specified as ``-`` to use the default flux bounds::
+The exchange set can also be specified using a TSV-file as the following
+fragment shows. The second column specifies the compartment while third and
+fourth columns specify the lower and upper bounds, respectively. Both can be
+omitted or specified as ``-`` to use the default flux bounds::
 
     # Acetate exchange with default lower and upper bounds
     ac      e
@@ -228,8 +233,9 @@ or specified as ``-`` to use the default flux bounds::
     # CO2 exchange with production limit of 50 and default uptake limit
     co2     e       -       50
 
-Multiple medium files can be included from the main ``model.yaml`` file, and
-these will be combined to form the final medium used for the simulations.
+Multiple exchange files can be included from the main ``exchange.yaml`` file,
+and these will be combined to form the final set of exchange reactions used for
+the simulations.
 
 Reaction flux limits
 --------------------

docs/tutorial.rst

@@ -5,7 +5,7 @@ PSAMM Tutorials
 .. toctree::
    :maxdepth: 2
    :glob:
-
+   
    tutorial/psamm-install.rst
    tutorial/import_export.rst
    tutorial/curation.rst

psamm/balancecheck.py

@@ -53,11 +53,11 @@ def charge_balance(model):
     """
 
     compound_charge = {}
-    for compound in model.parse_compounds():
-        if hasattr(compound, 'charge') and compound.charge is not None:
+    for compound in model.compounds:
+        if compound.charge is not None:
             compound_charge[compound.id] = compound.charge
 
-    for reaction in model.parse_reactions():
+    for reaction in model.reactions:
         charge = reaction_charge(reaction.equation, compound_charge)
         yield reaction, charge
 
@@ -101,7 +101,7 @@ def formula_balance(model):
 
     # Mapping from compound id to formula
     compound_formula = {}
-    for compound in model.parse_compounds():
+    for compound in model.compounds:
         if compound.formula is not None:
             try:
                 f = Formula.parse(compound.formula).flattened()
@@ -111,5 +111,5 @@ def formula_balance(model):
                     'Error parsing formula for compound {}: {}'.format(
                         compound.id, compound.formula), exc_info=True)
 
-    for reaction in model.parse_reactions():
+    for reaction in model.reactions:
         yield reaction, reaction_formula(reaction.equation, compound_formula)

psamm/command.py

@@ -13,7 +13,7 @@
 # You should have received a copy of the GNU General Public License
 # along with PSAMM.  If not, see <http://www.gnu.org/licenses/>.
 #
-# Copyright 2014-2015  Jon Lund Steffensen <jon_steffensen@uri.edu>
+# Copyright 2014-2017  Jon Lund Steffensen <jon_steffensen@uri.edu>
 
 """Command line interface.
 
@@ -40,9 +40,9 @@
 from six import add_metaclass, iteritems, itervalues, text_type
 
 from . import __version__ as package_version
-from .datasource.native import NativeModel
+from .datasource import native, sbml
+from .datasource.context import FilePathContext
 from .lpsolver import generic
-from . import util
 
 logger = logging.getLogger(__name__)
 
@@ -76,15 +76,11 @@ def __init__(self, model, args):
         self._model = model
         self._args = args
 
-        name = self._model.name
-        if name is None:
-            name = text_type(self._model.context)
-        logger.info('Model: {}'.format(name))
+        if self._model.name is not None:
+            logger.info('Model: {}'.format(self._model.name))
 
-        if self._model.context is not None:
-            version = util.git_try_describe(self._model.context.basepath)
-            if version is not None:
-                logger.info('Model Git version: {}'.format(version))
+        if self._model.version_string is not None:
+            logger.info('Model version: {}'.format(self._model.version_string))
 
     @classmethod
     def init_parser(cls, parser):
@@ -552,7 +548,80 @@ def main(command_class=None, args=None):
     parsed_args = parser.parse_args(args)
 
     # Load model definition
-    model = NativeModel.load_model_from_path(parsed_args.model)
+    model = native.ModelReader.reader_from_path(
+        parsed_args.model).create_model()
+
+    # Instantiate command with model and run
+    command = parsed_args.command(model, parsed_args)
+    try:
+        command.run()
+    except CommandError as e:
+        parser.error(text_type(e))
+
+
+def main_sbml(command_class=None, args=None):
+    # Set up logging for the command line interface
+    if 'PSAMM_DEBUG' in os.environ:
+        level = getattr(logging, os.environ['PSAMM_DEBUG'].upper(), None)
+        if level is not None:
+            logging.basicConfig(level=level)
+    else:
+        logging.basicConfig(level=logging.INFO)
+        base_logger = logging.getLogger('psamm')
+        if len(base_logger.handlers) == 0:
+            handler = logging.StreamHandler()
+            handler.setFormatter(
+                logging.Formatter(u'%(levelname)s: %(message)s'))
+            base_logger.addHandler(handler)
+            base_logger.propagate = False
+
+    logger.warning(
+        'This command is experimental. It currently only fully parses level 3'
+        ' SBML files!')
+
+    title = 'Metabolic modeling tools (SBML)'
+    if command_class is not None:
+        title, _, _ = command_class.__doc__.partition('\n\n')
+
+    parser = argparse.ArgumentParser(description=title)
+    parser.add_argument('model', metavar='file', help='SBML file')
+    parser.add_argument(
+        '-V', '--version', action='version',
+        version='%(prog)s ' + package_version)
+
+    if command_class is not None:
+        # Command explicitly given, only allow that command
+        command_class.init_parser(parser)
+        parser.set_defaults(command=command_class)
+    else:
+        # Discover all available commands
+        commands = {}
+        for entry in pkg_resources.iter_entry_points('psamm.commands'):
+            canonical = entry.name.lower()
+            if canonical not in commands:
+                command_class = entry.load()
+                commands[canonical] = command_class
+            else:
+                logger.warning('Command {} was found more than once!'.format(
+                    canonical))
+
+        # Create parsers for subcommands
+        subparsers = parser.add_subparsers(title='Commands', metavar='command')
+        for name, command_class in sorted(iteritems(commands)):
+            title, _, _ = command_class.__doc__.partition('\n\n')
+            subparser = subparsers.add_parser(
+                name, help=title.rstrip('.'),
+                formatter_class=argparse.RawDescriptionHelpFormatter,
+                description=_trim(command_class.__doc__))
+            subparser.set_defaults(command=command_class)
+            command_class.init_parser(subparser)
+
+    parsed_args = parser.parse_args(args)
+
+    # Load model definition
+    context = FilePathContext(parsed_args.model)
+    with context.open('r') as f:
+        model = sbml.SBMLReader(f, context=context).create_model()
 
     # Instantiate command with model and run
     command = parsed_args.command(model, parsed_args)

psamm/commands/chargecheck.py

@@ -13,7 +13,7 @@
 # You should have received a copy of the GNU General Public License
 # along with PSAMM.  If not, see <http://www.gnu.org/licenses/>.
 #
-# Copyright 2014-2015  Jon Lund Steffensen <jon_steffensen@uri.edu>
+# Copyright 2014-2017  Jon Lund Steffensen <jon_steffensen@uri.edu>
 
 from __future__ import unicode_literals
 
@@ -49,10 +49,10 @@ def run(self):
         """Run charge balance command"""
 
         # Load compound information
-        compound_name = {}
-        for compound in self._model.parse_compounds():
-            compound_name[compound.id] = (
-                compound.name if compound.name is not None else compound.id)
+        def compound_name(id):
+            if id not in self._model.compounds:
+                return id
+            return self._model.compounds[id].properties.get('name', id)
 
         # Create a set of excluded reactions
         exclude = set(self._args.exclude)
@@ -71,8 +71,7 @@ def run(self):
                 unchecked += 1
             elif abs(charge) > self._args.epsilon:
                 unbalanced += 1
-                rxt = reaction.equation.translated_compounds(
-                    lambda x: compound_name.get(x, x))
+                rxt = reaction.equation.translated_compounds(compound_name)
                 print('{}\t{}\t{}'.format(reaction.id, charge, rxt))
 
         logger.info('Unbalanced reactions: {}/{}'.format(unbalanced, count))

psamm/commands/console.py

@@ -13,14 +13,14 @@
 # You should have received a copy of the GNU General Public License
 # along with PSAMM.  If not, see <http://www.gnu.org/licenses/>.
 #
-# Copyright 2014-2015  Jon Lund Steffensen <jon_steffensen@uri.edu>
+# Copyright 2014-2017  Jon Lund Steffensen <jon_steffensen@uri.edu>
 
 from __future__ import unicode_literals
 
-from ..command import Command, MetabolicMixin
+from ..command import Command
 
 
-class ConsoleCommand(MetabolicMixin, Command):
+class ConsoleCommand(Command):
     """Start an interactive Python console with the model loaded."""
 
     @classmethod
@@ -55,9 +55,10 @@ def open_ipython_kernel(self, message, namespace):
         embed_kernel(local_ns=namespace)
 
     def run(self):
-        message = ('Native model has been loaded into: "model"\n' +
-                   'Metabolic model has been loaded into: "mm"')
-        namespace = {'model': self._model, 'mm': self._mm}
+        message = ('Model has been loaded into: "model"\n' +
+                   'Use "model.create_metabolic_model() to create the'
+                   ' low-level metabolic model representation.')
+        namespace = {'model': self._model}
         console_type = self._args.type
 
         if console_type == 'python':