Framework for CLI applications

[dag]

Motivation

Though argparse is an improvement over optparse, it still feels more like a low-level toolkit for building more high-level systems. ConfigParser doesn't handle deep nesting and isn't type-aware, in itself.

Ideas

Many of these ideas come from, and are partially implemented in, my unpublished experimental rewrite of Attest, and the idea itself of a CLI framework is a result of it, after noticing a pattern and experiencing some annoyances.

• At the top sits an "application" object that implements the application logic which includes things like the configuration builder, the command-line argument parser etc
• When an application is run an "environment" object wraps contextual state such as os.environ, sys.args, a built config etc
  • But maybe config building should be up to the environment as a built config is immutable and one will likely build it conditionally based on the os.environ and sys.args etc
  • Maybe the environment thus could be an intermediary step towards a fixed "state" object (or simply put everything in the built config)
  • Overkill? :) My experience is that making objects do "too much" and with mutable state usually results in pain in one form or another.
• System for dependency injection from these objects for easy testing and reuse
• Configuration system based around primitive types
  • Nodes as dict-like bunches (done)
  • A system to merge configurations by recursing into nodes (done)
    • Support for includes in configuration files
  • Backends for reading configuration from files:
    • YAML, could use custom "tags" for directives (done)
    • ConfigParser, using dots in option keys for nesting nodes, and some form of automatic type conversion (done)
    • JSON, "because we can" (done)
    • Python, like Sphinx build configs, using ast.literal_eval (or something like it) in "safe" mode
  • Mapping of backends by filename extension: (done)
    • Smart resolution of config filenames, in order:
      • If the name refers to an existing file it is loaded
      • If the filename doesn't end in a known extension, try appending each known extension
      • If more than one file was resolved from the backend extensions, it could:
        • Load them all, but the order would be arbitrary
        • Load the first, but again the definition of "first" would be arbitrary
        • Skip this config name silently
        • Raise some conflict error - probably the best option
    • Load fully qualified config filenames by delegating to the backend mapped to the file extension
  • Syntax like Pyramid asset specs for loading configs with pkg_resources (done)
  • Schemas for untyped backends? Or alternatively infer types from string values. Allow pluggable converters?
    • Could generate documentation from a schema (sphinx/manage/--help etc)
    • Otherwise research if PyYAML supports reading nearby comments
  • Interpolation using newstyle string formatting
  • Some form of standardized handling of importable dotted names, with support for trying a sequence of dotted names (getting the first available one) and possibly handling of class instantiation and function arguments, similar to the functions in paste.util.import_string
  • API for "safe" reading of configuration, using yaml.safe_load and disable string importing etc.
    • Options for whether to fail for unsafe values or if to simply ignore them (leaving any previous or default values in place)
  • Maybe separate the concept of a configuration "builder" and a resulting immutable configuration "state" (though maybe load importable strings lazily) (done)
• Command-line argument parsing with argparse
  • Integration with the configuration system
    • At the most basic level, provide switches for setting and unsetting config options
    • Maybe optionally build the whole parser from the configuration, based either on a schema or the default values, and provide a means to map short options to long ones
  • Maybe expose a more high-level API
  • Maybe a system to expose functions based on argspec etc
  • Combinable sub-commands á la Paver and setuptools
  • Some way to merge multiple parsers into one (does argparse do this already?)
  • Maybe include an improved help formatter class - I don't quite like any of the built in ones
    • Could use ansi escapes to tty outputs
• XHTML-like markup language for terminal output
  • Either built in templating or use Genshi for dynamic functionality
  • Styling with CSS
  • Attribute on text elements for Pygments lexer
  • Handle things like automatic wrapping of content marked up as paragraphs, based on width of terminal, padding etc
  • All this might be too much work :) but the idea is to make outputting to the terminal less imperative and more declarative and semantically rich, to make it easier to test, refactor, reuse ... XML is used because while it's rather crappy for data serialization, marking up text with semantics is where it shines.
• Support for plugging in configurable...
  • Logging
  • Daemonization
  • Diffing, with support for Pygments and the various formats supported by difflib
  • Paging

Existing Options

• http://packages.python.org/pyCLI/
• http://builtoncement.org/

Name ideas

• "mainly": it's a tool for the "main" function
• words with "cli" in them: climactic/(anti-)climax, clingy, clinic, decline, incline, disincline
• words with "tty" in them: kitty, bratty, chatty, fatty, petty, potty, pretty, shitty, witty
• words with "term" in them: determination, intermediary, mastermind, terminology, termite, watermelon