From 96e6111ea228f9019d8bb35d2665e3c04ea53ae9 Mon Sep 17 00:00:00 2001 From: Aki Sasaki Date: Sat, 20 Jun 2015 21:43:46 -0700 Subject: [PATCH 1/3] Docs: link everything! --- docs/commands.rst | 39 ++++++++++++++++++++------------------ docs/config.rst | 48 +++++++++++++++++++++++++++++++---------------- 2 files changed, 53 insertions(+), 34 deletions(-) diff --git a/docs/commands.rst b/docs/commands.rst index 10d10b4..68d1965 100644 --- a/docs/commands.rst +++ b/docs/commands.rst @@ -7,15 +7,13 @@ Commands Command and run() ################# -The ``Command`` object simply takes an external command and runs it, logging stdout and stderr as each message arrives. The main benefits of using ``Command`` are logging and timeouts. ``Command`` takes two timeouts: ``output_timeout``, which is how long the command can go without outputting anything before timing out, and ``max_timeout``, which is the total amount of time that can elapse from the start of the command. +The Command_ object simply takes an external command and runs it, logging stdout and stderr as each message arrives. The main benefits of using Command_ are logging and timeouts. Command_ takes two timeouts: ``output_timeout``, which is how long the command can go without outputting anything before timing out, and ``max_timeout``, which is the total amount of time that can elapse from the start of the command. (The command is run via ``subprocess.Popen`` and timeouts are monitored via the `multiprocessing` module.) After the command is run, it runs the ``detect_error_cb`` callback function to determine whether the command was run successfully. -The process of creating and running a ``Command`` is twofold: ``Command.__init__`` and ``Command.run()``. As a shortcut, there is a run_ function that will do both steps for you. - -.. _run: scriptharness.commands.html#scriptharness.commands.run +The process of creating and running a Command_ is twofold: `Command.__init__`_ and `Command.run`_. As a shortcut, there is a run_ function that will do both steps for you. .. _ParsedCommand-and-parse: @@ -26,11 +24,9 @@ ParsedCommand and parse() Ideally, external command output would be for humans only, and the exit code would be meaningful. In practice, this is not always the case. Exit codes aren't always helpful or even meaningful, and sometimes critical information is buried in a flood of output. -``ParsedCommand`` takes the output of a command and parses it for matching substrings or regular expressions, using :ref:`ErrorLists-and-OutputParser` to determine the log level of a line of output. Because it subclasses ``Command``, ``ParsedCommand`` also has built-in ``output_timeout`` and ``max_timeout`` support. +ParsedCommand_ takes the output of a command and parses it for matching substrings or regular expressions, using :ref:`ErrorLists-and-OutputParser` to determine the log level of a line of output. Because it subclasses Command_, ParsedCommand_ also has built-in ``output_timeout`` and ``max_timeout`` support. -As with ``Command`` and ``run()``, ``ParsedCommand`` has a shortcut function, parse_. - -.. _parse: scriptharness.commands.html#scriptharness.commands.parse +As with Command_ and run_, ParsedCommand_ has a shortcut function, parse_. .. _ErrorLists-and-OutputParser: @@ -41,8 +37,6 @@ ErrorLists and OutputParser The ErrorList_ object describes which lines of output are of special interest. It's a class for better validation. -.. _ErrorList: scriptharness.errorlists.html#scriptharness.errorlists.ErrorList - An example error_list:: [ @@ -66,9 +60,8 @@ Any output line that matches the first regex will be ignored (discarded), becaus The final substring has an explanation that will be logged immediately after the matching line, to explain vague error messages. Because it has a defined `exception`, it will raise. -``ParsedCommand`` sends its output to the OutputParser_ object, which passes it on to the ``ErrorList``. It keeps track of the number of errors and warnings, as well as handling any context line buffering through the ``OutputBuffer``. +ParsedCommand_ sends its output to the OutputParser_ object, which passes it on to the ErrorList_. It keeps track of the number of errors and warnings, as well as handling any context line buffering through the OutputBuffer_. -.. _OutputParser: scriptharness.log.html#scriptharness.log.OutputParser .. _OutputBuffer-and-context-lines: @@ -90,8 +83,6 @@ The OutputBuffer_ holds the buffered output for ``pre_context_lines``, and keeps If multiple lines match, and a line of output is marked as multiple levels, the highest level will win. E.g., ``logging.CRITICAL`` will beat ``logging.ERROR``, which will beat ``logging.WARNING``, etc. -.. _OutputBuffer: scriptharness.log.html#scriptharness.log.OutputBuffer - .. _Output-get_output-and-get_text_output: @@ -101,11 +92,23 @@ Output, get_output(), and get_text_output() Sometimes you need to manipulate the output from a command, not just log it or perform general error parsing. There's ``subprocess.check_output()``, but that doesn't log or have full timeout support. -Enter Output_. This also inherits ``Command``, but because ``Output.run`` is a completely different method than ``Command.run``, it has its own timeout implementation. (It does still support both ``output_timeout`` and ``max_timeout``.) It redirects STDOUT and STDERR to temp files. +Enter Output_. This also inherits Command_, but because Output.run_ is a completely different method than Command.run_, it has its own timeout implementation. (It does still support both ``output_timeout`` and ``max_timeout``.) It redirects STDOUT and STDERR to temp files. -.. _Output: scriptharness.commands.html#scriptharness.commands.Output - -Much like ``Command`` has its helper ``run`` function, ``Output`` has `two` helper functions: get_output_ and get_text_output_. The former yields the ``Output`` object, and the caller can either access the ``NamedTemporaryFile`` ``Output.stdout`` and ``Output.stderr`` objects, or use the ``Output.get_output()`` method. Because of this, it is suitable for binary or lengthy output. ``get_text_output()`` will get the STDOUT contents for you, log them, and return them to you. +Much like Command_ has its helper run_ function, Output_ has `two` helper functions: get_output_ and get_text_output_. The former yields the Output_ object, and the caller can either access the ``NamedTemporaryFile`` Output.stdout_ and Output.stderr_ objects, or use the Output.get_output_ method. Because of this, it is suitable for binary or lengthy output. get_text_output_ will get the STDOUT contents for you, log them, and return them to you. +.. _Command: scriptharness.commands.html#scriptharness.commands.Command +.. _Command.__init__: scriptharness.commands.html#scriptharness.commands.Command.__init__ +.. _Command.run: scriptharness.commands.html#scriptharness.commands.Command.run +.. _ErrorList: scriptharness.errorlists.html#scriptharness.errorlists.ErrorList +.. _Output: scriptharness.commands.html#scriptharness.commands.Output +.. _Output.get_output: scriptharness.commands.html#scriptharness.commands.Output.get_output +.. _Output.run: scriptharness.commands.html#scriptharness.commands.Output.run +.. _Output.stdout: scriptharness.commands.html#scriptharness.commands.Output.stdout +.. _Output.stderr: scriptharness.commands.html#scriptharness.commands.Output.stderr +.. _OutputBuffer: scriptharness.log.html#scriptharness.log.OutputBuffer +.. _OutputParser: scriptharness.log.html#scriptharness.log.OutputParser +.. _ParsedCommand: scriptharness.commands.html#scriptharness.commands.ParsedCommand .. _get_output: scriptharness.commands.html#scriptharness.commands.get_output .. _get_text_output: scriptharness.commands.html#scriptharness.commands.get_text_output +.. _parse: scriptharness.commands.html#scriptharness.commands.parse +.. _run: scriptharness.commands.html#scriptharness.commands.run diff --git a/docs/config.rst b/docs/config.rst index f4d8aaf..c7930be 100644 --- a/docs/config.rst +++ b/docs/config.rst @@ -7,11 +7,11 @@ Configuration Overview The runtime configuration of a Script is built from several layers. -* There is a ``ConfigTemplate`` that can have default values for certain config variables. These defaults are the basis of the config dict. (See :ref:`Config-Templates` for more details on ``ConfigTemplates``). +* There is a ConfigTemplate_ that can have default values for certain config variables. These defaults are the basis of the config dict. (See :ref:`Config-Templates` for more details on ConfigTemplate_). -* The script can define an ``initial_config`` dict that is laid on top of the ``ConfigTemplate`` defaults, so any shared config variables are overwritten by the ``initial_config``. +* The script can define an ``initial_config`` dict that is laid on top of the ConfigTemplate_ defaults, so any shared config variables are overwritten by the ``initial_config``. -* The ``ConfigTemplate`` has a ``get_parser()`` method that generates an ``argparse.ArgumentParser``. This parser parses the commandline options. +* The ConfigTemplate.get_parser_ method generates an ``argparse.ArgumentParser``. This parser parses the commandline options. * If the commandline options specify any files via the ``--config-file`` option, then those files are read, and the contents are overlaid on top of the config. The first file specified will be overlaid first, then the second, and so on. @@ -30,30 +30,30 @@ ConfigTemplates It's very powerful to be able to build a configuration dict that can hold any key value pairs, but it's non-trivial for users to verify if their config is valid or if there are options that they're not taking advantage of. -To make the config more well-defined, we have the ``ConfigTemplate``. The ``ConfigTemplate`` is comprised of ``ConfigVariables``, and is based on the ``argparse.ArgumentParser``, but with these qualities: +To make the config more well-defined, we have the ConfigTemplate_. The ConfigTemplate_ is comprised of ConfigVariable_ objects, and is based on the ``argparse.ArgumentParser``, but with these qualities: -* The ``ConfigTemplate`` can keep track of all config variables, including ones that aren't available as commandline options. The option-less config variables must be specified via default, config file, or ``initial_config``. +* The ConfigTemplate_ can keep track of all config variables, including ones that aren't available as commandline options. The option-less config variables must be specified via default, config file, or ``initial_config``. -* The templates can be added together, via ``ConfigTemplate.update()``. +* The templates can be added together, via ConfigTemplate.update_. -* Each ``ConfigVariable`` self-validates, and the ``ConfigTemplate`` makes sure there are no conflicting commandline options. +* Each ConfigVariable_ self-validates, and the ConfigTemplate_ makes sure there are no conflicting commandline options. -* There is a ``ConfigTemplate.remove_option()`` method to remove a commandline option from the corresponding ``ConfigVariable``. This may be needed if you want to add two config templates together, but they both have a ``-f`` commandline option specified, for example. +* There is a ConfigTemplate.remove_option_ method to remove a commandline option from the corresponding ConfigVariable_. This may be needed if you want to add two config templates together, but they both have a ``-f`` commandline option specified, for example. -* The ``ConfigTemplate.validate_config()`` method validates the built configuration. Each ``ConfigVariable`` can define whether they're required, whether they require or are incompatible with other variables (``required_vars`` and ``incompatible_vars``), and each can define their own ``validate_cb`` callback function. +* The ConfigTemplate.validate_config_ method validates the built configuration. Each ConfigVariable_ can define whether they're required, whether they require or are incompatible with other variables (``required_vars`` and ``incompatible_vars``), and each can define their own ``validate_cb`` callback function. -* There is a ``ConfigTemplate.add_argument()`` for those who want to maintain argparse syntax. +* There is a ConfigTemplate.add_argument_ for those who want to maintain argparse syntax. -Parent parsers are supported, to group commandline options in the ``--help`` output. Subparsers are not currently supported, though it may be possible to replace the ``ConfigTemplate.parser`` with a subparser-enabled parser at the expense of validation and the ability to ``ConfigTemplate.update()``. +Parent parsers are supported, to group commandline options in the ``--help`` output. Subparsers are not currently supported, though it may be possible to replace the ConfigTemplate.parser_ with a subparser-enabled parser at the expense of validation and the ability to ConfigTemplate.update_. -When supporting downstream scripts, it's best to keep each ``ConfigTemplate`` modular. It's easy to combine them via ``ConfigTemplate.update()``, but less trivial to remove functionality. The action config template, for instance, can be added to the base config template right before running ``parse_args()``. +When supporting downstream scripts, it's best to keep each ConfigTemplate_ modular. It's easy to combine them via ConfigTemplate.update_, but less trivial to remove functionality. The action config template, for instance, can be added to the base config template right before running parse_args_. ############################ LoggingDict and ReadOnlyDict ############################ -Each Script has a config dict. By default, this dict is a ``LoggingDict``, which logs any changes made to the config. +Each Script_ has a config dict. By default, this dict is a LoggingDict_, which logs any changes made to the config. For example, if the config looked like:: @@ -71,7 +71,7 @@ then updating the config might log:: 00:11:22 INFO - root.config['baz'] update: y now 8 -Alternatively, someone could change the script class to ``StrictScript``, which uses ``ReadOnlyDict``. Once the ``ReadOnlyDict`` is locked, it cannot be modified. +Alternatively, someone could change the script class to StrictScript_, which uses ReadOnlyDict_. Once the ReadOnlyDict_ is locked, it cannot be modified. By either explicitly logging any changes to the config, and/or preventing any changes to the config, it's easier to debug any unexpected behavior. @@ -82,12 +82,28 @@ By either explicitly logging any changes to the config, and/or preventing any ch Contexts ######## -As each Action is run, it passes a ``Context`` to the action function. The ``Context`` is a ``namedtuple`` with the following properties: +As each Action is run, it passes a Context_ to the action function. The Context_ is a ``namedtuple`` with the following properties: * script (Script object): the Script calling the Action -* config (dict): by default this is a ``LoggingDict`` +* config (dict): by default this is a LoggingDict_ * logger (logging.Logger): the logger for the Script * action (Action object): this is only defined during the ``RUN_ACTION``, ``PRE_ACTION``, and ``POST_ACTION`` phases. * phase (str): this will be one of ``PRE_RUN``, ``POST_RUN``, ``PRE_ACTION``, ``POST_ACTION``, or ``POST_FATAL``, depending on which phase we're in. The logger and config (and to a lesser degree, the script and action) objects are all available to each function called for convenience and consistency. + + +.. _ConfigTemplate: scriptharness.config.html#scriptharness.config.ConfigTemplate +.. _ConfigTemplate.add_argument: scriptharness.config.html#scriptharness.config.ConfigTemplate.add_argument +.. _ConfigTemplate.get_parser: scriptharness.config.html#scriptharness.config.ConfigTemplate.get_parser +.. _ConfigTemplate.parser: scriptharness.config.html#scriptharness.config.ConfigTemplate.parser +.. _ConfigTemplate.remove_option: scriptharness.config.html#scriptharness.config.ConfigTemplate.remove_option +.. _ConfigTemplate.update: scriptharness.config.html#scriptharness.config.ConfigTemplate.update +.. _ConfigTemplate.validate_config: scriptharness.config.html#scriptharness.config.ConfigTemplate.validate_config +.. _ConfigVariable: scriptharness.config.html#scriptharness.config.ConfigVariable +.. _Context: scriptharness.script.html#scriptharness.script.Context +.. _LoggingDict: scriptharness.structures.html#scriptharness.structures.LoggingDict +.. _ReadOnlyDict: scriptharness.structures.html#scriptharness.structures.ReadOnlyDict +.. _Script: scriptharness.script.html#scriptharness.script.Script +.. _StrictScript: scriptharness.script.html#scriptharness.script.StrictScript +.. _parse_args: scriptharness.config.html#scriptharness.config.parse_args From d9e0a3bf37d313f6a3bd0155e8dc1ffd416ead40 Mon Sep 17 00:00:00 2001 From: Aki Sasaki Date: Sat, 20 Jun 2015 21:52:54 -0700 Subject: [PATCH 2/3] Neatened links! add () to function/method names --- docs/commands.rst | 24 ++++++++++++------------ docs/config.rst | 26 +++++++++++++------------- docs/quickstart.rst | 35 ++++++++++++++++++----------------- docs/quickstart.rst.j2 | 6 +++--- 4 files changed, 46 insertions(+), 45 deletions(-) diff --git a/docs/commands.rst b/docs/commands.rst index 68d1965..c7af405 100644 --- a/docs/commands.rst +++ b/docs/commands.rst @@ -13,7 +13,7 @@ The Command_ object simply takes an external command and runs it, logging stdout After the command is run, it runs the ``detect_error_cb`` callback function to determine whether the command was run successfully. -The process of creating and running a Command_ is twofold: `Command.__init__`_ and `Command.run`_. As a shortcut, there is a run_ function that will do both steps for you. +The process of creating and running a Command_ is twofold: `Command.__init__()`_ and `Command.run()`_. As a shortcut, there is a `run()`_ function that will do both steps for you. .. _ParsedCommand-and-parse: @@ -26,7 +26,7 @@ Ideally, external command output would be for humans only, and the exit code wou ParsedCommand_ takes the output of a command and parses it for matching substrings or regular expressions, using :ref:`ErrorLists-and-OutputParser` to determine the log level of a line of output. Because it subclasses Command_, ParsedCommand_ also has built-in ``output_timeout`` and ``max_timeout`` support. -As with Command_ and run_, ParsedCommand_ has a shortcut function, parse_. +As with Command_ and `run()`_, ParsedCommand_ has a shortcut function, `parse()`_. .. _ErrorLists-and-OutputParser: @@ -92,23 +92,23 @@ Output, get_output(), and get_text_output() Sometimes you need to manipulate the output from a command, not just log it or perform general error parsing. There's ``subprocess.check_output()``, but that doesn't log or have full timeout support. -Enter Output_. This also inherits Command_, but because Output.run_ is a completely different method than Command.run_, it has its own timeout implementation. (It does still support both ``output_timeout`` and ``max_timeout``.) It redirects STDOUT and STDERR to temp files. +Enter Output_. This also inherits Command_, but because `Output.run()`_ is a completely different method than `Command.run()`_, it has its own timeout implementation. (It does still support both ``output_timeout`` and ``max_timeout``.) It redirects STDOUT and STDERR to temp files. -Much like Command_ has its helper run_ function, Output_ has `two` helper functions: get_output_ and get_text_output_. The former yields the Output_ object, and the caller can either access the ``NamedTemporaryFile`` Output.stdout_ and Output.stderr_ objects, or use the Output.get_output_ method. Because of this, it is suitable for binary or lengthy output. get_text_output_ will get the STDOUT contents for you, log them, and return them to you. +Much like Command_ has its helper `run()`_ function, Output_ has `two` helper functions: `get_output()`_ and `get_text_output()`_. The former yields the Output_ object, and the caller can either access the ``NamedTemporaryFile`` Output.stdout_ and Output.stderr_ objects, or use the `Output.get_output()`_ method. Because of this, it is suitable for binary or lengthy output. `get_text_output()`_ will get the STDOUT contents for you, log them, and return them to you. .. _Command: scriptharness.commands.html#scriptharness.commands.Command -.. _Command.__init__: scriptharness.commands.html#scriptharness.commands.Command.__init__ -.. _Command.run: scriptharness.commands.html#scriptharness.commands.Command.run +.. _Command.__init__(): scriptharness.commands.html#scriptharness.commands.Command.__init__ +.. _Command.run(): scriptharness.commands.html#scriptharness.commands.Command.run .. _ErrorList: scriptharness.errorlists.html#scriptharness.errorlists.ErrorList .. _Output: scriptharness.commands.html#scriptharness.commands.Output -.. _Output.get_output: scriptharness.commands.html#scriptharness.commands.Output.get_output -.. _Output.run: scriptharness.commands.html#scriptharness.commands.Output.run +.. _Output.get_output(): scriptharness.commands.html#scriptharness.commands.Output.get_output +.. _Output.run(): scriptharness.commands.html#scriptharness.commands.Output.run .. _Output.stdout: scriptharness.commands.html#scriptharness.commands.Output.stdout .. _Output.stderr: scriptharness.commands.html#scriptharness.commands.Output.stderr .. _OutputBuffer: scriptharness.log.html#scriptharness.log.OutputBuffer .. _OutputParser: scriptharness.log.html#scriptharness.log.OutputParser .. _ParsedCommand: scriptharness.commands.html#scriptharness.commands.ParsedCommand -.. _get_output: scriptharness.commands.html#scriptharness.commands.get_output -.. _get_text_output: scriptharness.commands.html#scriptharness.commands.get_text_output -.. _parse: scriptharness.commands.html#scriptharness.commands.parse -.. _run: scriptharness.commands.html#scriptharness.commands.run +.. _get_output(): scriptharness.commands.html#scriptharness.commands.get_output +.. _get_text_output(): scriptharness.commands.html#scriptharness.commands.get_text_output +.. _parse(): scriptharness.commands.html#scriptharness.commands.parse +.. _run(): scriptharness.commands.html#scriptharness.commands.run diff --git a/docs/config.rst b/docs/config.rst index c7930be..6053d7e 100644 --- a/docs/config.rst +++ b/docs/config.rst @@ -11,7 +11,7 @@ The runtime configuration of a Script is built from several layers. * The script can define an ``initial_config`` dict that is laid on top of the ConfigTemplate_ defaults, so any shared config variables are overwritten by the ``initial_config``. -* The ConfigTemplate.get_parser_ method generates an ``argparse.ArgumentParser``. This parser parses the commandline options. +* The `ConfigTemplate.get_parser()`_ method generates an ``argparse.ArgumentParser``. This parser parses the commandline options. * If the commandline options specify any files via the ``--config-file`` option, then those files are read, and the contents are overlaid on top of the config. The first file specified will be overlaid first, then the second, and so on. @@ -34,19 +34,19 @@ To make the config more well-defined, we have the ConfigTemplate_. The ConfigTe * The ConfigTemplate_ can keep track of all config variables, including ones that aren't available as commandline options. The option-less config variables must be specified via default, config file, or ``initial_config``. -* The templates can be added together, via ConfigTemplate.update_. +* The templates can be added together, via `ConfigTemplate.update()`_. * Each ConfigVariable_ self-validates, and the ConfigTemplate_ makes sure there are no conflicting commandline options. -* There is a ConfigTemplate.remove_option_ method to remove a commandline option from the corresponding ConfigVariable_. This may be needed if you want to add two config templates together, but they both have a ``-f`` commandline option specified, for example. +* There is a `ConfigTemplate.remove_option()`_ method to remove a commandline option from the corresponding ConfigVariable_. This may be needed if you want to add two config templates together, but they both have a ``-f`` commandline option specified, for example. -* The ConfigTemplate.validate_config_ method validates the built configuration. Each ConfigVariable_ can define whether they're required, whether they require or are incompatible with other variables (``required_vars`` and ``incompatible_vars``), and each can define their own ``validate_cb`` callback function. +* The `ConfigTemplate.validate_config()`_ method validates the built configuration. Each ConfigVariable_ can define whether they're required, whether they require or are incompatible with other variables (``required_vars`` and ``incompatible_vars``), and each can define their own ``validate_cb`` callback function. -* There is a ConfigTemplate.add_argument_ for those who want to maintain argparse syntax. +* There is a `ConfigTemplate.add_argument()`_ for those who want to maintain argparse syntax. -Parent parsers are supported, to group commandline options in the ``--help`` output. Subparsers are not currently supported, though it may be possible to replace the ConfigTemplate.parser_ with a subparser-enabled parser at the expense of validation and the ability to ConfigTemplate.update_. +Parent parsers are supported, to group commandline options in the ``--help`` output. Subparsers are not currently supported, though it may be possible to replace the ConfigTemplate.parser_ with a subparser-enabled parser at the expense of validation and the ability to `ConfigTemplate.update()`_. -When supporting downstream scripts, it's best to keep each ConfigTemplate_ modular. It's easy to combine them via ConfigTemplate.update_, but less trivial to remove functionality. The action config template, for instance, can be added to the base config template right before running parse_args_. +When supporting downstream scripts, it's best to keep each ConfigTemplate_ modular. It's easy to combine them via `ConfigTemplate.update()`_, but less trivial to remove functionality. The action config template, for instance, can be added to the base config template right before running `parse_args()`_. ############################ @@ -94,16 +94,16 @@ The logger and config (and to a lesser degree, the script and action) objects ar .. _ConfigTemplate: scriptharness.config.html#scriptharness.config.ConfigTemplate -.. _ConfigTemplate.add_argument: scriptharness.config.html#scriptharness.config.ConfigTemplate.add_argument -.. _ConfigTemplate.get_parser: scriptharness.config.html#scriptharness.config.ConfigTemplate.get_parser +.. _ConfigTemplate.add_argument(): scriptharness.config.html#scriptharness.config.ConfigTemplate.add_argument +.. _ConfigTemplate.get_parser(): scriptharness.config.html#scriptharness.config.ConfigTemplate.get_parser .. _ConfigTemplate.parser: scriptharness.config.html#scriptharness.config.ConfigTemplate.parser -.. _ConfigTemplate.remove_option: scriptharness.config.html#scriptharness.config.ConfigTemplate.remove_option -.. _ConfigTemplate.update: scriptharness.config.html#scriptharness.config.ConfigTemplate.update -.. _ConfigTemplate.validate_config: scriptharness.config.html#scriptharness.config.ConfigTemplate.validate_config +.. _ConfigTemplate.remove_option(): scriptharness.config.html#scriptharness.config.ConfigTemplate.remove_option +.. _ConfigTemplate.update(): scriptharness.config.html#scriptharness.config.ConfigTemplate.update +.. _ConfigTemplate.validate_config(): scriptharness.config.html#scriptharness.config.ConfigTemplate.validate_config .. _ConfigVariable: scriptharness.config.html#scriptharness.config.ConfigVariable .. _Context: scriptharness.script.html#scriptharness.script.Context .. _LoggingDict: scriptharness.structures.html#scriptharness.structures.LoggingDict .. _ReadOnlyDict: scriptharness.structures.html#scriptharness.structures.ReadOnlyDict .. _Script: scriptharness.script.html#scriptharness.script.Script .. _StrictScript: scriptharness.script.html#scriptharness.script.StrictScript -.. _parse_args: scriptharness.config.html#scriptharness.config.parse_args +.. _parse_args(): scriptharness.config.html#scriptharness.config.parse_args diff --git a/docs/quickstart.rst b/docs/quickstart.rst index b0e2575..b9795c1 100644 --- a/docs/quickstart.rst +++ b/docs/quickstart.rst @@ -3,9 +3,7 @@ Quickstart ========== -Here's an example script. The file is also viewable here_. - -.. _here: https://github.com/scriptharness/python-scriptharness/blob/master/examples/quickstart.py +Here's an example script, quickstart.py_. :: @@ -241,31 +239,34 @@ see what the config is without running anything, you can use the You can always use the ``--help`` option:: $ ./quickstart.py --help - usage: quickstart.py [-h] [--config-file CONFIG_FILE] [--dump-config] - [--opt-config-file CONFIG_FILE] - [--add-actions ACTION [ACTION ...]] - [--action-group {none,all}] [--list-actions] - [--actions ACTION [ACTION ...]] + usage: quickstart.py [-h] [--opt-config-file CONFIG_FILE] [--dump-config] + [--config-file CONFIG_FILE] [--skip-actions ACTION [ACTION ...]] + [--actions ACTION [ACTION ...]] + [--action-group {none,all}] [--list-actions] + [--add-actions ACTION [ACTION ...]] [--new-argument NEW_ARGUMENT] optional arguments: -h, --help show this help message and exit - --config-file CONFIG_FILE, --cfg CONFIG_FILE, -c CONFIG_FILE - Specify required config files/urls - --dump-config Log the built configuration and exit. --opt-config-file CONFIG_FILE, --opt-cfg CONFIG_FILE Specify optional config files/urls - --add-actions ACTION [ACTION ...] - Specify the actions to add to the default set. + --dump-config Log the built configuration and exit. + --config-file CONFIG_FILE, --cfg CONFIG_FILE, -c CONFIG_FILE + Specify required config files/urls + --skip-actions ACTION [ACTION ...] + Specify the actions to skip. + --actions ACTION [ACTION ...] + Specify the actions to run. --action-group {none,all} Specify the action group to use. --list-actions List all actions (default prepended with '*') and exit. - --actions ACTION [ACTION ...] - Specify the actions to run. - --skip-actions ACTION [ACTION ...] - Specify the actions to skip. + --add-actions ACTION [ACTION ...] + Specify the actions to add to the default set. --new-argument NEW_ARGUMENT help message for --new-argument + + +.. _quickstart.py: https://github.com/scriptharness/python-scriptharness/blob/master/examples/quickstart.py \ No newline at end of file diff --git a/docs/quickstart.rst.j2 b/docs/quickstart.rst.j2 index 6624b28..9de5588 100755 --- a/docs/quickstart.rst.j2 +++ b/docs/quickstart.rst.j2 @@ -3,9 +3,7 @@ Quickstart ========== -Here's an example script. The file is also viewable here_. - -.. _here: https://github.com/scriptharness/python-scriptharness/blob/{{git_branch}}/examples/quickstart.py +Here's an example script, quickstart.py_. :: @@ -71,3 +69,5 @@ You can always use the ``--help`` option:: $ ./quickstart.py --help {{help_output}} + +.. _quickstart.py: https://github.com/scriptharness/python-scriptharness/blob/{{git_branch}}/examples/quickstart.py From e76ba5975c0c9fabda1e356e4b3a385701bea088 Mon Sep 17 00:00:00 2001 From: Aki Sasaki Date: Sat, 20 Jun 2015 22:55:18 -0700 Subject: [PATCH 3/3] Script docs and other doc cleanups --- README | 1 + README.rst | 1 + docs/README.rst | 1 + docs/README.rst.j2 | 1 + docs/config.rst | 20 +------------- docs/index.rst | 1 + docs/quickstart.rst | 20 +++++++------- docs/script.rst | 60 +++++++++++++++++++++++++++++++++++++++++ scriptharness/script.py | 1 + 9 files changed, 77 insertions(+), 29 deletions(-) create mode 100644 docs/script.rst diff --git a/README b/README index 70a5089..e0b3b4c 100644 --- a/README +++ b/README @@ -71,6 +71,7 @@ Linux and OS X # You can run |tox -e ENV| to run a specific env, e.g. |tox -e py27| pip install tox tox + # alternately, ./run_tests.sh Windows diff --git a/README.rst b/README.rst index fbfaf92..fe827a0 100644 --- a/README.rst +++ b/README.rst @@ -51,6 +51,7 @@ Linux and OS X # You can run |tox -e ENV| to run a specific env, e.g. |tox -e py27| pip install tox tox + # alternately, ./run_tests.sh Windows ------- diff --git a/docs/README.rst b/docs/README.rst index 173a2c3..cf37849 100644 --- a/docs/README.rst +++ b/docs/README.rst @@ -47,6 +47,7 @@ Linux and OS X # You can run |tox -e ENV| to run a specific env, e.g. |tox -e py27| pip install tox tox + # alternately, ./run_tests.sh Windows ------- diff --git a/docs/README.rst.j2 b/docs/README.rst.j2 index ba9d712..b58b7ef 100644 --- a/docs/README.rst.j2 +++ b/docs/README.rst.j2 @@ -47,6 +47,7 @@ Linux and OS X # You can run |tox -e ENV| to run a specific env, e.g. |tox -e py27| pip install tox tox + # alternately, ./run_tests.sh Windows ------- diff --git a/docs/config.rst b/docs/config.rst index 6053d7e..79e968b 100644 --- a/docs/config.rst +++ b/docs/config.rst @@ -5,7 +5,7 @@ Configuration Configuration Overview ###################### -The runtime configuration of a Script is built from several layers. +The runtime configuration of a Script_ is built from several layers. * There is a ConfigTemplate_ that can have default values for certain config variables. These defaults are the basis of the config dict. (See :ref:`Config-Templates` for more details on ConfigTemplate_). @@ -76,23 +76,6 @@ Alternatively, someone could change the script class to StrictScript_, which use By either explicitly logging any changes to the config, and/or preventing any changes to the config, it's easier to debug any unexpected behavior. -.. _Contexts: - -######## -Contexts -######## - -As each Action is run, it passes a Context_ to the action function. The Context_ is a ``namedtuple`` with the following properties: - -* script (Script object): the Script calling the Action -* config (dict): by default this is a LoggingDict_ -* logger (logging.Logger): the logger for the Script -* action (Action object): this is only defined during the ``RUN_ACTION``, ``PRE_ACTION``, and ``POST_ACTION`` phases. -* phase (str): this will be one of ``PRE_RUN``, ``POST_RUN``, ``PRE_ACTION``, ``POST_ACTION``, or ``POST_FATAL``, depending on which phase we're in. - -The logger and config (and to a lesser degree, the script and action) objects are all available to each function called for convenience and consistency. - - .. _ConfigTemplate: scriptharness.config.html#scriptharness.config.ConfigTemplate .. _ConfigTemplate.add_argument(): scriptharness.config.html#scriptharness.config.ConfigTemplate.add_argument .. _ConfigTemplate.get_parser(): scriptharness.config.html#scriptharness.config.ConfigTemplate.get_parser @@ -101,7 +84,6 @@ The logger and config (and to a lesser degree, the script and action) objects ar .. _ConfigTemplate.update(): scriptharness.config.html#scriptharness.config.ConfigTemplate.update .. _ConfigTemplate.validate_config(): scriptharness.config.html#scriptharness.config.ConfigTemplate.validate_config .. _ConfigVariable: scriptharness.config.html#scriptharness.config.ConfigVariable -.. _Context: scriptharness.script.html#scriptharness.script.Context .. _LoggingDict: scriptharness.structures.html#scriptharness.structures.LoggingDict .. _ReadOnlyDict: scriptharness.structures.html#scriptharness.structures.ReadOnlyDict .. _Script: scriptharness.script.html#scriptharness.script.Script diff --git a/docs/index.rst b/docs/index.rst index b00e2c1..097944e 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -12,6 +12,7 @@ quickstart enabling_actions config + script commands scriptharness releasenotes diff --git a/docs/quickstart.rst b/docs/quickstart.rst index b9795c1..17eee6b 100644 --- a/docs/quickstart.rst +++ b/docs/quickstart.rst @@ -239,31 +239,31 @@ see what the config is without running anything, you can use the You can always use the ``--help`` option:: $ ./quickstart.py --help - usage: quickstart.py [-h] [--opt-config-file CONFIG_FILE] [--dump-config] - [--config-file CONFIG_FILE] - [--skip-actions ACTION [ACTION ...]] - [--actions ACTION [ACTION ...]] + usage: quickstart.py [-h] [--dump-config] [--config-file CONFIG_FILE] + [--opt-config-file CONFIG_FILE] [--action-group {none,all}] [--list-actions] + [--actions ACTION [ACTION ...]] [--add-actions ACTION [ACTION ...]] + [--skip-actions ACTION [ACTION ...]] [--new-argument NEW_ARGUMENT] optional arguments: -h, --help show this help message and exit - --opt-config-file CONFIG_FILE, --opt-cfg CONFIG_FILE - Specify optional config files/urls --dump-config Log the built configuration and exit. --config-file CONFIG_FILE, --cfg CONFIG_FILE, -c CONFIG_FILE Specify required config files/urls - --skip-actions ACTION [ACTION ...] - Specify the actions to skip. - --actions ACTION [ACTION ...] - Specify the actions to run. + --opt-config-file CONFIG_FILE, --opt-cfg CONFIG_FILE + Specify optional config files/urls --action-group {none,all} Specify the action group to use. --list-actions List all actions (default prepended with '*') and exit. + --actions ACTION [ACTION ...] + Specify the actions to run. --add-actions ACTION [ACTION ...] Specify the actions to add to the default set. + --skip-actions ACTION [ACTION ...] + Specify the actions to skip. --new-argument NEW_ARGUMENT help message for --new-argument diff --git a/docs/script.rst b/docs/script.rst new file mode 100644 index 0000000..6c4c8e1 --- /dev/null +++ b/docs/script.rst @@ -0,0 +1,60 @@ +Scripts and Actions +=================== + +################## +Scripts and Phases +################## + +The Script_ is generally what one would think of as the script itself: it parses the commandline arguments and runs each enabled Action_. There's the possibility of enabling `running multiple Scripts in parallel`_ at some point. + +It's possible to add callbacks, called listeners, to the Script_. These get triggered in phases. The list of phases are in ALL_PHASES_; the phases that allow listeners are in LISTENER_PHASES_. + +* The ``PRE_RUN`` phase is first, before any actions are run. +* The ``PRE_ACTION`` phase happens before every enabled action, but a listener can be added to a subset of those actions if desired. +* The ``ACTION`` phase is when the enabled Action_ is run. No listener can be added to the ``ACTION`` phase. +* The ``POST_ACTION`` phase happens after every enabled action, but a listener can be added to a subset of those actions if desired. +* The ``POST_RUN`` phase happens after all enabled actions are run. +* The ``POST_FATAL`` phase happens after a ScriptHarnessFatal_ exception is raised, but before the script exits. + + +.. _Contexts: + +######## +Contexts +######## + +Each listener or Action_ function is passed a Context_. The Context_ is a ``namedtuple`` with the following properties: + +* script (Script_): the Script_ calling the function +* config (dict): by default this is a LoggingDict_ +* logger (logging.Logger): the logger for the Script_ +* action (Action_): this is only defined during the ``RUN_ACTION``, ``PRE_ACTION``, and ``POST_ACTION`` phases; it is ``None`` in the other phases. +* phase (str): this will be one of ``PRE_RUN``, ``POST_RUN``, ``PRE_ACTION``, ``POST_ACTION``, or ``POST_FATAL``, depending on which phase we're in. + +The logger and config (and to a lesser degree, the script and action) objects are all available to each function called for convenience and consistency. + + +####### +Actions +####### + +Each action can be enabled or disabled via commandline options (see :ref:`Enabling-and-Disabling-Actions`). By default they look for a function with the same name as the action name, with ``-`` replaced by ``_``. However, any function or method may be specified as the Action.function_. + +When run, the Action_ calls the Action.function_ with a Context_. The function should raise ScriptHarnessError_ on error, or ScriptHarnessFatal_ on fatal error. + +Afterwards, the Action.history_ contains the ``return_value``, status_, ``start_time``, and ``end_time``. + + +.. _ALL_PHASES: scriptharness.script.html#scriptharness.script.ALL_PHASES +.. _LISTENER_PHASES: scriptharness.script.html#scriptharness.script.LISTENER_PHASES +.. _Action: scriptharness.actions.html#scriptharness.actions.Action +.. _Action.function: scriptharness.actions.html#scriptharness.actions.Action.function +.. _Action.history: scriptharness.actions.html#scriptharness.actions.Action.history +.. _Context: scriptharness.script.html#scriptharness.script.Context +.. _LoggingDict: scriptharness.structures.html#scriptharness.structures.LoggingDict +.. _Script: scriptharness.script.html#scriptharness.script.Script +.. _Script.run(): scriptharness.script.html#scriptharness.script.Script.run +.. _ScriptHarnessError: scriptharness.exceptions.html#scriptharness.exceptions.ScriptHarnessError +.. _ScriptHarnessFatal: scriptharness.exceptions.html#scriptharness.exceptions.ScriptHarnessFatal +.. _running multiple Scripts in parallel: https://github.com/scriptharness/python-scriptharness/issues/12 +.. _status: scriptharness.status.html diff --git a/scriptharness/script.py b/scriptharness/script.py index 67aa7da..b89083d 100644 --- a/scriptharness/script.py +++ b/scriptharness/script.py @@ -381,6 +381,7 @@ def run(self): self.end_message() +# StrictScript {{{1 class StrictScript(Script): """A subclass of Script that uses a ReadOnlyDict for config, and locks its attributes.