upgrade yaml lib dep

overview

No functional change.

This change forces the ruamel yaml library dependency to upgrade - as of version 5.9.0 pypyr depends on ruamel.yaml >= 0.17.32.

If you do not have the latest version of ruamel and you use !jsonify, you will see:

AttributeError: 'RoundTripConstructor' object has no attribute 'construct_unknown'

what's changed

• fix venv flaky test by @yaythomas in #330
• ruamel version pin by @yaythomas in #331

Full Changelog: v5.9.0...v5.9.1

string loader

summary

• Add new pypyr.loaders.string loader that loads pipelines directly from strings.

from pypyr import pipelinerunner

pipeline = """\
steps:
- name: pypyr.steps.set
  in:
    set:
      test: 1
"""

context = pipelinerunner.run(pipeline_name=pipeline,
                             loader="pypyr.loaders.string")

assert context["test"] == 1

• Fix bug with !jsonify custom tags failing to load with error "ruamel.yaml.constructor.ConstructorError: could not determine a constructor for the tag '!jsonify'".
  • this was due to a regression in the ruamel dependency

what's changed

• fix jsonify tag ruamel by @yaythomas in #328
• String Loader by @lucasrcezimbra in #326
• Bump version: 5.8.0 → 5.9.0 by @yaythomas in #329

new contributors

• @lucasrcezimbra made their first contribution in #326

Full Changelog: v5.8.0...v5.9.0

no_cache & clear_all

summary

• convenience function to clear all caches in one call. Closes #316.

import pypyr.cache.admin as cache_admin

cache_admin.clear_all()

• disable caching entirely with new no_cache mode. Closes #317

from pypyr.config import config
from pypyr import pipelinerunner

# disable all caching
config.no_cache = True

# This will NOT save `my-pipe` to cache once its loaded.
context = pipelinerunner.run(pipeline_name='my-pipe')

What's Changed

• clear all cache, codecov token, tox 4 by @yaythomas in #318
• no cache mode by @yaythomas in #319
• Bump version: 5.7.1 → 5.8.0 by @yaythomas in #320

Full Changelog: v5.7.1...v5.8.0

python 3.11 maintenance release

summary

• maintenance release, no new features
• bug fix for pypyr.steps.filereplace to honor flat format :ff directive
• python 3.11 compatibility confirmed
  • on python 3.11 runtimes, the tomli dependency won't install separately anymore since it's part of stdlib now

what's changed

• pypyr.steps.filereplace get_formatted X2 by @yaythomas in #304
• new codecov uploader by @yaythomas in #305
• py 3.11 & conditional tomli dep by @yaythomas in #306
• Bump version: 5.7.0 → 5.7.1 by @yaythomas in #307

Full Changelog: v5.7.0...v5.7.1

switch & argskwargs

summary

• New switch step for IF-ELSE style branching in your pipelines!
• New argskwargs parser to combine plain args and key-value pairs (key=value) from the cli
• keyvaluepairs and dict parsers now support having = in the value, so you can have key=one+one=two parse to {'key': 'one+one=two'}
  • these parsers now also support passing args with no =, in which case arg1 becomes {'arg1': ''}

what's changed

• switch by @yaythomas in #299
• typing on dsl.cof by @yaythomas in #300
• gh action checkout v3 by @yaythomas in #301
• argskwargs parser & partitioning on = for kvp by @yaythomas in #302

Full Changelog: v5.6.0...v5.7.0

venv create & new flit build internals

summary

• you can now use a custom error message when pypyr.steps.assert raises an exception.
• new pypyr.steps.venv step to create venvs in parallel from yaml or toml config.
• new venv-create pipeline so you can provision venvs concurrently from declarative config without having to write your own pipeline or script.
• pypyr is now built & packaged by the excellent PEP517 compliant flit.

what's changed

• flake8 new version line length by @yaythomas in #286
• from setup.py to flit by @yaythomas in #290
• remove redundant dict on parser context init by @yaythomas in #292
• venv creator & log to stdout by @yaythomas in #296
• ops/build uses venv for publish verify by @yaythomas in #297

Full Changelog: v5.5.0...v5.6.0

concurrent cmds & cmdOut dot notation

summary

• cmdOut for pypyr.steps.cmd & pypyr.steps.shell gets dotted attribute access.
  • instead of cmdOut[returncode] access value like this: cmdOut.returncode
  • backwards compatible - the old-style dict-like accessors will still work.
  • closes #272
• Introduce pypyr.steps.cmds & pypyr.steps.shells to run programs or shell statements asynchronously as parallel (concurrent) sub-processes.
  • cmdOut for these works as with the serial versions of the steps using the new dotted attribute access introduced in this release.
  • closes #273
• Set default encoding for all the cmd/shell steps using default_cmd_encoding in config. You can use the $env variable PYPYR_CMD_ENCODING to initialize this value, in addition to the usual config files.
  • closes #275

what's changed

• async cmds + shells by @yaythomas in #274
• release 5.5 & default_cmd_encoding for serial by @yaythomas in #276

Full Changelog: v5.4.0...v5.5.0

run multiple commands & shell statements in same step

Summary

• pypyr.steps.cmd & pypyr.steps.shell now also takes a list input to run multiple commands/shell statements in the same step!

- name: pypyr.steps.cmd
  in:
    cmd:
      - echo 1
      - echo 2

- name: pypyr.steps.cmd
  in:
    cmd:
      run:
        - echo 3
        - echo 4
      save: False
      cwd: mydir/subdir

• Both the cmd and shell steps expanded to allow:
  • decode output in different encodings
  • save output as raw bytes or as encoded text.
  • File output for stdout/stderr.
    • File create mode of append or overwrite.
  • stderr can redirect to /dev/stdout
  • Both stdout & stderr can redirect to /dev/null

#  when save: True
- name: pypyr.steps.cmd
  comment: when save is True
  in:
    cmd:
      run: curl https://myurl.blah/diblah
      save: True
      cwd: .
      bytes: False
      encoding: utf-8

#  when save: False (this is default)
- name: pypyr.steps.cmd
  comment: when save is False (the default when `save` not set)
  in:
    cmd:
      run: curl --cert certfile --key keyfile https://myurl.blah/diblah
      cwd: ..
      stdout: ./path/out.txt
      stderr: ./path/err.txt
      append: False

What's Changed

• cmd & shell list input by @yaythomas in #271

Full Changelog: v5.3.0...v5.4.0

shortcuts

Summary

Create shortcuts to your pypyr run commands & their input args. This is handy for creating short & sweet aliases for longer pipeline run commands.

So if you have a pipeline you normally run like this:

$ pypyr arb/my-pipeline arg1=1234 arg2="/path/to long/annoying path to type/x" arg3="arb'hello"

You can create a shortcut alias for this like so:

shortcuts:
  my-shortcut:
    pipeline: arb/my-pipeline
    args:
      arg1: 1234
      arg2: /path/to long/annoying path to type/x
      arg3: "arb'hello"

And now for the same pipeline + inputs you can just run:

$ pypyr my-shortcut

You can create your shortcut definition in pyproject.toml or in the pypyr yaml config file.

You can make your shortcuts project specific, or for the entire user profile, or globally for your entire system.

What's Changed

• Shortcuts by @yaythomas in #268
• Bump version: 5.2.0 → 5.3.0 by @yaythomas in #269

Full Changelog: v5.2.0...v5.3.0

encoding & config

Summary

• pypyr is now configurable with yaml or pyproject.toml!
• Explicitly over-ride encoding on any filesystem steps

What's Changed

• friendly err on malformed pipeline. by @yaythomas in #250
• config & encoding by @yaythomas in #255
• file encoding from config by @yaythomas in #259
• windows tests & config platform indicators by @yaythomas in #261
• typings for config, platform by @yaythomas in #262
• FakePath PathLike & parent also fake by @yaythomas in #264
• pypyr.steps.python & version bump 5.2.0 by @yaythomas in #266

Full Changelog: v5.1.0...v5.2.0