Various Python utility functions
pip install bx_py_utils
Here only a simple list about existing utilities. Please take a look into the sources and tests for deeper informations.
anonymize()
- Anonymize the given string with special handling for eMail addresses.
FnmatchExclude()
- Helper for auto docexclude_func
that exclude files via fnmatch pattern.assert_readme()
- Check and update README file with generate_modules_doc()assert_readme_block()
- Check and update README file: Asset that "text_block" is present between the markers.generate_modules_doc()
- Generate a list of function/class information via pdoc.get_code_location()
- Return start and end line number for an object via inspect.
ClientSideCertManager()
- Helper to manage client-side TLS certificate via AWS Secrets Manager by
SecretsManager()
- Access AWS Secrets Manager values
removeprefix()
- Backport ofremoveprefix
from PEP-616 (Python 3.9+)removesuffix()
- Backport ofremovesuffix
from PEP-616 (Python 3.9+)
dict_get()
- nested dictget()
pluck()
- Extract values from a dict, if they are present
Doc-Write, see: https://github.com/boxine/bx_py_utils/blob/master/bx_py_utils/doc_write/README.md
OverrideEnviron()
- Context manager to change 'os.environ' temporarily.cgroup_memory_usage()
- Returns the memory usage of the cgroup the Python interpreter is running in.
print_exc_plus()
- Print traceback information with a listing of all the local variables in each frame.
EmptyFileError()
- Will be raised from get_and_assert_file_size() if a 0-bytes file was found.FileError()
- Base error class for all 'file_utils' exceptions.FileHasher()
- Context Manager for generate different hashes from file content while processing a file.FileSizeError()
- File size is not the same as the expected size.NamedTemporaryFile2()
- Generates a temp file with the given filename without any random name sequence.OverlongFilenameError()
- cut_filename() error: The file name can not be shortened, because sterm is to short.TempFileHasher()
- File like context manager that combines NamedTemporaryFile2 and FileHasher.cut_filename()
- Short the file name (and keep the last suffix). Raise OverlongFilenameError if it can't fit.get_and_assert_file_size()
- Check file size of given file object. Raise EmptyFileError for empty files or return sizesafe_filename()
- Makes an arbitrary input suitable to be used as a filename.
filename_matcher()
- Enhance fnmatch that accept a list of patterns.
introspection_query()
- Generate GraphQL introspection query with variable nested depth.
url_safe_encode()
- Encode bytes into a URL safe string.url_safe_hash()
- Generate a URL safe hash withmax_size
from given string/bytes.
ElementsNotFoundError()
- Happens if requested HTML elements cannot be foundInvalidHtml()
- XMLSyntaxError with better error messages: used in validate_html()get_html_elements()
- Returns the selected HTML elements as stringpretty_format_html()
- Pretty format given HTML document via BeautifulSoup (Needs 'beautifulsoup4' package)validate_html()
- Validate a HTML document via XMLParser (Needs 'lxml' package)
pformat()
- Format given object: Try JSON fist and fallback to pformat()
human_timedelta()
- Converts a time duration into a friendly text representation.
chunk_iterable()
- Returns a generator that yields slices of iterable of the givenchunk_size
.
ChangeCurrentWorkDir()
- Context Manager change the "CWD" to an other directory.MockCurrentWorkDir()
- Context Manager to move the "CWD" to a temp directory.assert_is_dir()
- Check if given path is a directoryassert_is_file()
- Check if given path is a file
processify()
- Decorator to run a function as a process.
get_pyproject_config()
- Get a config section from "pyproject.toml". The path can be optional specify.
rison_dumps()
- Encode as RISON, a URL-safe encoding format.
FrameNotFound()
- Base class for lookup errors.last_frame_outside_path()
- Returns the stack frame that is the direct successor of given "file_path".
compare_sentences()
- Calculates the Levenshtein distance between text1 and text2. With filter functionality.ensure_lf()
- Replace line endings to unix-style.get_words()
- Extract words from a text. With filter functionality.levenshtein_distance()
- Calculates the Levenshtein distance between two strings.startswith_prefixes()
- >>> startswith_prefixes('foobar', prefixes=('foo','bar'))uuid_from_text()
- Generate a UUID instance from the given text in a determinism may via SHA224 hash.
assert_equal()
- Check if the two objects are the same. Display a nice diff, usingpformat()
assert_text_equal()
- Check if the two text strings are the same. Display an error message with a diff.pformat_ndiff()
- Generate andiff
from two objects, usingpformat()
pformat_unified_diff()
- Generate a unified diff from two objects, usingpformat()
text_ndiff()
- Generate andiff
between two text strings.text_unified_diff()
- Generate a unified diff between two text strings.
MassContextManager()
- A context manager that enter/exit a list of mocks.
parse_dt()
- Helper for easy generate adatetime
instance via string.
DenyAnyRealRequestContextManager()
- Context manager that denies any request via docket/urllib3. Will raise DenyCallError.deny_any_real_request()
- Deny any request via docket/urllib3. Useful for tests, because they should mock all requests.
FileWatcher()
- Helper to record which new files have been created.
NoLogs()
- Context manager to Suppress all logger outputsRaiseLogUsage()
- A log handler, that raise an error on every log output.
SecretsManagerMock()
- Mock forbx_py_utils.aws.secret_manager.SecretsManager()
MockedBoto3Session()
- Mock forboto3.session.Session()
MockUUIDGenerator()
- Helper to mockuuid.uuid4()
with reproducible results (e.g. for snapshot tests)
A simple mock for Boto3's S3 modules.
PseudoS3Client()
- Simulates a boto3 S3 client object in tests
RedirectOut()
- Redirect stdout + stderr into a buffer (with optional strip the output)
assert_json_requests_mock()
- Check the requests mock history. In this case all requests must be JSON.assert_json_requests_mock_snapshot()
- Check requests mock history via snapshot. Accepts only JSON requests.assert_requests_mock()
- Check the requests mock history. Accept mixed "text" and "JSON".assert_requests_mock_snapshot()
- Check requests mock history via snapshot. Accept mixed "text" and "JSON".
Assert complex output via auto updated snapshot files with nice diff error messages.
SnapshotChanged()
- Assertion failed.assert_binary_snapshot()
- Assert binary data via snapshot fileassert_html_snapshot()
- Assert "html" string via snapshot file with validate and pretty formatassert_py_snapshot()
- Assert complex python objects vio PrettyPrinter() snapshot file.assert_snapshot()
- Assert given data serialized to JSON snapshot file.assert_text_snapshot()
- Assert "text" string via snapshot fileget_snapshot_file()
- Generate a file path use stack information to fill not provided path components.
MockTimeMonotonicGenerator()
- Helper to mocktime.monotonic()
in tests.
BaseDocTests()
- Helper to include all doctests in unittests, without change unittest setup. Just add a normal TestCase.assert_no_flat_tests_functions()
- Check if there exists normal test functions (That will not be executed by normal unittests)
cutout()
- Mark a point in a long text by line no + column with context lines around.
Quick hint about snapshot. If you have many snapshots in your project and you need to change many with a code change, then you can run the tests without a snapshot change leading to an error, by set RAISE_SNAPSHOT_ERRORS=0
in your environment.
e.g.:
RAISE_SNAPSHOT_ERRORS=0 python3 -m unittest
Renew all snapshot files with:
make update-test-snapshot-files
We split bx_py_utils
and moved all Django related utilities into the separated project:
So, bx_py_utils
is better usable in non-Django projects, because Django will not installed as decency of "bx_py_utils"
To start developing e.g.:
~$ git clone https://github.com/boxine/bx_py_utils.git
~$ cd bx_py_utils
~/bx_py_utils$ make
install-poetry install or update poetry
install install project via poetry
update update the sources and installation and generate "conf/requirements.txt"
lint Run code formatters and linter
fix-code-style Fix code formatting
tox-listenvs List all tox test environments
tox Run tests via tox with all environments
test Run tests
coverage Run tests with coverage
update-test-snapshot-files Update all snapshot files (by remove and recreate all snapshot files)
mypy Run mypy
safety Run https://github.com/pyupio/safety
publish Release new version to PyPi
clean Remove created files from the test project
MIT. Patches welcome!
We’ve been rethinking the listening experience for kids and have created an ecosystem where haptic and listening experience are combined via smart technology - the Toniebox.
We are constantly looking for engineers to join our team in different areas. If you’d be interested in contributing to our platform, have a look at: https://tonies.com/jobs/