Below is a list of tooling that the engineers have noted or used on their projects, that might prove useful in other projects or want to be reviewed.
ruff: linter for Python Reviews: "Very fast and seems to capture most linting issues" - Tom F
black: Formatter for Python Reviews: "Used for a number of years and seems to capture most formatting issues" - Tom F
Pydantic: data validation library for Python. Reviews:
Pandera: Data validation library for dataframes (Pydantic intergration available) Reviews:
Prefect: Data workflow tool Reviews:
Great Expectations: Data quality tool for pipelines Reviews:
checkov: Scans cloud infrastructure for misconfigurations Reviews:
OpenTofu: Open source alternative to Terraform Reviews:
Overwatch: Log tracking in Databricks Reviews:
Mindforger: Markdown Editor Reviews:
Obsidian: writing app useful for note taking and documentation Reviews:
Json Crack: open-source data visualization app capable of visualizing data formats such as JSON, YAML, XML, CSV and more, Reviews:
GraphViz: Graphviz is open source graph visualization software. Reviews:
Visidata: VisiData is an interactive multitool for tabular data. Reviews:
PlantUML: Diagram creator tool Reviews:
Quin: PySpark methods Reviews:
Github Copilot: AI programmer Reviews:
Direnv: Manage environment variables in shell Reviews:
Automated-Security-Helper: Security helper tool to analyse code Reviews:
Safety: Safety checks for Python projects Reviews: "Quick and easy to use and identified vunerable packages" - Tom F
Nixos: A package manager & system configuration Reviews:
Poetry: A python virtual environment Reviews: "A good virtual environment manager and easy to export" - Tom F
MlOps tooling: List of MlOps tooling