Scrape various sources using a spreadsheet as an input list of dependencies.
Determine if a dependency is maintained.
(NOT FOR PRODUCTION USE, NOT MAINTAINED)
Developing this project, I hope to learn more about some of these:
- VSCode Profiles
- VSCode Extensions
- Copilot Features
- copilot-debug
- python dependencies and their maintainers
- pytest plugins and their popularity and use
This project will be generated and tailored using Copilot Edits and other AI tools.
I'm using Windows locally so this could be Windows-centric.
Windows Python developers have a challenge for some Python packages, since some are developed with a Mac/Linux-centric focus.
The user elevation system in Windows causes some issues with testing public packages and using temporary folder fixtures.
There may be issues with virtualenv, uv, and Copilot behind firewalls and security software.
- Don't use virtualenv, use uv instead
- uv python install might be blocked,
--no-python-downloadsorUV_PYTHON_DOWNLOADSmight help. - There are some experimental settings with Copilot which can be set to avoid certificate issues.
The default Python profile does not contain some extensions I use:
Some extensions I may use:
Copilot (and anything from Microsoft or Github)
Markdownlint: igorshubovych/markdownlint-cli#457 (comment)
code --list-extensions --profile PythonAIcharliermarsh.ruff
donjayamanne.python-environment-manager
github.copilot
github.copilot-chat
github.vscode-pull-request-github
ms-azuretools.vscode-docker
ms-python.debugpy
ms-python.python
ms-python.vscode-pylance
ms-toolsai.jupyter
ms-toolsai.jupyter-keymap
ms-toolsai.jupyter-renderers
ms-toolsai.vscode-jupyter-cell-tags
ms-toolsai.vscode-jupyter-slideshow
ms-vscode-remote.remote-containers
ms-vscode-remote.remote-ssh
ms-vscode-remote.remote-ssh-edit
ms-vscode-remote.remote-wsl
ms-vscode-remote.vscode-remote-extensionpack
ms-vscode.remote-explorer
ms-vscode.remote-server
njpwerner.autodocstring
tamasfe.even-better-toml
To me, uv is now the 2024 standard for packaging projects. Although it gets updated too frequently!
UV requires some flags for building.
https://packaging.python.org/en/latest/guides/writing-pyproject-toml/
https://docs.astral.sh/uv/guides/install-python/#getting-started
py --list-paths
When searching for a Python version, the following locations are checked:
Managed Python installations in the UV_PYTHON_INSTALL_DIR.
A Python interpreter on the PATH as python, python3, or python3.x on macOS and Linux, or python.exe on Windows.
On Windows, the Python interpreters in the Windows registry and Microsoft Store Python interpreters (see py --list-paths) that match the requested version.
https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html
python.exe -m pipuv pip installuv python install 3.11 3.12 3.13https://github.com/willkg/dotfiles/blob/main/dotfiles/bin/uv-python-symlink
Generate a .python-version file.
uv python pin 3.13pip install --upgrade --user uv
uv venv
.venv/Scripts/Activateuv pip install -r pyproject.tomluv pip install -r pyproject.toml --all-extrasruff format .
Future use, toml formatter:
https://github.com/tox-dev/toml-fmt/tree/main/pyproject-fmt
https://pyproject-fmt.readthedocs.io/en/latest/index.html
I encountered a bug similar to this with ruff.
https://github.com/astral-sh/ruff-vscode
ruff check .
https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff
https://github.com/DavidAnson/markdownlint#optionsconfig
Eventually uv may come out with a replacement task runner.
Generally I prefer using invoke due to its ability to run non-python code.
For this project will test with nox.
https://nox.thea.codes/en/stable/cookbook.html
https://github.com/tox-dev/tox-uv
https://bluesock.org/~willkg/blog/dev/switch_pyenv_to_uv.html
uv tool install PACKAGE
uv tool install --with tox-uv toxhttps://pythonbasics.org/pandas-web-scraping/
https://github.com/scrapy/scrapy
https://github.com/scrapy-plugins/scrapy-playwright
https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation
- using main convention to allow python -m isitmaintained to run. Need to check updated pattern for main convention.
- various plugins for pytest
- various plugins for opentelemetry
- pandas pivottablejs, qgrid
- Jupyter debugging in vscode
- copilot-debug in terminal and vscode
- Additional copilot extensions and experimental features
https://github.com/pandas-dev/pandas/blob/main/pandas/plotting/__init__.py
https://gist.github.com/ahgraber/9ad4d0086a3f239f7872b7f33ebbe4c5