CHIA-597: Use poetry for package management

[altendky]

Devs should continue to use the install.sh / Install.ps1 scripts for convenience. These scripts do the following:

Download and install poetry into .penv
Setup a chia-blockchain specific .venv
Create a symlink between venv and .venv

You can then use activate in the same way as before

This approach keeps the poetry deployment in .penv separate from the chia-blockchain install in .venv. If you want to run the poetry command you will need to run it from the .penv directory.

You can install poetry globally if you want, but then you may need to pay attention to the versioning in poetry specifically (running a different version from the one installed in install.sh may have unexpected results). We are still exploring options on better ways to handle the poetry install.

Draft for:

• Some TODOs
• Lots of consideration
• Not being a preliminary exploration
• Switch the artifact check to use poetry
• miniupnpc extra
• ipython
• making lgtm ignore .venv
• figure out m1 trying to build chiavdf from source https://github.com/Chia-Network/chia-blockchain/runs/8059378328?check_suite_focus=true
• handle automatic versioning
• catch up pre-version checking
• review details of automatic versioning

[github-actions]

File	Coverage	Missing Lines
chia/_tests/core/data_layer/util.py	62.5%	lines 143, 160-161

Total	Missing	Coverage
9 lines	3 lines	66%

[emlowe]

@SocketSecurity ignore pypi/dill@0.3.7
@SocketSecurity ignore pypi/clvm-rs@0.2.5

[github-actions]

File	Coverage	Missing Lines
chia/_tests/core/data_layer/util.py	62.5%	lines 143, 160-161

Total	Missing	Coverage
9 lines	3 lines	66%

[arvidn]

when testing this on Mac (M1) I get this warning:

Warning: The file chosen for install of twine 5.1.0 (twine-5.1.0-py3-none-any.whl) is yanked. Reason for being yanked: https://github.com/pypa/twine/issues/1125

I suppose this is related to #18253

[arvidn]

looks good. It would be nice to have some documented guidance on how treat the lockfile, and PRs making changes to it. I assume it would be accepted with caution, as we want to lock down our supply-chain risk.

[Starttoaster]

Waiting on coverage-diff override

[emlowe]

Sorry Clicked the wrong thing and ended up closing and reopening the PR

[emlowe]

looks good. It would be nice to have some documented guidance on how treat the lockfile, and PRs making changes to it. I assume it would be accepted with caution, as we want to lock down our supply-chain risk.

Yes, generally you will want to use poetry to make any changes, eg:

.penv/bin/poetry update urllib3

However, I'm also hopeful that now all the dependencies are listed in the lock file, we can relay on dependabot creating PRs to update anything

[emlowe]

Well, dependabot seems to have a flow in that it updates the lock files but not the toml (dependabot/dependabot-core#8603)

So that might take some manual effort per the reporter until that gets resolved....

[github-actions]

File	Coverage	Missing Lines
chia/_tests/core/data_layer/util.py	62.5%	lines 143, 160-161

Total	Missing	Coverage
9 lines	3 lines	66%

[coveralls-official]

Pull Request Test Coverage Report for Build 9860288518

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 6 of 9 (66.67%) changed or added relevant lines in 2 files are covered.
• 39 unchanged lines in 8 files lost coverage.
• Overall coverage decreased (-0.03%) to 90.955%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
chia/_tests/core/data_layer/util.py	5	8	62.5%

Files with Coverage Reduction	New Missed Lines	%
chia/wallet/wallet_node.py	1	88.72%
chia/_tests/util/test_build_job_matrix.py	1	96.77%
chia/daemon/client.py	1	73.94%
chia/introducer/introducer.py	1	78.26%
chia/data_layer/data_layer_wallet.py	2	91.93%
chia/server/node_discovery.py	4	79.08%
chia/server/address_manager.py	7	90.48%
chia/_tests/core/util/test_lockfile.py	22	77.73%

Totals
Change from base Build 9846325110:	-0.03%
Covered Lines:	101594
Relevant Lines:	111650

---

💛 - Coveralls

[github-actions]

File	Coverage	Missing Lines
chia/_tests/core/data_layer/util.py	62.5%	lines 143, 160-161

Total	Missing	Coverage
9 lines	3 lines	66%

[github-actions]

File	Coverage	Missing Lines
chia/_tests/core/data_layer/util.py	62.5%	lines 143, 160-161

Total	Missing	Coverage
9 lines	3 lines	66%