Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CHIA-598] Create check for a virtual project structure #17810

Open
wants to merge 50 commits into
base: main
Choose a base branch
from

Conversation

Quexington
Copy link
Contributor

@Quexington Quexington commented Apr 2, 2024

The idea behind this PR is to create a stepping stone into a piecemeal solution of sorting out the complicated interdependencies of the monorepo. The change is simple:

  1. Add an annotation to each file describing the wider "package" it is a part of
  2. Add a check that every python file has an annotation
  3. Add a check that no virtual "packages" have cyclical dependencies
  4. Use a configuration file to ignore most of these cycles

The annotations in this PR are not intended to be permanent or indicative of some firmer opinion. They are simply a set of opinions that can be changed arbitrarily at a later date. The annotation format can also be migrated relatively simply if we decide the existing format is not great. The idea is that these changes can be made by smaller groups with better focus over time rather than needing to have a big, likely highly controversial migration.

Ideally, one day there will be no dependency cycles ignored and we can draw a definitive dependency graph and at that point potentially start breaking bits out into their own repos, solidifying APIs, and publishing the constituent packages with independent stable versions.

As of 4/24/24 here's the existing virtual dependency graph:

{
    "chia-blockchain": [
        "development",
        "harvester",
        "supported_wallets",
        "cmds",
        "key_management",
        "streamable",
        "timelord",
        "cryptography",
        "todo",
        "full_node",
        "plotting",
        "wallet",
        "protocol",
        "service",
        "data_layer",
        "seeder",
        "farmer",
        "simulator",
        "consensus",
        "database",
        "introducer",
        "chips",
        "concurrency",
        "rpc"
    ],
    "obsolete": [
        "protocol",
        "consensus",
        "clvm",
        "supported_wallets",
        "mempool",
        "streamable",
        "cryptography",
        "chialisp"
    ],
    "development": [
        "protocol",
        "consensus",
        "clvm",
        "database",
        "mempool",
        "chia-blockchain",
        "streamable",
        "wallet",
        "obsolete",
        "cryptography",
        "todo",
        "simulator",
        "full_node"
    ],
    "cmds": [
        "protocol",
        "consensus",
        "harvester",
        "service",
        "supported_wallets",
        "database",
        "mempool",
        "chia-blockchain",
        "data_layer",
        "key_management",
        "wallet",
        "streamable",
        "simulator",
        "farmer",
        "rpc",
        "full_node"
    ],
    "wallet": [
        "development",
        "supported_wallets",
        "chia-blockchain",
        "cmds",
        "streamable",
        "key_management",
        "obsolete",
        "cryptography",
        "todo",
        "network",
        "full_node",
        "pooling",
        "protocol",
        "service",
        "clvm",
        "data_layer",
        "chialisp_puzzles",
        "consensus",
        "database",
        "mempool",
        "chips",
        "chialisp",
        "rpc"
    ],
    "supported_wallets": [
        "protocol",
        "consensus",
        "service",
        "clvm",
        "database",
        "mempool",
        "chia-blockchain",
        "cmds",
        "chialisp_puzzles",
        "wallet",
        "key_management",
        "streamable",
        "chips",
        "obsolete",
        "cryptography",
        "todo",
        "chialisp",
        "full_node"
    ],
    "data_layer": [
        "protocol",
        "service",
        "consensus",
        "clvm",
        "supported_wallets",
        "database",
        "mempool",
        "chia-blockchain",
        "cmds",
        "chialisp_puzzles",
        "wallet",
        "streamable",
        "full_node",
        "concurrency",
        "cryptography",
        "todo",
        "chialisp",
        "rpc"
    ],
    "farmer": [
        "protocol",
        "consensus",
        "service",
        "harvester",
        "pooling",
        "chia-blockchain",
        "cmds",
        "chialisp_puzzles",
        "wallet",
        "key_management",
        "plot_sync",
        "streamable",
        "cryptography",
        "todo",
        "rpc",
        "full_node"
    ],
    "key_management": [
        "protocol",
        "service",
        "chia-blockchain",
        "cmds",
        "streamable"
    ],
    "full_node": [
        "protocol",
        "consensus",
        "service",
        "development",
        "clvm",
        "database",
        "mempool",
        "chia-blockchain",
        "cmds",
        "streamable",
        "obsolete",
        "concurrency",
        "cryptography",
        "todo",
        "chialisp",
        "network",
        "rpc"
    ],
    "service": [
        "protocol",
        "consensus",
        "plotting",
        "chia-blockchain",
        "cmds",
        "streamable",
        "key_management",
        "obsolete",
        "concurrency",
        "cryptography",
        "todo",
        "network",
        "rpc"
    ],
    "pooling": [
        "protocol",
        "service",
        "consensus",
        "supported_wallets",
        "clvm",
        "database",
        "mempool",
        "chia-blockchain",
        "cmds",
        "chialisp_puzzles",
        "wallet",
        "streamable",
        "obsolete",
        "chialisp",
        "cryptography",
        "farmer",
        "full_node"
    ],
    "plotting": [
        "consensus",
        "chia-blockchain",
        "streamable",
        "key_management",
        "cryptography",
        "todo"
    ],
    "rpc": [
        "service",
        "consensus",
        "mempool",
        "chia-blockchain",
        "streamable",
        "wallet",
        "obsolete"
    ],
    "simulator": [
        "development",
        "harvester",
        "supported_wallets",
        "chia-blockchain",
        "cmds",
        "streamable",
        "key_management",
        "timelord",
        "obsolete",
        "cryptography",
        "todo",
        "full_node",
        "plotting",
        "wallet",
        "protocol",
        "service",
        "clvm",
        "chialisp_puzzles",
        "seeder",
        "farmer",
        "consensus",
        "database",
        "mempool",
        "introducer",
        "concurrency",
        "chialisp",
        "rpc"
    ],
    "consensus": [
        "protocol",
        "service",
        "database",
        "mempool",
        "chialisp_puzzles",
        "streamable",
        "obsolete",
        "concurrency",
        "cryptography",
        "todo",
        "full_node"
    ],
    "mempool": [
        "protocol",
        "consensus",
        "service",
        "development",
        "clvm",
        "database",
        "streamable",
        "obsolete",
        "concurrency",
        "todo",
        "chialisp",
        "full_node"
    ],
    "harvester": [
        "protocol",
        "consensus",
        "service",
        "plotting",
        "chia-blockchain",
        "plot_sync",
        "streamable",
        "cryptography",
        "todo",
        "rpc"
    ],
    "introducer": [
        "protocol",
        "service",
        "chia-blockchain",
        "streamable",
        "todo",
        "network",
        "rpc"
    ],
    "plot_sync": [
        "protocol",
        "service",
        "consensus",
        "plotting",
        "streamable",
        "todo"
    ],
    "protocol": [
        "service",
        "consensus",
        "mempool",
        "streamable",
        "full_node"
    ],
    "seeder": [
        "protocol",
        "service",
        "consensus",
        "chia-blockchain",
        "streamable",
        "todo",
        "rpc",
        "full_node"
    ],
    "timelord": [
        "protocol",
        "service",
        "consensus",
        "chia-blockchain",
        "streamable",
        "todo",
        "rpc"
    ],
    "network": [
        "protocol",
        "service",
        "streamable",
        "obsolete",
        "cryptography"
    ],
    "clvm": [
        "streamable",
        "cryptography"
    ],
    "chips": [],
    "streamable": [
        "cryptography"
    ],
    "concurrency": [
        "service"
    ],
    "cryptography": [
        "consensus",
        "streamable"
    ],
    "database": [],
    "todo": [
        "protocol",
        "streamable"
    ],
    "testing": [],
    "chialisp_puzzles": [
        "consensus",
        "clvm",
        "streamable",
        "wallet",
        "cryptography",
        "chialisp",
        "full_node"
    ],
    "chialisp": [
        "consensus",
        "clvm",
        "concurrency"
    ]
}

@Quexington Quexington added the Added Required label for PR that categorizes merge commit message as "Added" for changelog label Apr 2, 2024
@Chia-Network Chia-Network deleted a comment from github-actions bot Apr 8, 2024
@github-actions github-actions bot added the merge_conflict Branch has conflicts that prevent merge to main label Apr 9, 2024
@Chia-Network Chia-Network deleted a comment from github-actions bot Apr 9, 2024
@Chia-Network Chia-Network deleted a comment from github-actions bot May 8, 2024
@Chia-Network Chia-Network deleted a comment from github-actions bot May 8, 2024
@github-actions github-actions bot removed the merge_conflict Branch has conflicts that prevent merge to main label May 8, 2024
@Quexington Quexington changed the title [Experiment] Create check for a virtual project structure Create check for a virtual project structure May 8, 2024
@Chia-Network Chia-Network deleted a comment from github-actions bot May 8, 2024
@Quexington Quexington changed the title Create check for a virtual project structure [CHIA-598] Create check for a virtual project structure May 22, 2024
@github-actions github-actions bot added merge_conflict Branch has conflicts that prevent merge to main labels Jun 8, 2024
@Chia-Network Chia-Network deleted a comment from github-actions bot Jun 10, 2024
@Chia-Network Chia-Network deleted a comment from github-actions bot Jun 10, 2024
@github-actions github-actions bot removed the merge_conflict Branch has conflicts that prevent merge to main label Jun 11, 2024
@Chia-Network Chia-Network deleted a comment from github-actions bot Jun 11, 2024
@Chia-Network Chia-Network deleted a comment from github-actions bot Jun 12, 2024
Copy link
Contributor

File Coverage Missing Lines
chia/util/virtual_project_analysis.py 98.2% lines 89, 146, 190, 196, 524
Total Missing Coverage
505 lines 5 lines 99%

@github-actions github-actions bot added the merge_conflict Branch has conflicts that prevent merge to main label Jun 17, 2024
@github-actions github-actions bot removed the merge_conflict Branch has conflicts that prevent merge to main label Jun 18, 2024
@Chia-Network Chia-Network deleted a comment from github-actions bot Jun 18, 2024
@Chia-Network Chia-Network deleted a comment from github-actions bot Jun 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Added Required label for PR that categorizes merge commit message as "Added" for changelog coverage-diff
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants