[dg] Warn when package defines entry point but its missing from manifest (BUILD-1030) #29514
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
smackesey
force-pushed
the
sean/dg/plugin-refactor
branch
from
b448f3f to
8c2fab3
Compare
smackesey
force-pushed
the
sean/dg/warn-on-missing-plugin-in-manifest
branch
2 times, most recently
from
928ae24 to
f2b4fe1
Compare
smackesey
force-pushed
the
sean/dg/plugin-refactor
branch
from
5128ede to
485bcfb
Compare
smackesey
force-pushed
the
sean/dg/warn-on-missing-plugin-in-manifest
branch
from
f2b4fe1 to
5238ead
Compare
This was referenced Apr 23, 2025
smackesey
force-pushed
the
sean/dg/warn-on-missing-plugin-in-manifest
branch
from
5238ead to
fc12ece
Compare
smackesey
force-pushed
the
sean/dg/plugin-refactor
branch
from
485bcfb to
65056ab
Compare
smackesey
force-pushed
the
sean/dg/warn-on-missing-plugin-in-manifest
branch
from
fc12ece to
877f7da
Compare
smackesey
changed the base branch from
sean/dg/plugin-refactor
to
graphite-base/29514
gibsondan
approved these changes
Apr 23, 2025
Comment on lines
108
to
116
| def install_editable_dagster_packages_to_venv( | ||
| venv_path: Path, package_rel_paths: Sequence[str] | ||
| ) -> None: | ||
| dagster_git_repo_dir = str(discover_git_root(Path(__file__))) | ||
| install_args: list[str] = [] | ||
| for path in package_rel_paths: | ||
| full_path = Path(dagster_git_repo_dir) / "python_modules" / path | ||
| install_args.extend(["-e", str(full_path)]) | ||
| install_to_venv(venv_path, install_args) |
Member
There was a problem hiding this comment.
should this be in test utils? would you use it outside of a test?
Collaborator
Author
There was a problem hiding this comment.
good call, moved
smackesey
force-pushed
the
sean/dg/warn-on-missing-plugin-in-manifest
branch
from
877f7da to
888d70a
Compare
smackesey
force-pushed
the
graphite-base/29514
branch
from
65056ab to
0e8f88f
Compare
smackesey
changed the base branch from
graphite-base/29514
to
sean/dg/plugin-refactor
smackesey
added a commit
that referenced
this pull request
Apr 23, 2025
## Summary & Motivation - Change `dagster-components list library` -> `list plugins`. This should have been done when the `dg` command was converted. - The output format has been changed. `list libraries` outputs a `list[PluginObjectSnap]`. `list plugins` outputs a `PluginManifest`. The `PluginManifest` contains a `list[PluginObjectSnap]` but also a list of all modules scanned. This means it contains information about which plugins were scanned but contained no objects. This is useful to detect the situation where a project declares an entry point but it isn't registered (see [upstack](#29514)). - `dg` will detect when it's dealing with a version with `list plugins` or `list libraries` and call the appropriate command. It also normalizes the differing output formats to the same `PluginManifest` format on the `dg` end. ## How I Tested These Changes New unit tests, including backcompat tests.
smackesey
force-pushed
the
sean/dg/warn-on-missing-plugin-in-manifest
branch
from
888d70a to
5c199f0
Compare
smackesey
added a commit
that referenced
this pull request
Apr 23, 2025
## Summary & Motivation - Change `dagster-components list library` -> `list plugins`. This should have been done when the `dg` command was converted. - The output format has been changed. `list libraries` outputs a `list[PluginObjectSnap]`. `list plugins` outputs a `PluginManifest`. The `PluginManifest` contains a `list[PluginObjectSnap]` but also a list of all modules scanned. This means it contains information about which plugins were scanned but contained no objects. This is useful to detect the situation where a project declares an entry point but it isn't registered (see [upstack](#29514)). - `dg` will detect when it's dealing with a version with `list plugins` or `list libraries` and call the appropriate command. It also normalizes the differing output formats to the same `PluginManifest` format on the `dg` end. ## How I Tested These Changes New unit tests, including backcompat tests.
smackesey
added a commit
that referenced
this pull request
Apr 23, 2025
…est (BUILD-1030) (#29514) ## Summary & Motivation Emit a warning when a project defines a `dagster_dg.plugin` entry point, but the targeted module is not present in the `PluginManifest` retrieved via `dagster-components list plugins`. This can help users avoid the frustrating situation where they've defined an entry point but their custom components aren't showing up. The warning is: ``` Your package defines a `dagster_dg.plugin` entry point, but this module was not found in the plugin manifest for the current environment. This means either that your project is not installed in the current environment, or that the entry point metadata was added after your module was installed. Python entry points are registered at package install time. Please reinstall your package into the current environment to ensure the entry point is registered. Entry point module: `foo_bar.lib` ``` ## How I Tested These Changes New unit tests ## Changelog `dg` will now emit a warning prompting the user to reinstall the package if it detects an entry point in project metadata that does not show up when running in `dg list plugins`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary & Motivation
Emit a warning when a project defines a
dagster_dg.pluginentry point, but the targeted module is not present in thePluginManifestretrieved viadagster-components list plugins. This can help users avoid the frustrating situation where they've defined an entry point but their custom components aren't showing up. The warning is:How I Tested These Changes
New unit tests
Changelog
dgwill now emit a warning prompting the user to reinstall the package if it detects an entry point in project metadata that does not show up when running indg list plugins.