Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose target wildcard matching to Starlark #7763

Open
aherrmann opened this issue Mar 19, 2019 · 3 comments
Open

Expose target wildcard matching to Starlark #7763

aherrmann opened this issue Mar 19, 2019 · 3 comments

Comments

@aherrmann
Copy link
Contributor

@aherrmann aherrmann commented Mar 19, 2019

Description of the feature request:

Please expose Bazel's capability of matching target labels against wildcards to Starlark. Given a list of target patterns and a list of targets, return the list of all targets that match any of the given patterns. Patterns should have the same feature set as patterns allowed on the command line. E.g. specific //package:target, wild-card //package:all, or //package/.... Optionally, it could allow patterns relative to a specific target, e.g. :all matching all targets in the same package. Here's a suggestion of what the API might look like:

def match_targets(patterns, targets, relative_to = None):
    """
    Args:
      patterns: List of patterns.
      targets: List of target labels.
      relative_to: Match relative patterns relative to this target.
    
    Returns:
      List of labels contained in `targets` that match any of the given `patterns`.

    """

Feature requests: what underlying problem are you trying to solve with this feature?

The use-case that prompts this feature request comes from rules_haskell, where we would like to implement a REPL rule based on an aspect with the following interface:

haskell_repl(
    name = "my-repl",
    # Collect all transitive Haskell dependencies from these targets.
    deps = [
        "//package-a:target-1",
        "//package-b:target-2",
    ],
    # Load targets by source that match these patterns.
    from_source = [
        "//package-a/...",
        "//packaga-b/...",
        "//common/...",
    ],
    # Don't load targets by source that match these patterns, instead load their binaries.
    from_binary = [
        "//package-a/vendored/...",
    ],
)

See below for some background and motivation.

Currently, we re-implement a subset of Bazel's target pattern matching in Starlark to enable this interface. This is duplicated effort that should not be necessary. And we would prefer to offer our users Bazel's full pattern capabilities.

Note, that we cannot use genquery in this case, because it does not allow wild-cards, and also does not take configuration into account (query vs. cquery).

Background on Haskell REPL

With Haskell's REPL there is an important distinction between loading a target by source or as binary. If you load by source, this allows you to modify the source file, instruct the REPL to reload sources, and observe your code changes in the REPL right away, allowing for a fast feedback loop. Loading by binary instead means that code changes only become visible if you close the REPL, rebuild the binaries, and reopen the REPL. In practice this is too slow for productive development.

However, some targets should not be loaded from source, but instead as binary. Reasons for this could be that they require compiler flags that are incompatible with other modules, or you're not interested in modifying their source and want the speed benefit of loading them as binary.

What operating system are you running Bazel on?

NixOS

What's the output of bazel info release?

release 0.23.1- (@non-git)

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

Using the Nix package manager.

Have you found anything relevant by searching the web?

Only genquery which is not applicable.

@laurentlb

This comment has been minimized.

Copy link
Member

@laurentlb laurentlb commented Mar 19, 2019

Currently, we re-implement a subset of Bazel's target pattern matching in Starlark to enable this interface. This is duplicated effort that should not be necessary. And we would prefer to offer our users Bazel's full pattern capabilities.

Can you detail which capabilities you'd like?

In the example:

    from_source = [
        "//package-a/...",
    ],

It seems like you only need to check a prefix: str.startswith("//package-a").

@aherrmann

This comment has been minimized.

Copy link
Contributor Author

@aherrmann aherrmann commented Mar 19, 2019

@laurentlb At the moment our own implementation supports absolute labels in the local workspace, e.g.
//some/package:some_target, with the wildcards ..., :all, and :*. Also the //some/package shortcut is implemented. Relative labels are not implemented. Relative labels would be nice. Otherwise, for the time being that seems sufficient.

Note, I'm not asking to add additional pattern matching capabilities to Bazel. Instead, my point is that this kind of pattern matching is something that Bazel already implements internally. It would be better if rules could just use Bazel's builtin pattern matching to make sure that the set of supported patterns is identical to what users expect from their experience with Bazel itself, and also so that the pattern matching stays in sync if it's ever extended in Bazel.

@adam-azarchs

This comment has been minimized.

Copy link

@adam-azarchs adam-azarchs commented Sep 24, 2019

This would be useful for my use case as well. I want to create a test rule to verify that every file in some set of files in the source tree is depended on, at least transitively, by some test rule, in order to enforce that developers don't forget to update BUILD files when appropriate.

I can do this from the command line easily enough with bazel query but executing that within a test running on bazel doesn't work, as the work tree is locked by the parent bazel process. genquery as currently implemented doesn't work either, because the inherent nature of the test means we can't know the query scope in advance. Asking developers to add test_suite targets to every new package and then add those to the parent packages going up the tree would be a prohibitive burden for our organization. With the current state of affairs our choices seem to be either that or moving away from the goal of being able to say "if bazel test //... passes then the CI should pass too."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.