Complete support for variants #4020

Open
stuhood opened this Issue Nov 2, 2016 · 2 comments

Comments

Projects
None yet
1 participant
@stuhood
Member

stuhood commented Nov 2, 2016

The native engine currently has a few stubbings that prevent Variants from working (marked with TODOs mentioning this ticket):

  1. Variant merging is implemented by core::Variants, but not used.
  2. Requesting the Variants product is implemented by nodes::Select, but not merging them (in particular: this will either require lifting the user-defined variants into rust for merging, or pushing down rust's variants to python for merging)
  3. nodes::SelectDependencies does not parse Variants from Address objects (the new path/to:target@key=value syntax) like we used to:
    for dependency in getattr(dep_product, self.field or 'dependencies'):
    variants = self.variants
    if isinstance(dependency, Address):
    # If a subject has literal variants for particular dependencies, they win over all else.
    dependency, literal_variants = parse_variants(dependency)
    variants = Variants.merge(variants, literal_variants)

Additionally, it has recently occurred to me that the RuleGraph gives us the ability to statically determine whether there are any consumers of Variants in a subgraph. This would allow us to eliminate multiple inefficiencies from the original implementation:

  1. We can know statically whether a Variant will be consumed in a subgraph, and so we can avoid propagating Variants in subgraphs that don't consume them (which avoids duplicating Nodes with unused Variant values).
  2. Rather than eagerly computing "default" Variant values at every Select Node (which was removed as part of this ticket), we can compute them only in subgraphs where they might be consumed.

stuhood added a commit that referenced this issue Nov 7, 2016

First round of native engine feedback
Applies the review feedback from [r/4270](https://rbcommons.com/s/twitter/r/4270/), and fixes one minor issue with the bootstrap script.

- `exit 1` in `build-support/bootstrap_native` if compilation fails. This allows for iterating on e.g. `./pants --enable-v2-engine list ..` until the native code successfully builds.
- Used `git-ls-files` to determine which files to hash.
- Derive/implement `Default` for Key/Value.
- Whitespace in generic parameter lists.
- Allow for visualizing executions to a directory via a `--native-engine-visualize-to=$dir` option.
- Convert EntryId to a sealed type to avoid accidental random values.
- Improve a few comments.
- Suffix python keywords rather than prefixing them.
- Whitespace fixes.
- Silenced known unused code warning for [4020](#4020).
- Attempt to cache `~/.cargo` in travis.
- Don't nuke `$HOME` in hermetic tests.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/173991370

Bugs closed: 4037

Reviewed at https://rbcommons.com/s/twitter/r/4359/

@stuhood stuhood removed the native label Mar 27, 2017

@stuhood

This comment has been minimized.

Show comment
Hide comment
Member

stuhood commented Jan 11, 2018

@stuhood

This comment has been minimized.

Show comment
Hide comment
@stuhood

stuhood May 7, 2018

Member

See #5788, which talks about a challenging/interesting usecase for Variants.

Member

stuhood commented May 7, 2018

See #5788, which talks about a challenging/interesting usecase for Variants.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment