Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

incompatible_depset_is_not_iterable: Make depset not iterable #5816

Closed
c-parsons opened this issue Aug 8, 2018 · 9 comments

Comments

@c-parsons
Copy link
Contributor

commented Aug 8, 2018

This is a tracking issue for offering a migration solution for
--incompatible_depset_is_not_iterable

This flag would enforce that depset is no longer treatable as an iterable. Thus, len(depset()) would throw an error.
This is useful in it prevents users from using this gravely-inefficient construct (expansion of depsets should be avoidable at all costs, as they are incredibly expensive operations)

@c-parsons c-parsons self-assigned this Aug 8, 2018

@dslomov dslomov changed the title Make depset not iterable incompatible_depset_is_not_iterable: Make depset not iterable Oct 31, 2018

@dslomov

This comment has been minimized.

Copy link
Contributor

commented Oct 31, 2018

What is missing for migration: migration docs, length of migration window. After these are done, please add "migration-ready" label

bttk added a commit to bttk/bazel-skylib that referenced this issue Nov 13, 2018

Convert depset into a list before iterating
With this change tests pass even if
--incompatible_depset_is_not_iterable is used.

Bazel issue:
bazelbuild/bazel#5816

bttk added a commit to bttk/bazel-skylib that referenced this issue Nov 13, 2018

Convert depset into a list before iterating
With this change tests pass even if
--incompatible_depset_is_not_iterable is used.

Bazel issue:
bazelbuild/bazel#5816

bttk added a commit to bttk/bazel-skylib that referenced this issue Nov 13, 2018

Remove supports for make(depset()) from new_sets
With `--incompatible_depset_is_not_iterable` depset is no longer
iterable. Remove tests that assumed it is.

Bazel issue:
bazelbuild/bazel#5816

laurentlb added a commit to bazelbuild/bazel-skylib that referenced this issue Nov 14, 2018

Remove supports for make(depset()) from new_sets (#68)
With `--incompatible_depset_is_not_iterable` depset is no longer
iterable. Remove tests that assumed it is.

Bazel issue:
bazelbuild/bazel#5816
@manekinekko

This comment has been minimized.

Copy link

commented Jan 19, 2019

Hi, how would you coerce a depset to a flat list without using calling .to_list() (not recommended) nor having to iterate?

@c-parsons

This comment has been minimized.

Copy link
Contributor Author

commented Jan 22, 2019

The recommendation is to avoid flattening a depset in Starlark at all costs -- depsets can be passed to action creation directly (via actions.run)

@m3rcuriel

This comment has been minimized.

Copy link
Contributor

commented May 6, 2019

I'm curious then as to what the recommended syntax would be for checking if a label exists in a depset, i.e what would formerly be if blah in your_depset

@c-parsons

This comment has been minimized.

Copy link
Contributor Author

commented May 6, 2019

From https://docs.bazel.build/versions/master/skylark/lib/depset.html :

Depsets are not implemented as hash sets and do not support fast membership tests. If you need a general set datatype, you can simulate one using a dictionary where all keys map to True.

You cannot check if a label exists in a depset without flattening it.

@laurentlb

This comment has been minimized.

Copy link
Member

commented May 20, 2019

Status update. Flag flip still blocked by:

@davido

This comment has been minimized.

Copy link
Contributor

commented May 21, 2019

@laurentlb My impression was, that it is fixed in all our transitive dependencies and in our own build tool chain. Can you point me to a log, specific to this breakage?

@davido

This comment has been minimized.

Copy link
Contributor

commented May 21, 2019

Thanks. I see it now:

File "/var/lib/buildkite-agent/builds/bk-docker-n5kr/bazel-downstream-projects/gerrit/tools/bzl/junit.bzl", line 38, in _AsClassName
[x.path for x in fname.files]
type 'depset' is not iterable. Use the `to_list()` method to get a list. Use --incompatible_depset_is_not_iterable=false to temporarily disable this check.

I filed: [1].

bazel-io pushed a commit that referenced this issue May 21, 2019

Update tests to avoid implicit depset iteration
Progress towards #5816

RELNOTES: None.
PiperOrigin-RevId: 249341642

bazel-io pushed a commit that referenced this issue Jun 3, 2019

Flip the --incompatible_depset_is_not_iterable flag
RELNOTES[INC]: Depsets can't be iterated over unless they're converted to lists
using the .to_list() method. Use --incompatible_depset_is_not_iterable=false to
temporarily restore the previous behaviour.

#5816

PiperOrigin-RevId: 251249558

@laurentlb laurentlb closed this Jun 3, 2019

irengrig added a commit to irengrig/bazel that referenced this issue Jun 18, 2019

Update tests to avoid implicit depset iteration
Progress towards bazelbuild#5816

RELNOTES: None.
PiperOrigin-RevId: 249341642

irengrig added a commit to irengrig/bazel that referenced this issue Jun 18, 2019

Flip the --incompatible_depset_is_not_iterable flag
RELNOTES[INC]: Depsets can't be iterated over unless they're converted to lists
using the .to_list() method. Use --incompatible_depset_is_not_iterable=false to
temporarily restore the previous behaviour.

bazelbuild#5816

PiperOrigin-RevId: 251249558

pmuetschard added a commit to pmuetschard/gapid that referenced this issue Jun 21, 2019

pmuetschard added a commit to pmuetschard/gapid that referenced this issue Jun 21, 2019

pmuetschard added a commit to google/gapid that referenced this issue Jun 24, 2019

irengrig added a commit to irengrig/bazel that referenced this issue Jul 15, 2019

Flip the --incompatible_depset_is_not_iterable flag
RELNOTES[INC]: Depsets can't be iterated over unless they're converted to lists
using the .to_list() method. Use --incompatible_depset_is_not_iterable=false to
temporarily restore the previous behaviour.

bazelbuild#5816

PiperOrigin-RevId: 251249558
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.