[bug] Options for transitive dependencies of "compatible" packages are not locked correctly

[ytimenkov]

Environment Details (include every applicable attribute)

• Conan version: 1.40 and develop

Steps to reproduce (Include if Applicable)

I've prepared a test which shows the problem: develop...ytimenkov:comptatible-options-lockfile

When package which provides "compatible packages" is locked as a compatible one, its options are not correctly stored in the lockfile:
Expected: All options for the package and its transitive package are stored in the lockfile (the latter is in form dep:option=Value
Actual: Only package's options are stored in the lockfile.

This results in lockfile being unusable (can't be used for the following install command) since it fails with message like:

ERROR: pkg/1.0@: Locked options do not match computed options
Locked options:
coverage=False
testing=False
Computed options:
coverage=False
testing=False
boost:fPIC=True
boost:shared=False

It works fine if the package is consumed as not compatible (i.e when settings match or overridden per package via -s pkg:setting=value)

(Something suggests that plain copying of options at around https://github.com/conan-io/conan/blob/develop/conans/client/graph/graph_binaries.py#L187 is not enough...).

[memsharded]

Thanks @ytimenkov for the reproducible test, that really helped.

I am investigating this possible fix here: #9890. Lets see how it goes. Irrespective of if this can progress, I would like to comment a few things:

• This might be solved in Conan 2.0, we are trying to simplify the lockfiles so they don't have these issues.
• We are considering and we will submit to the Tribe the possibility of removing the compatible_packages. It seems it is too complicated ,confusing for users and fragile, for the provided value. Explicitly defining things in profiles files works very well, the less magic the better.

[ytimenkov]

@memsharded Thanks for addressing this!

I won't be screaming if you remove compatible packages (CP) feature if it complicates things much, but I find the concept of it quite handy.

The major difference is that with CP it's the package which advertises its capabilities rather than "pull" with profiles. Somewhat in line with newly-introduced "package types".

We have a number of libraries which have "stable C-ABI" and everything statically linked into them so they are compiler-independent and usually built with experimental new features/compilers, until the rest catches up.

To consume them we need to set per-package setting and it gets really hairy when number of such libraries grows and it needs to be propagated through the whole graph to every consumer.

Globally distributed profiles won't solve it because we want to evolve and it's a (compiler, lib version) pair.

Of course I could remove compiler from settings if it's C-ABI, but then I lose the option to conditionally consume a newer version for testing (for example when we rolling out a version built with new compiler I can override per-package setting and run the test, and have it for some time, before making new compiler "default"). Probably this use-case can be solved by channel 🤷‍♂️ (you're deprecate them only for conan-center, aren't you?).

(sure I can copy-paste this to tribe poll later 😏)