new update_conandata() helper

[memsharded]

Changelog: Feature: New from conan.tools.files import update_conandata() helper to add data to conandata.yml in the export() method.
Docs: conan-io/docs#2422

[santhonisz]

Hi @memsharded,

This looks to be a very useful function and one we were literally about to look at implementing ourselves. However can I ask you to consider an additional usage scenario:

We would like to capture to some data in the conandata.yml for all our package recipes (literally 100s) when running in CI, and so were planning on using a post_export hook. I've checked and the passed conanfile arg doesn't have the export_folder attribute available for this hook type, hence this new function wouldn't be usable in this scenario.

Does it sound reasonable to add support for this use case?

[memsharded]

(BTW, this new helper is being added as it will be necessary to migrate some features to 2.0).

Does it sound reasonable to add support for this use case?

Yes, it could be possible, but I have had a look and seems it could be a bit dirty now in the 1.X, we can try to have a second look, when we revisit the hooks for Conan 2.0, we want to understand better their usage. Can you please describe a bit more what information you are adding to conandata.yml by the hook? Why in a hook and not directly in the recipe. As a rule of thumb:

• Anything that is necessary for the recipe and package to work correctly, should be in the recipe, it shouldn't rely on hooks to be installed to operate
• Hooks can be used as an extra thing, to augment, check, validate, run tools, etc, but if we drop them, the recipes and packages should still be valid.

[santhonisz]

We currently only use hooks when running in our CI environment. They are stored in a dedicated git config repo and installed via Conan Package Tool's CONAN_CONFIG_URL env var. We primarily use them to perform validation against our organisations requirements in regards to our (private) packages e.g. the revision_mode attribute must be set to scm, the license attribute must be set etc. The advantage of doing this in a hook is that we can perform such additional actions against any package being built, while for developers the recipe still functions perfectly well even though they will not be using said hooks.

In the case at hand, we are now wanting to store some CI-related information in the conandata.yml for all packages. This information is primarily SCM-related which, among other things, may be used by recipes in their source method at a later date (much like your unit test). We expect the usage of the source method in their recipes to be opt-in for our teams (we currently do not build "missing" packages either locally or in CI), however we see real value in capturing this information across the board. A hook allows us to do this for all packages built in CI without having to update each individual recipe, of which there are 100s.

Hope that clarifies things a bit.

[memsharded]

In the case at hand, we are now wanting to store some CI-related information in the conandata.yml for all packages. This information is primarily SCM-related which, among other things, may be used by recipes in their source method at a later date (much like your unit test). We expect the usage of the source method in their recipes to be opt-in for our teams (we currently do not build "missing" packages either locally or in CI), however we see real value in capturing this information across the board. A hook allows us to do this for all packages built in CI without having to update each individual recipe, of which there are 100s.

Thanks very much @santhonisz for the details.

As this is closed, I am going to create an issue for future consideration then, now it would be dirty to make the export_folder available to the hooks, but after 2.0 cleaning and refactoring, it might be more doable.