[question] how to inject SCM credentials when building packages from sources

[GordonJess]

I've already read a related issues on this topic (5595) but wasn't able to find a solution to my problem.

When I try to rebuild a package from source, conan tries to checkout the url which was automatically stored in the scm::url attribute. But since the url is stored without the credentials since 4207 (which I do agree with), the cloning of sources fails during authentication. I also want to avoid storing credentials in scm::username or scm::password fields.

I did read a suggestion to use the proxies section in conan.conf to embed user/password in urls, but I from what I can tell the user:pass here is only for authenticating against the proxy itself, not the request url. Or maybe I'm missing something...?

Git credential caches are also not a good option for me since they still require the password to be manually input every so often, which would be a real pain for the CI slaves.

Ideally I'd like to provide the credentials to conan on the command line. or for conan to take the values from environmental variables (like the solution for artifactory credentials). Of course this might be an issue when one command rebuilds several packages coming from different sources which need different creds. Maybe storing default creds per url in conan.conf can have default credentials per url.

How are other users typically handling this? Any suggestions would be much appreciated!

[czoido]

Hi @GordonJess,

We are going to release some new things for scm in 1.22 that maybe could help you:

• Opt-in to store SCM evaluated data into 'conandata.yml' file #6334
• Docs about 'scm_to_conandata' docs#1522

You will be able to define a scm_to_conandata in the conan.conf so that if you are using auto your scm dictionary is evaluated in the conandata.yml (but not the username and password). That way you can get the username and password from the environment and that information won't be stored anywhere.

[GordonJess]

Brilliant! Looking forward to it!

[jgsogo]

#6334 was merged and released as an opt-in in v1.22. I think it should close this issue too, right?

[jonathangoorin]

@czoido Hi I'm new to conan - and I can't quite figure how to use your solution.
I Also have private repository that I need to clone in source method and for that I have to pass the github credentials from jenkins

If you can point me up to location for the RTFM, that'd be also great :)

[czoido]

Hi @jonathangoorin,
This is the section in the documentation: https://docs.conan.io/en/latest/creating_packages/package_repo.html#capturing-the-remote-and-commit-scm
We also have an example in the release post: https://blog.conan.io/2020/02/11/New-conan-release-1-22.html
Hope this helps, feel free to ask if you have any doubts on how to use it.

[Parcley]

I don't really understand how this solves the question:
In this case, we can only store credentials in conandata.yml for packages, which we are creating on our own.

But image if I have two packages:

• package rootPackage, which has a dependency on dependency
• package dependency, which has its source code in a git repo (which requires credentials)

If I want to install all dependencies for rootPackage, dependency is being built (e.g. when using --build=missing and the correct binary is not available). When trying to build dependency, its sources need to be downloaded and now we cannot download the sources (typical error messages will be: fatal: could not read Username for '<repo>': No such device or address).

Now this could be solved by storing git credentials for dependency's git repository in my computer's git configuration files. But this is not always a good idea for automated CI machines, which do not want to permanently store credentials.

A solution, which might work, is to add new environment variables (e.g. CONAN_SCM_USERNAME and CONAN_SCM_PASSWORD), which will auttomatically be used for the Scm class, if no explicit credentials are provided in the conanfile.

the logic of reading from an environment variable should probably be near this code:

conan/conans/client/tools/scm.py

Line 50 in e17ab7e

self._password = password

[ptomasz1]

I agree with @Parcley - proposed solution solved completely different issue.

Are you going to address the right problem?

[memsharded]

Conan 2.0 provides a source_credentials.json file to define credentials for tools.file.download(), please check: https://docs.conan.io/2/reference/config_files/source_credentials.html

Otherwise, this issue is quite outdated, the scm component has been removed in Conan 2.0 (and already legacy in Conan 1.X, with alternative approaches). For git credentials, the recommendation is to use orthogonal authentication methods, like ssh-keys, GIT_ASKPASS or something similar, and if env-vars are to be be used, for sure recipes can os.getenv() and load those env-vars to provide auth for their git-clone operations.

As this is outdated issue referring to removed scm, I am going to close it as outdated. If you think there is still a gap for Conan 2.0 @ptomasz1, it would be great to have a new ticket, referring to the new approaches (raw conandata.yml + source() method, without scm component). Thanks!