Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix channels_remap handling in extra_envs configuration #808

Merged
merged 15 commits into from
Jun 22, 2024
2 changes: 1 addition & 1 deletion CONSTRUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ See notes in `channels_remap` for details about local channels.
_required:_ no<br/>
_type:_ list<br/>

A list of `src/dest` channel pairs. When building the installer, conda will
A list of `src/dest` channel URL pairs. When building the installer, conda will
marcoesters marked this conversation as resolved.
Show resolved Hide resolved
use the `src` channels to solve and fetch the packages. However, the resulting
installation will see the packages as coming from the `dest` equivalent.
This allows an installer to be built against a different set of channels than
Expand Down
2 changes: 1 addition & 1 deletion constructor/construct.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
'''),

('channels_remap', False, list, '''
A list of `src/dest` channel pairs. When building the installer, conda will
A list of `src/dest` channel URL pairs. When building the installer, conda will
marcoesters marked this conversation as resolved.
Show resolved Hide resolved
use the `src` channels to solve and fetch the packages. However, the resulting
installation will see the packages as coming from the `dest` equivalent.
This allows an installer to be built against a different set of channels than
Expand Down
13 changes: 10 additions & 3 deletions constructor/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -149,10 +149,16 @@ def main_build(dir_path, output_dir='.', platform=cc_platform,
if env_name in ("base", "root"):
raise ValueError(f"Environment name '{env_name}' cannot be used")
for config_key, value in env_config.copy().items():
if isinstance(value, (list, tuple)):
env_config[config_key] = [val.strip() for val in value]
if config_key == "environment_file":
env_config[config_key] = abspath(join(dir_path, value))
elif config_key == "channels_remap":
env_config[config_key] = [
{"src": item["src"].strip(), "dest": "dest".strip()} for item in value
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{"src": item["src"].strip(), "dest": "dest".strip()} for item in value
{"src": item["src"].strip(), "dest": item["dest"].strip()} for item in value

The way this code is written, it looks like the dest will always be "dest". I'm surprised the tests pass - sounds like the example is maybe not as representative as they should be?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, good catch. This would only be caught if we try to update the resulting installation or run conda list on them because that's where the remapped channels would show up.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extended a test

]
elif isinstance(value, (list, tuple)):
env_config[config_key] = [val.strip() for val in value]
else:
env_config[config_key] = value
marcoesters marked this conversation as resolved.
Show resolved Hide resolved

try:
exe_name, exe_version = identify_conda_exe(info.get("_conda_exe"))
Expand Down Expand Up @@ -418,7 +424,8 @@ def main():
easiest way to obtain one is to install the 'conda-standalone' package.
Alternatively, you can download an executable manually and supply its
path with the --conda-exe argument. Self-contained executables can be
downloaded from https://repo.anaconda.com/pkgs/misc/conda-execs/""".lstrip())
downloaded from https://repo.anaconda.com/pkgs/misc/conda-execs/ and/or
https://github.com/conda/conda-standalone/releases""".lstrip())

out_dir = normalize_path(args.output_dir)
main_build(dir_path, output_dir=out_dir, platform=args.platform,
Expand Down
2 changes: 1 addition & 1 deletion docs/source/construct-yaml.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ See notes in `channels_remap` for details about local channels.
_required:_ no<br/>
_type:_ list<br/>

A list of `src/dest` channel pairs. When building the installer, conda will
A list of `src/dest` channel URL pairs. When building the installer, conda will
use the `src` channels to solve and fetch the packages. However, the resulting
installation will see the packages as coming from the `dest` equivalent.
This allows an installer to be built against a different set of channels than
Expand Down
7 changes: 5 additions & 2 deletions examples/extra_envs/construct.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@ name: ExtraEnvs
version: X
installer_type: all
channels:
- http://repo.anaconda.com/pkgs/main/
- https://conda.anaconda.org/conda-forge
specs:
- python=3.9
- conda # conda is required for extra_envs
- console_shortcut # [win]
- miniforge_console_shortcut # [win]
exclude: # [unix]
- tk # [unix]
extra_envs:
Expand All @@ -16,6 +16,9 @@ extra_envs:
- pip
channels:
- conda-forge
channels_remap:
- src: https://conda.anaconda.org/conda-forge
dest: conda-forge
exclude:
- setuptools
dav1d:
Expand Down
19 changes: 19 additions & 0 deletions news/808-fix-env-channels-remap
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
### Enhancements

* <news item>

### Bug fixes

* Fix `channels_remap` sanitization when included as part of an item in `extra_envs`. (#808)

### Deprecations

* <news item>

### Docs

* Clarify that channels must be passed as URLs in `channels_remap`. (#808)

### Other

* Mention `conda/conda-standalone` repository as a source for conda-standalone downloads in relevant error messages. (#808)
10 changes: 5 additions & 5 deletions tests/test_examples.py
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ def _sentinel_file_checks(example_path, install_dir):
if (example_path / script).exists() and not (install_dir / sentinel).exists():
raise AssertionError(
f"Sentinel file for {script_prefix}_install not found! "
f"{install_dir} contents:\n" + "\n".join(sorted(install_dir.iterdir()))
f"{install_dir} contents:\n" + "\n".join(sorted(map(str, install_dir.iterdir())))
)


Expand Down Expand Up @@ -511,12 +511,12 @@ def test_example_signing(tmp_path, request):

@pytest.mark.skipif(sys.platform != "win32", reason="Windows only")
@pytest.mark.skipif(
not shutil.which("azuresigntool") and not os.environ.get("AZURE_SIGNTOOL_PATH"),
reason="AzureSignTool not available"
not shutil.which("azuresigntool") and not os.environ.get("AZURE_SIGNTOOL_PATH"),
reason="AzureSignTool not available",
)
@pytest.mark.parametrize(
"auth_method",
os.environ.get("AZURE_SIGNTOOL_TEST_AUTH_METHODS", "token,secret").split(","),
"auth_method",
os.environ.get("AZURE_SIGNTOOL_TEST_AUTH_METHODS", "token,secret").split(","),
)
def test_azure_signtool(tmp_path, request, monkeypatch, auth_method):
"""Test signing installers with AzureSignTool.
Expand Down
Loading