Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document microarchitecture-optimized builds #2091

Merged
merged 2 commits into from
Feb 28, 2024

Conversation

jaimergp
Copy link
Member

@jaimergp jaimergp commented Feb 28, 2024

PR Checklist:

We talked about this in today's conda community meeting so let's get add it to the shiny docs :D Preview link: https://deploy-preview-2091--conda-forge-previews.netlify.app/docs/maintainer/knowledge_base/#microarch

cc @mbargull

@jaimergp jaimergp requested a review from a team as a code owner February 28, 2024 20:00
Copy link

netlify bot commented Feb 28, 2024

Deploy Preview for conda-forge-previews ready!

Name Link
🔨 Latest commit e496664
🔍 Latest deploy log https://app.netlify.com/sites/conda-forge-previews/deploys/65dfa569fa3af4000810b73d
😎 Deploy Preview https://deploy-preview-2091--conda-forge-previews.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@jaimergp jaimergp added the Docs label Feb 28, 2024
Copy link
Member

@beckermr beckermr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huzzah!

@beckermr beckermr merged commit 9388b1e into conda-forge:main Feb 28, 2024
5 checks passed
@traversaro
Copy link
Contributor

Sorry, a bit late to the part. In https://github.com/conda-forge/microarch-level-feedstock it is written:

The run_exports only has a lower bound and therefore a level=2 build can be installed on a level=3 user system. A tighter bound is not added because we want to be able to test both level=2 and level=3 on a CI machine with level=3. Therefore in order to prioritise the highest level, use the build number to prioritise the level.

So the example should also include the logic to increase the build number of packages with higher level?

Comment on lines +1287 to +1292
```yaml title="recipe/conda_build_config.yaml"
microarch_level: # [unix and x86_64]
- 1 # [unix and x86_64]
- 3 # [unix and x86_64]
- 4 # [unix and x86_64]
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, the very recent archspec 0.2.3 now has windows support, in large part due to @isuruf's work on this. 🥳

Not sure what else is necessary to wire this up though, just tried on a fully up-to-date environment:

>conda info
    [...]
       virtual packages : __archspec=1=x86_64
                          __conda=24.1.2=0
                          __win=0=0

@stuarteberg
Copy link

stuarteberg commented May 29, 2024

We ran into a strange bug when we recently used this feature to add microarch-optimized builds to the graph-tool feedstock (via conda-forge/graph-tool-feedstock#140).

This didn't work for us:

microarch_level:  # [unix and x86_64]
  - 1  # [unix and x86_64]
  - 3  # [unix and x86_64]
  - 4  # [unix and x86_64]

...because, weirdly, NON-x86 builds (such as linux_aarch64) were being given a microarch_level environment variable!! (It was set as microarch_level=4.) As far as I can tell, the only explanation is that conda-build has a bug. The relevant files in .ci_support for the non-x86 variants made no mention whatsoever of microarch_level. It should not have had a value at all since the selectors excluded it for those variants.

Fortunately, there was an easy workaround: Always list at least one microarch_level, regardless of the target CPU.

microarch_level:
  - 1
  - 3  # [unix and x86_64]
  - 4  # [unix and x86_64]

Should we modify the conda-forge docs to match? (I have not filed a bug report with conda-build because it will presumably be somewhat time-consuming to produce a minimal example.)

@jaimergp
Copy link
Member Author

A PR is always welcome! Maybe a note with the "gotcha" is enough:

:::note

In non x86 architectures...

:::

@mbargull
Copy link
Member

As far as I can tell, the only explanation is that conda-build has a bug.

It is probably not a bug, but the (IMO, unfortunate) feature of conda-build in that it searches for uses of environment variable corresponding to the variant entries in build scripts and thus considers microarch_level as a "used" due to the ${microarch_level} in your script.
It might suffice to just say ${microarch_level:-1} instead in the build script to avoid it.

stuarteberg added a commit to stuarteberg/conda-forge.github.io that referenced this pull request May 30, 2024
…t one microarch_level, to avoid a bug in conda-build

As described here: conda-forge#2091 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging this pull request may close these issues.

None yet

7 participants