fix: success check does not consider pkg_dir #836

[jpfeuffer]

completely untested but it is an obvious bug.
If you want to change the package dirs earlier in the code somewhere, please suggest changes

[jpfeuffer]

@bgruening @daler Ready to review. Somehow I cannot request it from the normal UI.

[daler]

@jpfeuffer just to clarify, is OpenMS is using bioconda-utils for internal builds?

[jpfeuffer]

Hey @daler
yes, correct. We want to ensure the package builds on bioconda in the moment we trigger a release. And that nightly builds in-between are compatible with the rest of bioconda. If that is possible with pure conda build without too much hassle we can also use that.

[daler]

Ah, ok. Can you point to where (and how) you are calling it then? I bet if it's not exactly how we call it in bioconda-recipes then that might introduce behavior that we don't otherwise test for.

[jpfeuffer]

Sure! Yes, you might not be using the --docker-specific --pkg_dir option to specify a directory to be mounted into the docker container. We use it, so we can reuse already-built packages for the next run and generally have access to the finished packages after the build. I did not check if we can change the upload behavior otherwise that would have been an alternative.

# output must be a valid conda "channel" (otherwise I think I encountered problems)
mkdir -p output/noarch
touch output/noarch/repodata.json
conda activate bioconda
mamba update -y bioconda-utils
conda index output

git clone https://github.com/bioconda/bioconda-recipes && cd bioconda-recipes # we usually use our own fork with ntly changes
bioconda-utils build --docker --pkg_dir $WORKSPACE/output --packages openms-meta # feel free to choose any pkg
bioconda-utils build --docker --pkg_dir $WORKSPACE/output --packages pyopenms

Note: you cannot run them together because of #751
I think it also needed to be an absolute path (therefore $WORKSPACE)

[daler]

So the default behavior is for --pkg_dir to not be set, which in turn defaults to the identified conda-build directory on the host, which is where the built recipes will be copied to and mounted read-only into the container. So the default behavior does, I think, what you are aiming to do.

Does it work if you remove the --pkg_dir arg?

[jpfeuffer]

Yes, I think it works without --pkg_dir but somehow we ended up using a subfolder in the workspace (persistency, permission reasons I guess). We might be able to reevaluate the thing but I think it would be great if the build also works with --pkg_dir.

[daler]

I think it would be ok to merge as-is since we don't use --pkg_dir anywhere in production, but merging without tests always makes me nervous. And there are currently bigger issues to resolve that affect the entire channel (and very limited time to do so).

Can you write a test for this in the test dir, that fails in master branch but passes with your addition?

[jpfeuffer]

@daler Ready. I am 90% sure it will fail on master.

[jpfeuffer]

@daler Do you need anything else?

[timosachsenberg]

Hi, has there been any progress with this PR?

[johanneskoester]

The platform can be inferred from RepoData().native_platform().

[johanneskoester]

Also, make sure that you indent only with 4 spaces.

[poshul]

FAILED test/test_utils.py::test_single_build_pkg_dir - AssertionError: bug: ensure to load config before instantiating RepoData. ERROR test/test_utils.py::test_single_build_only[with docker] - AssertionError: bug: ensure to load config before instantiating RepoData.

[jpfeuffer]

Sorry, I don't know what that means. I merely added the review suggestion. I guess this error is also why the suggested code was not used in the place where I copied it from 🤷.
I guess we have to wait for further suggestions..

[timosachsenberg]

any update here?

[poshul]

@johanneskoester @daler any updates on this?

[poshul]

@johanneskoester @daler any updates?