Allow proxies for nvidia downloads via environment variables #11

h-vetinari · 2019-06-26T11:33:53Z

Checklist

Used a fork of the feedstock to propose changes
Bumped the build number (if the version is unchanged)
Reset the build number to 0 (if the version changed)
Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
Ensured the license file is being packaged.

closes #10

Happy to bikeshed over environment variable names. Unfortunately I can't avoid adding one each for developer.nvidia.com resp. developer.download.nvidia.com, because the files and md5s actually lie on separate urls.

conda-forge-linter · 2019-06-26T11:33:58Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

h-vetinari · 2019-06-26T11:34:10Z

@conda-forge-admin, please rerender

conda-forge-linter · 2019-06-26T12:00:35Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipe) and found some lint.

Here's what I've got...

For recipe:

Selectors are suggested to take a <two spaces>#<one space>[<expression>] form. See lines [20]

conda-forge-linter · 2019-06-26T12:06:10Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

…da-forge-pinning 2019.06.19

h-vetinari · 2019-06-26T13:03:56Z

@conda-forge/cudatoolkit-dev
I managed that the builds complete with a rerendered recipe (on azure), but I cannot figure out the test errors on linux. I tried to build the recipe with the state of this PR on a Ubuntu 18.04 machine and it ran through without errors.

andersy005 · 2019-06-26T13:58:13Z

Thank you for the PR, @h-vetinari!

I tried to build the recipe with the state of this PR on a Ubuntu 18.04 machine and it ran through without errors.

I've seen similar behavior in the past, and I don't know what causes it. I will try to re-run the CI builds and see what happens.

scopatz · 2019-06-26T14:16:00Z

recipe/meta.yaml

  skip: True  # [win]

 requirements:
+  build:
+    - python >=3.6 [osx]  # osx only has python 2 by default


Selectors need to be behind a comment, ie maye this should be:

# osx only has python 2 by default - python >=3.6 # [osx]

Ah yes, silly oversight. I had swapped things around from this after the linter complained.

scopatz · 2019-06-26T14:24:56Z

Also, linux builds seems to be missing files now: https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=45047

h-vetinari · 2019-06-26T14:26:10Z

It's the same error that showed up on mac before I added python to the build setup. Guess we need the build dep for linux too.

h-vetinari · 2019-06-26T14:38:20Z

So, now we're back to the ClobberWarning (that didn't leave by either specifying python or tqdm as a test dep), and the invisible test errors.

scopatz · 2019-06-26T14:47:19Z

Yeah I don't understand that one

h-vetinari · 2019-06-27T21:23:24Z

@andersy005 @xmnlab
Any ideas what I can do here to fix the tests? @andersy005 said he already had similar issues before, but so far they don't seem to disappear just from restarting the CI...

h-vetinari · 2019-07-03T16:10:30Z

@conda-forge/core
Could anyone help debug this? Or at least, how to get an error log from that failed test?

scopatz · 2019-07-03T22:05:53Z

I think you can cat / echo a log file to the screen if you need it.

h-vetinari · 2019-07-14T15:36:47Z

The only thing I can find from googling about these ClobberWarnings is this comment from @jakirkham, where it was necessary to add a package (here tdqm?) to both host and run dependencies.

I tried that with the last few commits but the warnings (and failure) remains the same.

I'm not entirely clear what's happening with that clobber-stuff. Especially since it's complaining that it's clobbering an existing path, but then shows source path: None in the Warning...

recipe/meta.yaml

h-vetinari · 2019-07-14T17:08:51Z

@andersy005
The build failed again, but curiously, I can't see a log, only:
Run docker build failed

andersy005 · 2019-07-14T17:09:06Z

@conda-forge-admin, please rerender

conda-forge-linter · 2019-07-14T17:09:53Z

Hi! This is the friendly automated conda-forge-webservice.

I tried to re-render for you, but it looks like there was nothing to do.

h-vetinari · 2019-07-14T17:13:56Z

Interestingly, the osx-build for aec7673 failed as well...

h-vetinari · 2019-07-14T17:17:07Z

Adding conda to host (I'm guessing without the other changes from #14) means we're back to the CondaMultiError in the linux build that I mentioned above.

Maybe it's better to just focus on #14 for the time being?

recipe/meta.yaml

conda-forge-linter · 2019-07-14T17:31:35Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipe) and found some lint.

Here's what I've got...

For recipe:

The requirements section contained an unexpected subsection name. test is not a valid subsection name.

recipe/meta.yaml

andersy005 · 2019-07-14T17:51:10Z

could build on Ubuntu 18.04 without a problem.

The CI is using vmImage: ubuntu-16.04, and I am not sure why this might be the culprit...

h-vetinari · 2019-07-14T18:10:34Z

@andersy005: The CI is using vmImage: ubuntu-16.04, and not I am not sure why this might be the culprit...

Yeah, I'm pretty sure the difference is somewhere else. Also didn't have ClobberWarnings, etc. at the time.

andersy005 · 2019-07-14T18:13:24Z

I was wrong about the VM being the issue.. The VM is running ubuntu-16-04, however, everything ends up running in a container with centos I think.

andersy005 · 2019-07-14T19:02:43Z

@h-vetinari,

I was able to build the package on both "Debian GNU/Linux 9 (stretch)" and "Ubuntu 18.04.2 LTS". At this point, I don't really know why building the package on centos fails.

andersy005 · 2019-07-14T21:20:52Z

I found out that cudatoolkit installer is hard-coded to use perl from /usr/bin/perl. The current centos image doesn't come with perl. The available installation is in /opt/conda/bin/perl

/opt/conda/pkgs/cudatoolkit-dev/cuda_10.0.130_410.48_linux: ./cuda-installer.pl: /usr/bin/perl: bad interpreter: No such file or directory

[conda@7cf862609754 ~]$ which perl
/opt/conda/bin/perl

andersy005 · 2019-07-14T21:48:01Z

@conda-forge-admin, please rerender

conda-forge-linter · 2019-07-14T21:48:33Z

Hi! This is the friendly automated conda-forge-webservice.

I tried to re-render for you but ran into some issues, please ping conda-forge/core for further assistance.

h-vetinari · 2019-07-15T06:00:54Z

@andersy005: I found out that cudatoolkit installer is hard-coded to use perl from /usr/bin/perl.

Thanks for the investigation! This can be easily added with yum_requirements.txt. Let's see if this works. :)

h-vetinari · 2019-07-15T06:27:56Z

OK, it didn't work. However, I can't see the yum install in the build log, so not sure what's happening there.

EDIT: the recipe needed to be rerendered, because the azure build_steps.sh gets generated based on (i.a.) the content of yum_requirements.txt

h-vetinari · 2019-07-15T11:46:45Z

@andersy005
Success! :)

This now builds successfully, using the old tests (and my fix for the chmod issue). If you want to copy some of your other changes from #14, feel free.

h-vetinari · 2019-07-15T11:54:56Z

I rebased and cleaned up the history a bit, so that there's not so many garbage commits floating around (if you don't squash-merge).

andersy005 · 2019-07-15T17:22:10Z

@h-vetinari, thank you for the PR and the additional fixes. I am merging this soon if there's no objection

…da-forge-pinning 2019.07.13

h-vetinari · 2019-07-16T05:53:11Z

@andersy005: @h-vetinari, thank you for the PR and the additional fixes. I am merging this soon if there's no objection

No problem! It was a team effort, because without your investigation about the hardlink to /usr/bin/perl I wouldn't have known what to do... ;-)

I rebased once more to correct a wrong commit message and add a comment to the yum-requirement.

h-vetinari · 2019-07-16T05:54:46Z

This should now also close #12 and therefore supersede #14. @jorisvandenbossche

andersy005 · 2019-07-16T16:39:36Z

Closes #14

conda-forge-admin and others added 2 commits June 26, 2019 14:40

MNT: Re-rendered with conda-build 3.18.4, conda-smithy 3.4.0, and con…

c82e280

…da-forge-pinning 2019.06.19

allow proxies for nvidia downloads through env vars

519aabc

h-vetinari force-pushed the master branch from a95f0ae to caab627 Compare June 26, 2019 12:42

scopatz reviewed Jun 26, 2019

View reviewed changes

h-vetinari force-pushed the master branch from caab627 to 62f08b8 Compare June 26, 2019 14:18

h-vetinari force-pushed the master branch from 62f08b8 to e22a336 Compare June 26, 2019 14:27

h-vetinari mentioned this pull request Jul 14, 2019

Remove conda as run dependency #14

Closed

4 tasks

andersy005 requested changes Jul 14, 2019

View reviewed changes

recipe/meta.yaml Outdated Show resolved Hide resolved

andersy005 reviewed Jul 14, 2019

View reviewed changes

recipe/meta.yaml Outdated Show resolved Hide resolved

andersy005 reviewed Jul 14, 2019

View reviewed changes

recipe/meta.yaml Show resolved Hide resolved

andersy005 mentioned this pull request Jul 14, 2019

Proposal: Add system wide perl installation to condaforge/linux-anvil-comp7 conda-forge/docker-images#110

Closed

switch order of chmod/copy due to read-only FS

0ab0960

h-vetinari force-pushed the master branch from 63dea41 to 83b059c Compare July 15, 2019 11:53

andersy005 approved these changes Jul 15, 2019

View reviewed changes

h-vetinari added 3 commits July 16, 2019 07:46

update requirements; remove conda from run req.

9f58aa2

add perl to yum-requirements

8693a9c

MNT: Re-rendered with conda-build 3.18.7, conda-smithy 3.4.0, and con…

a42998c

…da-forge-pinning 2019.07.13

h-vetinari force-pushed the master branch from 83b059c to a42998c Compare July 16, 2019 05:51

andersy005 closed this Jul 16, 2019

andersy005 reopened this Jul 16, 2019

andersy005 merged commit e4f7026 into conda-forge:master Jul 16, 2019

andersy005 mentioned this pull request Jul 16, 2019

Is conda needed as a run dependency? #12

Closed

h-vetinari mentioned this pull request Jul 24, 2019

Load environment variables already for env setup conda/conda-build#3629

Closed

andersy005 mentioned this pull request Jul 28, 2019

LinkError: post-link script failed for package conda-forge::cudatoolkit-dev-10.0-1 #9

Closed

h-vetinari mentioned this pull request Aug 29, 2019

Publish conda recipes to conda-forge? DeepGraphLearning/graphvite#2

Closed

Allow proxies for nvidia downloads via environment variables #11

Allow proxies for nvidia downloads via environment variables #11

Conversation

h-vetinari commented Jun 26, 2019

conda-forge-linter commented Jun 26, 2019

h-vetinari commented Jun 26, 2019

conda-forge-linter commented Jun 26, 2019

conda-forge-linter commented Jun 26, 2019

h-vetinari commented Jun 26, 2019

andersy005 commented Jun 26, 2019

scopatz Jun 26, 2019

Choose a reason for hiding this comment

h-vetinari Jun 26, 2019 • edited

Choose a reason for hiding this comment

scopatz commented Jun 26, 2019

h-vetinari commented Jun 26, 2019

h-vetinari commented Jun 26, 2019

scopatz commented Jun 26, 2019

h-vetinari commented Jun 27, 2019

h-vetinari commented Jul 3, 2019

scopatz commented Jul 3, 2019

h-vetinari commented Jul 14, 2019

h-vetinari commented Jul 14, 2019

andersy005 commented Jul 14, 2019

conda-forge-linter commented Jul 14, 2019

h-vetinari commented Jul 14, 2019

h-vetinari commented Jul 14, 2019 • edited

conda-forge-linter commented Jul 14, 2019

andersy005 commented Jul 14, 2019 • edited

h-vetinari commented Jul 14, 2019 • edited

andersy005 commented Jul 14, 2019

andersy005 commented Jul 14, 2019

andersy005 commented Jul 14, 2019 • edited

andersy005 commented Jul 14, 2019

conda-forge-linter commented Jul 14, 2019

h-vetinari commented Jul 15, 2019

h-vetinari commented Jul 15, 2019 • edited

h-vetinari commented Jul 15, 2019

h-vetinari commented Jul 15, 2019

andersy005 commented Jul 15, 2019

h-vetinari commented Jul 16, 2019

h-vetinari commented Jul 16, 2019

andersy005 commented Jul 16, 2019

h-vetinari Jun 26, 2019 •

edited

h-vetinari commented Jul 14, 2019 •

edited

andersy005 commented Jul 14, 2019 •

edited

h-vetinari commented Jul 14, 2019 •

edited

andersy005 commented Jul 14, 2019 •

edited

h-vetinari commented Jul 15, 2019 •

edited