Rebuild for python38 #31

regro-cf-autotick-bot · 2019-12-03T05:24:48Z

This PR has been triggered in an effort to update python38.

Notes and instructions for merging this PR:

Please merge the PR only after the tests have passed.
Feel free to push to the bot's branch to update this PR if needed.
Please note that if you close this PR we presume that the feedstock has been rebuilt, so if you are going to perform the rebuild yourself don't close this PR until the your rebuild has been merged.

This package has the following downstream children:
qcfractal
And potentially more.
If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one.

_{This PR was created by the cf-regro-autotick-bot.

The cf-regro-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. If you would like a local version of this bot, you might consider using rever. Rever is a tool for automating software releases and forms the backbone of the bot's conda-forge PRing capability. Rever is both conda (conda install -c conda-forge rever) and pip (pip install re-ver) installable.

Finally, feel free to drop us a line if there are any issues!

This PR was generated by https://circleci.com/gh/regro/circle_worker/12581, please use this URL for debugging}

…nda-forge-pinning 2019.11.26

conda-forge-linter · 2019-12-03T05:24:52Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

ocefpaf · 2019-12-03T13:26:38Z

There is no need to keep adding the bot-rerun label. There is a problem with the build here, hangs on macOS, that we need to identify and fix.

samuelcolvin · 2019-12-03T13:28:23Z

I've added that label once.

The PR was created but hung on one thing, that seems the obvious next step.

I'll leave corda forge well alone from now on.

ocefpaf · 2019-12-03T13:31:50Z

I've added that label once.

The label was also used in #30. It is a useful tool to re-send a PR when there are merge conflicts. Not the case here.

The PR was created but hung on one thing, that seems the obvious next step.

Yep. That is the problem that we need to address. Re-issuing the PR won't help.

I'll leave corda forge well alone from now on.

It is not a big deal. Just mentioned to avoid wasting your time re-labeling and hanging again.

cdeil · 2019-12-03T18:44:39Z

@ocefpaf - Any idea why the macOS build hangs?

I don't see a log or error message, just a black screen:

https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=100135&view=logs&jobId=a8dee1cf-d17f-5cb0-b53f-4a93dfdbfb0c

ocefpaf · 2019-12-03T18:47:08Z

@ocefpaf - Any idea why the macOS build hangs?

Sorry, no. I'm not a macOS person. Also, someone reported that this passed on a local build, so it may be related to the Azure image.

Pinging @conda-forge/core to see if someone has any idea how to debug this.

regro-cf-autotick-bot · 2019-12-03T21:10:53Z

Due to the bot-rerun label I'm closing this PR. I will make another one as appropriate. This was generated by https://circleci.com/gh/regro/circle_worker/12597

mbargull · 2019-12-03T22:05:14Z

Even if Azure does not display the log after the job, you can download it.

unzip -p log_93_100135.zip | tail -n50
2019-12-03T11:25:30.5835260Z       File "/usr/local/miniconda/conda-bld/pydantic_1575350881455/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.8/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
2019-12-03T11:25:30.5836740Z         main_content = runpy.run_path(main_path,
2019-12-03T11:25:30.5838610Z       File "/usr/local/miniconda/conda-bld/pydantic_1575350881455/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.8/runpy.py", line 262, in run_path
2019-12-03T11:25:30.5839940Z         return _run_module_code(code, init_globals, run_name,
2019-12-03T11:25:30.5841810Z       File "/usr/local/miniconda/conda-bld/pydantic_1575350881455/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.8/runpy.py", line 95, in _run_module_code
2019-12-03T11:25:30.5842910Z         _run_code(code, mod_globals, init_globals,
2019-12-03T11:25:30.5844750Z       File "/usr/local/miniconda/conda-bld/pydantic_1575350881455/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.8/runpy.py", line 85, in _run_code
2019-12-03T11:25:30.5845850Z         exec(code, run_globals)
2019-12-03T11:25:30.5847530Z       File "/private/tmp/pip-req-build-5vl_lt1e/setup.py", line 55, in <module>
2019-12-03T11:25:30.5848560Z         ext_modules = cythonize(
2019-12-03T11:25:30.5850430Z       File "/usr/local/miniconda/conda-bld/pydantic_1575350881455/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.8/site-packages/Cython/Build/Dependencies.py", line 1072, in cythonize
2019-12-03T11:25:30.5851530Z         pool = multiprocessing.Pool(
2019-12-03T11:25:30.5853380Z       File "/usr/local/miniconda/conda-bld/pydantic_1575350881455/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.8/multiprocessing/context.py", line 119, in Pool
2019-12-03T11:25:30.5854490Z         return Pool(processes, initializer, initargs, maxtasksperchild,
2019-12-03T11:25:30.5856350Z       File "/usr/local/miniconda/conda-bld/pydantic_1575350881455/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.8/multiprocessing/pool.py", line 212, in __init__
2019-12-03T11:25:30.5857450Z         self._repopulate_pool()
2019-12-03T11:25:30.5859440Z       File "/usr/local/miniconda/conda-bld/pydantic_1575350881455/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.8/multiprocessing/pool.py", line 303, in _repopulate_pool
2019-12-03T11:25:30.5860550Z         return self._repopulate_pool_static(self._ctx, self.Process,
2019-12-03T11:25:30.5862460Z       File "/usr/local/miniconda/conda-bld/pydantic_1575350881455/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.8/multiprocessing/pool.py", line 326, in _repopulate_pool_static
2019-12-03T11:25:30.5863760Z         w.start()
2019-12-03T11:25:30.5865620Z       File "/usr/local/miniconda/conda-bld/pydantic_1575350881455/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.8/multiprocessing/process.py", line 121, in start
2019-12-03T11:25:30.5867050Z         self._popen = self._Popen(self)
2019-12-03T11:25:30.5868960Z       File "/usr/local/miniconda/conda-bld/pydantic_1575350881455/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
2019-12-03T11:25:30.5870290Z         return Popen(process_obj)
2019-12-03T11:25:30.5872170Z       File "/usr/local/miniconda/conda-bld/pydantic_1575350881455/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
2019-12-03T11:25:30.5873270Z         super().__init__(process_obj)
2019-12-03T11:25:30.5875160Z       File "/usr/local/miniconda/conda-bld/pydantic_1575350881455/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
2019-12-03T11:25:30.5876270Z         self._launch(process_obj)
2019-12-03T11:25:30.5878120Z       File "/usr/local/miniconda/conda-bld/pydantic_1575350881455/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 42, in _launch
2019-12-03T11:25:30.5879230Z         prep_data = spawn.get_preparation_data(process_obj._name)
2019-12-03T11:25:30.5881150Z       File "/usr/local/miniconda/conda-bld/pydantic_1575350881455/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.8/multiprocessing/spawn.py", line 154, in get_preparation_data
2019-12-03T11:25:30.5882250Z         _check_not_importing_main()
2019-12-03T11:25:30.5884120Z       File "/usr/local/miniconda/conda-bld/pydantic_1575350881455/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.8/multiprocessing/spawn.py", line 134, in _check_not_importing_main
2019-12-03T11:25:30.5885780Z         raise RuntimeError('''
2019-12-03T11:25:30.5886830Z     RuntimeError:
2019-12-03T11:25:30.5887790Z             An attempt has been made to start a new process before the
2019-12-03T11:25:30.5888780Z             current process has finished its bootstrapping phase.
2019-12-03T11:25:30.5889700Z 
2019-12-03T11:25:30.5890850Z             This probably means that you are not using fork to start your
2019-12-03T11:25:30.5891790Z             child processes and you have forgotten to use the proper idiom
2019-12-03T11:25:30.5892710Z             in the main module:
2019-12-03T11:25:30.5893580Z 
2019-12-03T11:25:30.5895070Z                 if __name__ == '__main__':
2019-12-03T11:25:30.5896070Z                     freeze_support()
2019-12-03T11:25:30.5897050Z                     ...
2019-12-03T11:25:30.5897950Z 
2019-12-03T11:25:30.5898850Z             The "freeze_support()" line can be omitted if the program
2019-12-03T11:25:30.5899780Z             is not going to be frozen to produce an executable.
2019-12-03T11:25:30.6885810Z ##[error]The operation was canceled.
2019-12-03T11:25:30.6909590Z ##[section]Finishing: Build recipe

=> Maybe try a patch that removes https://github.com/samuelcolvin/pydantic/blob/v1.2/setup.py#L58 .
This could be a bug in Cython, maybe related to python/cpython#13603 ?

cdeil · 2019-12-03T22:11:33Z

Python multiprocessing isn't super stable, there's multiple issues across Python versions (different one in each) and operating system.

@samuelcolvin - I can see the appeal of a fast build, but could you consider whether for release the default could be to not use multi-threading or multi-processing in the pydantic build? Maybe there could still be a flag that lets devs activate it during development - I think that's the main use case where having to wait a bit is annoying?

djsutherland · 2019-12-04T04:11:02Z

@samuelcolvin, FYI bot-rerun is meant for things like merge conflicts on the PR, not for just rerunning builds after an intermittent failure / dependency update / etc. For the latter, you could close and reopen the PR (if you're a maintaner) or ask the bot to do that for you (if not), among other things.

(We should probably change the instructions in the bot comment, it's kind of confusing.)

samuelcolvin · 2019-12-04T11:26:56Z

@samuelcolvin, FYI bot-rerun is meant for things like merge conflicts on the PR, not for just rerunning builds after an intermittent failure / dependency update / etc

Yes, I know. @ocefpaf has already explained this above.

For those of us not using conda-forge day to day it seeed like the simplest way of re-running the build and thereby finding out if the problem is intermittent - I'm not clear why it's caused such ire.

Yes, clarifying the notes would help.

@samuelcolvin - I can see the appeal of a fast build, but could you consider whether for release the default could be to not use multi-threading or multi-processing in the pydantic build?

I'd be very happy to remove nthreads=4 since I think it only effects the cythonizing, not the compiling and therefore has virtually zero effect on performance. ~~However I would guess that causes cython to use threads not processes, so I'm not 100% convince it will fix the problem.~~ Let's try it and see.

Is there a way to check if conda-forge builds are succeeding without creating a new release? This would be useful to avoid having to make lots of releases.

samuelcolvin · 2019-12-04T11:37:02Z

~~However I would guess that causes cython to use threads not processes, so I'm not 100% convince it will fix the problem.~~ I was wrong:

:param nthreads: The number of concurrent builds for parallel compilation (requires the ``multiprocessing`` module).

samuelcolvin · 2019-12-04T11:44:09Z

hopefully fixed via pydantic/pydantic#1074

ocefpaf · 2019-12-04T12:14:35Z

I'm not clear why it's caused such ire.

It did not cause any hire 😄 We only took the opportunity to explain to our users how our infrastructure works. (And with that we take notes to improve the docs.)

BTW, if your goal is only to restart a job you can close and re-open a PR or use https://conda-forge.org/docs/maintainer/infrastructure.html?highlight=restart#conda-forge-admin-please-restart-ci

mbargull · 2019-12-04T12:16:18Z

Is there a way to check if conda-forge builds are succeeding without creating a new release? This would be useful to avoid having to make lots of releases.

Yes, as mentioned above, via a patch. I.e., turn pydantic/pydantic@f3e08d9 into a patch and add it to the recipe:

cd /path/to/repo/of/pydantic/
git format-patch -1 -o /path/to/repo/of/pydantic-feedstock/recipe/ f3e08d901f1bb3d6d0eae1572d332f5ff3e7909b
# the above outputs some 0001-filename.patch
# add that 0001-filename.patch to a list in source/patches inside meta.yaml

samuelcolvin · 2019-12-04T12:29:19Z

see #33

cdeil · 2019-12-04T14:07:55Z

@samuelcolvin @ocefpaf @mbargull - Thank you!

I know such issues are frustrating and the comments here were a bit heated. Please know that the software you write (pydantic) and the packaging work you do (conda-forge) is awesome and much appreciated! Package install issue workaround implemented, new release, packaged and distributed to all major platforms in a day, that's amazing!

samuelcolvin · 2019-12-29T11:42:13Z

Hi all, build for this library is now failing for the next release due to a problem downloading a patch, see #35.

Any idea why this is?

samuelcolvin · 2019-12-29T11:44:05Z

(I fixed the setup which caused the problem here in the v1.3 release so was hoping a successful release on feedstock, but no luck again.)

regro-cf-autotick-bot added 2 commits December 3, 2019 05:24

bump build number

7c2dbe1

MNT: Re-rendered with conda-build 3.18.11, conda-smithy 3.6.3, and co…

56b9d78

…nda-forge-pinning 2019.11.26

regro-cf-autotick-bot requested review from dgasmith and samuelcolvin as code owners December 3, 2019 05:24

samuelcolvin added the bot-rerun Instruct the bot to retry the PR label Dec 3, 2019

cdeil mentioned this pull request Dec 3, 2019

gammapy v0.15 conda-forge/gammapy-feedstock#14

Merged

3 tasks

cdeil mentioned this pull request Dec 3, 2019

Support Python 3.8 gammapy/gammapy#2466

Closed

regro-cf-autotick-bot closed this Dec 3, 2019

regro-cf-autotick-bot deleted the rebuild-python3801 branch December 3, 2019 21:10

samuelcolvin mentioned this pull request Dec 4, 2019

change cython nthreads default to zero pydantic/pydantic#1074

Merged

mbargull mentioned this pull request Dec 4, 2019

Re-enable compilation test; try out macOS+Py3.8 multiprocessing failure conda-forge/cython-feedstock#59

Open

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rebuild for python38 #31

Rebuild for python38 #31

regro-cf-autotick-bot commented Dec 3, 2019

conda-forge-linter commented Dec 3, 2019

ocefpaf commented Dec 3, 2019

samuelcolvin commented Dec 3, 2019

ocefpaf commented Dec 3, 2019

cdeil commented Dec 3, 2019

ocefpaf commented Dec 3, 2019

regro-cf-autotick-bot commented Dec 3, 2019

mbargull commented Dec 3, 2019

cdeil commented Dec 3, 2019

djsutherland commented Dec 4, 2019

samuelcolvin commented Dec 4, 2019 •

edited

Loading

samuelcolvin commented Dec 4, 2019

samuelcolvin commented Dec 4, 2019

ocefpaf commented Dec 4, 2019

mbargull commented Dec 4, 2019 •

edited

Loading

samuelcolvin commented Dec 4, 2019

cdeil commented Dec 4, 2019

samuelcolvin commented Dec 29, 2019

samuelcolvin commented Dec 29, 2019

Rebuild for python38 #31

Rebuild for python38 #31

Conversation

regro-cf-autotick-bot commented Dec 3, 2019

conda-forge-linter commented Dec 3, 2019

ocefpaf commented Dec 3, 2019

samuelcolvin commented Dec 3, 2019

ocefpaf commented Dec 3, 2019

cdeil commented Dec 3, 2019

ocefpaf commented Dec 3, 2019

regro-cf-autotick-bot commented Dec 3, 2019

mbargull commented Dec 3, 2019

cdeil commented Dec 3, 2019

djsutherland commented Dec 4, 2019

samuelcolvin commented Dec 4, 2019 • edited Loading

samuelcolvin commented Dec 4, 2019

samuelcolvin commented Dec 4, 2019

ocefpaf commented Dec 4, 2019

mbargull commented Dec 4, 2019 • edited Loading

samuelcolvin commented Dec 4, 2019

cdeil commented Dec 4, 2019

samuelcolvin commented Dec 29, 2019

samuelcolvin commented Dec 29, 2019

samuelcolvin commented Dec 4, 2019 •

edited

Loading

mbargull commented Dec 4, 2019 •

edited

Loading