Windows support and Ipopt 3.12.13 #34

richardotis · 2019-12-23T17:57:52Z

Checklist

Used a fork of the feedstock to propose changes
Reset the build number to 0 (if the version changed)
Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
Ensured the license file is being packaged.

Closes #1

This PR is an attempt to get Windows support working for ipopt using the m2w64 toolchain and cmake. The cmake scripts are based on the awesome efforts of @ycollet to ship CMake scripts for the whole Coin-OR ecosystem: https://github.com/ycollet/coinor-cmake
The scripts here are lightly modified to address some path differences, and for stability across future versions of Ipopt.

Do we need to switch all three platforms to CMake here, or is it okay to only use CMake for Windows?
I'm using m2w64-toolchain here, even though it's deprecated, because the compiler('m2w64_c') syntax didn't work locally when I ran conda build recipe -m .ci_support\win_c_compilervs2015cxx_compilervs2015.yaml. Help would be appreciated here.
The CMake scripts from @ycollet are LGPL licensed (for which I don't really understand the implications in the context of build scripts). I think that accepting this PR would mean this feedstock (but not the actual package) would become LGPL licensed. Someone from the @conda-forge/core team should probably chime in here.
There are likely a bunch of unnecessary CMake support macros here that can be deleted.
The requirements in the recipe are not fully optimized, and I am not 100% clear on the differences between build, host and run requirements (I read the documentation and still don't quite get it).
Ipopt has a bunch of build-time options, including for AMPL, HSL, Metis, performance options like fastmath and MKL, as well as minor(?) ones like link-time optimization. Which of these should be exposed as features in the recipe?
There is a lot of path- and link-hacking in the bld.bat to get the compiler to find and link all the needed libraries. I am not sure how portable it is across Windows version, nor am I clear on why CMake is not doing this automatically (it's there in the CMake scripts).

conda-forge-linter · 2019-12-23T17:59:27Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

chrisdembia · 2019-12-23T18:05:32Z

Neat! Where can I learn more about the m2w64 toolchain? Which compilers are used? Will this build of Ipopt be usable with Visual Studio?

richardotis · 2019-12-23T18:14:34Z

@chrisdembia I don't know the best place; everything I've learned about m2w64 has been random conda-forge feedstock issue trackers. There is some discussion here: ContinuumIO/anaconda-issues#1484 The toolchain package itself is deprecated but the technology has been rolled into conda-forge's official compiler support, so I think it's still okay to use (but it has a new name).

MSVC does not appear to compile Ipopt (I tried that first), and it is not officially supported upstream. The compiler stack used here is essentially MinGW.

chrisdembia · 2019-12-23T18:17:21Z

Okay thank you. I'm hoping to use Ipopt in a C++ project compiled with Visual Studio, and my understanding is I cannot use libraries compiled with MinGW.

I think Ipopt can be compiled with Visual Studio; it's just that its MUMPS dependency is written in Fortran and needs to be compiled with gfortran or another open-source Fortran compiler.

Anyway, I was just curious. You can ignore my comments.

richardotis · 2019-12-23T19:49:42Z

@conda-forge-admin, please rerender

…nda-forge-pinning 2019.12.18

richardotis · 2019-12-23T20:07:42Z

@conda-forge/core It looks to me like something weird is happening in the Linux build infrastructure. Configure seems to be pulling in the link flag -lm' (note the extra quote character). Is that normal? If not, it would explain the strange linking to Fortran libraries from C fails message we're getting here.

richardotis · 2019-12-23T20:24:14Z

I can reproduce the Linux error locally with conda build recipe -m .ci_support/linux_.yaml

richardotis · 2019-12-23T20:41:54Z

Here's an excerpt from the config.log:

configure:2027: $? = 0
configure:2029: /home/rotis/anaconda3/conda-bld/ipopt_1577132049035/_build_env/bin/x86_64-conda_cos6-linux-gnu-cc -v </dev/null >&5
Reading specs from /home/rotis/anaconda3/conda-bld/ipopt_1577132049035/_build_env/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/specs
COLLECT_GCC=/home/rotis/anaconda3/conda-bld/ipopt_1577132049035/_build_env/bin/x86_64-conda_cos6-linux-gnu-cc
COLLECT_LTO_WRAPPER=/home/rotis/anaconda3/conda-bld/ipopt_1577132049035/_build_env/bin/../libexec/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/lto-wrapper
Target: x86_64-conda_cos6-linux-gnu
Configured with: /home/conda/feedstock_root/build_artifacts/ctng-compilers_1570984655235/work/.build/x86_64-conda_cos6-linux-gnu/src/gcc/configure --build=x86_64-build_pc-linux-gnu --host=x86_64-build_pc-linux-gnu --target=x86_64-conda_cos6-linux-gnu --prefix=/home/conda/feedstock_root/build_artifacts/ctng-compilers_1570984655235/work/gcc_built --with-sysroot=/home/conda/feedstock_root/build_artifacts/ctng-compilers_1570984655235/work/gcc_built/x86_64-conda_cos6-linux-gnu/sysroot --enable-languages=c,c++,fortran,objc,obj-c++ --with-pkgversion='crosstool-NG 1.23.0.452-d158' --enable-__cxa_atexit --disable-libmudflap --enable-libgomp --disable-libssp --enable-libquadmath --enable-libquadmath-support --enable-libsanitizer --enable-libmpx --with-gmp=/home/conda/feedstock_root/build_artifacts/ctng-compilers_1570984655235/work/.build/x86_64-conda_cos6-linux-gnu/buildtools --with-mpfr=/home/conda/feedstock_root/build_artifacts/ctng-compilers_1570984655235/work/.build/x86_64-conda_cos6-linux-gnu/buildtools --with-mpc=/home/conda/feedstock_root/build_artifacts/ctng-compilers_1570984655235/work/.build/x86_64-conda_cos6-linux-gnu/buildtools --with-isl=/home/conda/feedstock_root/build_artifacts/ctng-compilers_1570984655235/work/.build/x86_64-conda_cos6-linux-gnu/buildtools --enable-lto --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --enable-threads=posix --enable-target-optspace --enable-plugin --enable-gold --disable-nls --disable-multilib --with-local-prefix=/home/conda/feedstock_root/build_artifacts/ctng-compilers_1570984655235/work/gcc_built/x86_64-conda_cos6-linux-gnu/sysroot --enable-long-long --enable-default-pie
Thread model: posix
gcc version 7.3.0 (crosstool-NG 1.23.0.452-d158)

Notice we're getting back --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm'

This looks like a bad interaction between the gcc output and Ipopt's configure script. This is a known upstream bug for a long time: https://projects.coin-or.org/Ipopt/ticket/215 - (So how did the Linux build ever work...?)

…ifying FLIBS

richardotis · 2019-12-23T21:17:58Z

This workaround commit was just a test to see if the Linux build will work if you hardcode FLIBS. The real solution (which should probably become its own PR) will be to patch the upstream BuildTools/coin.m4 to pass a correctly formed FLIBS. We're on the hook to do this because the bug is marked wontfix upstream, and the next version of Ipopt actually has a build system overhaul, so any upstream patch would probably be moot.

I'd like to get functioning Ipopt 3.12.x for all platforms before dealing with 3.13, though.

conda-forge-linter · 2019-12-31T16:06:24Z

Hi! This is the friendly automated conda-forge-linting service.

I was trying to look for recipes to lint for you, but it appears we have a merge conflict.
Please try to merge or rebase with the base branch to resolve this conflict.

Please ping the 'conda-forge/core' team (using the @ notation in a comment) if you believe this is a bug.

conda-forge-linter · 2019-12-31T16:17:18Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

richardotis · 2020-01-06T23:38:02Z

@conda-forge-admin, please rerender

moorepants · 2020-01-11T23:48:35Z

The CMake scripts from @ycollet are LGPL licensed (for which I don't really understand the implications in the context of build scripts). I think that accepting this PR would mean this feedstock (but not the actual package) would become LGPL licensed. Someone from the @conda-forge/core team should probably chime in here.

I'm not sure if we are supposed to license the build scripts anything other than the default. But I am sure that other feedstocks likely contain patches that are taken from non-BSD-3 licensed code.

moorepants · 2020-01-11T23:51:30Z

recipe/build.sh

@@ -8,6 +8,7 @@ mkdir build
 cd build

 ../configure \
+  FLIBS="-L$PREFIX/lib -L$BUILD_PREFIX/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0 -L$BUILD_PREFIX/bin/../lib/gcc -L$BUILD_PREFIX/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/lib/../lib -L$BUILD_PREFIX/bin/../x86_64-conda_cos6-linux-gnu/sysroot/lib/../lib -L$BUILD_PREFIX/bin/../x86_64-conda_cos6-linux-gnu/sysroot/usr/lib/../lib -L$BUILD_PREFIX/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/lib -L$BUILD_PREFIX/bin/../x86_64-conda_cos6-linux-gnu/sysroot/lib -L$BUILD_PREFIX/bin/../x86_64-conda_cos6-linux-gnu/sysroot/usr/lib -lrt -lgfortran -lm -lquadmath" \


Is this change necessary? Seems like these should be discovered rather than specified explicitly.

That shouldn't be here and looks like a rebase error on my part.

moorepants · 2020-01-27T20:53:39Z

@isuruf Would you be willing to answer our questions here and double check this windows build?

isuruf · 2020-01-27T20:59:53Z

Is this mingw approach used in other conda-forge packages and do the primary conda-forge folks approve of this method?

Only rarely. Most packages use MSVC.

Will downstream packages have to deal with any of the mingw toolchain and/or are there build/install consequences for those packages?

The C runtime that MinGW and MSVC are using are slightly different. Therefore, there might be incompatibilities. (segfaults when you try to free an object created in a different C runtime)
So, downstream packages will have to be built with MinGW too.

moorepants · 2020-01-27T21:02:20Z

Thanks! There is also the list of questions in the very first comment that we don't have answers to.

isuruf · 2020-01-27T21:02:20Z

Also, the C++ runtimes are completely different.

isuruf · 2020-01-27T21:06:24Z

No. Using CMake just for Windows is fine and preferred to keep backwards compatibility.
You need to rerender after using the compiler('m2w64_c') syntax
That's fine. You can keep the license of the scripts in the recipe.
Those are probably bugs in the cmake scripts.

isuruf · 2020-01-27T21:08:26Z

Since there's CMake support, why not try VS2015?

richardotis · 2020-01-27T22:56:22Z

@isuruf One part of the ipopt build process requires a working Fortran compiler, at least for the 3.12.x versions. Also, when I naively tried changing the build script to create CMake VS Makefiles, I received errors unrelated to a missing Fortran compiler. Generally speaking, upstream seemed to recommend the MSYS/MinGW option for building: https://projects.coin-or.org/Ipopt/browser/stable/3.12/Ipopt/doc/documentation.pdf?format=raw The story might have changed for 3.13, since they rewrote the build system, but I haven't looked at it yet.

I'm not saying it's impossible, but it's unclear how much work it would be.

moorepants · 2020-01-27T23:03:39Z

We are are using flang in th slycot feedstock with cmake:

https://github.com/conda-forge/slycot-feedstock

Maybe that's possible?

richardotis · 2020-02-17T19:26:53Z

These new commits for a MSVC-compatible build depend on conda-forge/mumps-feedstock#56

Even with that PR, locally I get some test failures for the built-in examples. However, the majority of them pass and I am not sure if some numerical instability is to be expected here.

…da-forge-pinning 2020.02.12

nniclausse · 2020-03-10T08:54:45Z

Hi, i tested the 3.12.13 package from the pycalphad channel, it does not include the file IpoptConfig.h which is needed by IpDebug.hpp, IpJournalist.hpp and IpTypes.hpp

richardotis · 2020-03-17T16:02:09Z

@nniclausse Are those files needed by downstream packages, or only at build time?

nniclausse · 2020-03-17T16:19:01Z

@nniclausse Are those files needed by downstream packages, or only at build time?

I'm trying to build an app that use ipopt, this app includes IpTNLP.hpp, which requires IpUtils.hpp -> IpTypes.hpp -> IpoptConfig.h -> config_ipopt_default.h

richardotis · 2020-03-17T16:23:41Z

Ah ok, I think the reason I didn't catch it is because I was testing against the C interface without making sure C++ still worked.

traversaro · 2020-10-09T16:50:25Z

Hi @richardotis, I am interesting in helping to get this merged, any suggestion on what are the next steps that need to be addressed? Thanks!

traversaro · 2020-11-15T11:21:23Z

I guess we can close this PR as it has been superseded by #47 @conda-forge/ipopt .

richardotis added 3 commits December 23, 2019 09:29

MNT: Re-render with conda-smithy 3.6.5

2c73854

ENH: Ipopt 3.12.13 and Windows support

1f77787

MNT: Reset build number for new version to 0

764f480

richardotis requested review from bluescarni and pstjohn as code owners December 23, 2019 17:57

richardotis mentioned this pull request Dec 23, 2019

Windows support via msys2 #18

Closed

richardotis added 6 commits December 23, 2019 10:21

FIX: Force non-detection of sh.exe in path

8410628

FIX: meta.yaml: Remove unnecessary liblapack deps

0281f4f

FIX: meta.yaml: Incorrect fortran compiler dep for Windows

34052dd

FIX: Include and library path adjustments

d36b9cc

FIX: More fixes for Windows compiler deps

4a3f488

FIX: Try to fix paths in CMakeLists

ab504fb

MNT: Re-rendered with conda-build 3.18.11, conda-smithy 3.6.5, and co…

9f4760c

…nda-forge-pinning 2019.12.18

FIX: Workaround for configure script parsing failure by manually spec…

4a0c7cf

…ifying FLIBS

richardotis added 3 commits December 31, 2019 07:45

FIX: Explicitly blow away FLIBS to avoid upstream configure script bug

c610bb2

bld.bat: Cleanup obsolete comments

4a5e352

CMakeLists_Ipopt: Fix library name to libipopt

26fc150

Merge remote-tracking branch 'upstream/master'

2c2a302

moorepants reviewed Jan 11, 2020

View reviewed changes

richardotis mentioned this pull request Feb 17, 2020

MSVC/Flang support (Windows) conda-forge/mumps-feedstock#56

Merged

4 tasks

richardotis added 2 commits February 17, 2020 11:16

ENH: MSVC/Flang support with tests

7908d2e

Temporarily turn off test during build

07f2d45

richardotis added 2 commits February 17, 2020 11:27

MNT: Re-rendered with conda-build 3.18.9, conda-smithy 3.6.8, and con…

1ab9e00

…da-forge-pinning 2020.02.12

BLD: meta.yaml: Skip building MS VC<14

c8765e3

richardotis mentioned this pull request Feb 17, 2020

Windows support with MSVC conda-forge/cyipopt-feedstock#16

Closed

4 tasks

richardotis added 3 commits February 17, 2020 17:09

Delete unnecessary CMake modules

6d00723

CMakeLists: Reduce non-Windows cruft

6bb24cb

Workaround CMake bug to allow MSVC to compile

02d2db0

bocklund mentioned this pull request Mar 10, 2020

Add scheil recipe conda-forge/staged-recipes#10673

Merged

8 tasks

traversaro mentioned this pull request Oct 30, 2020

Add support for Windows/MSVC #47

Merged

5 tasks

jschueller closed this Nov 15, 2020

traversaro mentioned this pull request Nov 17, 2020

Update naming convention on DLLs for windows to match COIN-OR #53

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Windows support and Ipopt 3.12.13 #34

Windows support and Ipopt 3.12.13 #34

richardotis commented Dec 23, 2019

conda-forge-linter commented Dec 23, 2019

chrisdembia commented Dec 23, 2019

richardotis commented Dec 23, 2019

chrisdembia commented Dec 23, 2019

richardotis commented Dec 23, 2019

richardotis commented Dec 23, 2019

richardotis commented Dec 23, 2019

richardotis commented Dec 23, 2019 •

edited

Loading

richardotis commented Dec 23, 2019

conda-forge-linter commented Dec 31, 2019

conda-forge-linter commented Dec 31, 2019

richardotis commented Jan 6, 2020

moorepants commented Jan 11, 2020

moorepants Jan 11, 2020

richardotis Jan 12, 2020

moorepants commented Jan 27, 2020

isuruf commented Jan 27, 2020

moorepants commented Jan 27, 2020

isuruf commented Jan 27, 2020

isuruf commented Jan 27, 2020

isuruf commented Jan 27, 2020

richardotis commented Jan 27, 2020

moorepants commented Jan 27, 2020

richardotis commented Feb 17, 2020

nniclausse commented Mar 10, 2020

richardotis commented Mar 17, 2020

nniclausse commented Mar 17, 2020

richardotis commented Mar 17, 2020

traversaro commented Oct 9, 2020

traversaro commented Nov 15, 2020

Windows support and Ipopt 3.12.13 #34

Windows support and Ipopt 3.12.13 #34

Conversation

richardotis commented Dec 23, 2019

conda-forge-linter commented Dec 23, 2019

chrisdembia commented Dec 23, 2019

richardotis commented Dec 23, 2019

chrisdembia commented Dec 23, 2019

richardotis commented Dec 23, 2019

richardotis commented Dec 23, 2019

richardotis commented Dec 23, 2019

richardotis commented Dec 23, 2019 • edited Loading

richardotis commented Dec 23, 2019

conda-forge-linter commented Dec 31, 2019

conda-forge-linter commented Dec 31, 2019

richardotis commented Jan 6, 2020

moorepants commented Jan 11, 2020

moorepants Jan 11, 2020

Choose a reason for hiding this comment

richardotis Jan 12, 2020

Choose a reason for hiding this comment

moorepants commented Jan 27, 2020

isuruf commented Jan 27, 2020

moorepants commented Jan 27, 2020

isuruf commented Jan 27, 2020

isuruf commented Jan 27, 2020

isuruf commented Jan 27, 2020

richardotis commented Jan 27, 2020

moorepants commented Jan 27, 2020

richardotis commented Feb 17, 2020

nniclausse commented Mar 10, 2020

richardotis commented Mar 17, 2020

nniclausse commented Mar 17, 2020

richardotis commented Mar 17, 2020

traversaro commented Oct 9, 2020

traversaro commented Nov 15, 2020

richardotis commented Dec 23, 2019 •

edited

Loading