Implementing SNOPT trial version in DAFoam

[Ananth-Narayan-IITM]

Hi people,

I wish to work on Topology optimization, example cases- Channel_TopoCHT/ but we need SNOPT instead of default optimizer IPOPT. I tried couple of methods to get this done, but cannot generate the wrapper library using request files from UCSD Optimization Software and got these files

./libsnopt7.a
./libsnopt7.so
./snopt_cwrap.h
./snopt.h
./sqopt_cwrap.h
./sqopt.h
./snopt.lic

I am quite unsure how to get these things working for running Channel_TopoCHT/ tutorial case.

I have explained the steps I followed under #520 (comment) and thought of creating detailed discussion to those who needs SNOPT.

[krotname]

For the Channel_TopoCHT case the DAFoam side is the easy part: the run script is already written around om.pyOptSparseDriver(), and the DAFoam docs list -optimizer SNOPT as one of the supported choices in those tutorial scripts. So once pyOptSparse can import SNOPT, you should be able to run the case with the same script plus -optimizer SNOPT.

The hard part is that the files you listed look like the prebuilt/library distribution, not the SNOPT source layout that pyOptSparse's normal source build expects. The current pyOptSparse SNOPT docs say the source-build path expects the SNOPT zip's src folder files copied into pyoptsparse/pySNOPT/source, excluding snopth.f, and only SNOPT 7.7.x is officially supported there.

If you installed pyOptSparse from conda, the SNOPT module is not included because of licensing. In that case pyOptSparse expects you to build the SNOPT Python binding separately and point it at that directory with:

export PYOPTSPARSE_IMPORT_SNOPT_FROM=/path/to/snopt-bind

For the library-only route, the SNOPT FAQ also says pyOptSparse needs a modified pyoptsparse/pySNOPT/setup.py, the library path set in that setup, LD_LIBRARY_PATH pointing at the SNOPT library directory on Linux, and the SNOPT license configured.

A quick sanity check before going back to the DAFoam case is:

python - <<'PY'
from pyoptsparse import SNOPT
print(SNOPT)
PY

If that import fails, Channel_TopoCHT will fail too, because DAFoam/OpenMDAO only see whatever optimizers pyOptSparse exposes. If it succeeds, then switch the tutorial run to SNOPT, e.g. the same runScript.py flow with -optimizer SNOPT.

Refs:

• DAFoam runScript optimizer section: https://dafoam.github.io/get-started-runscript.html
• pyOptSparse SNOPT install notes: https://mdolab-pyoptsparse.readthedocs-hosted.com/en/latest/optimizers/SNOPT.html
• SNOPT FAQ for prebuilt libraries with pyOptSparse: https://ccom.ucsd.edu/~optimizers/docs/snopt/faqs.html#other-issues

[Ananth-Narayan-IITM]

I have the DAFoam v5.0 from Docker, so there everything was precompiled. As pointed getting the proper files is hardest part.

For the library-only route, the SNOPT FAQ also says pyOptSparse needs a modified pyoptsparse/pySNOPT/setup.py
Have you tried modifying setup.py, I cannot see setup.py in the directory mentioned

If you have used SNOPT optimizer, can you explain from where to get those files.

Thank you

[krotname]

Correction to my previous note: I rechecked current pyOptSparse locally and pyoptsparse/pySNOPT/setup.py is not present in the current source tree, so do not look for that file in the DAFoam Docker image or in a current pyOptSparse checkout. Current pyOptSparse builds through Meson.

I have not used the licensed SNOPT files myself, so I cannot point to a public download or share them. SNOPT is commercial/licensed; the source or library package has to come from the SNOPT vendor/license holder, or from your institution if it already has a license.

For the current documented source-build path, the expected layout is:

1. get the licensed SNOPT source distribution that contains a src directory;
2. copy the SNOPT src files into pyoptsparse/pySNOPT/source, excluding snopth.f;
3. build/install pyOptSparse from source;
4. verify before DAFoam with:

python - <<'PY'
from pyoptsparse import SNOPT
print(SNOPT)
PY

If you installed pyOptSparse from conda/prebuilt packages, SNOPT is not included because of licensing. For that route the pyOptSparse docs describe building a compatible snopt Python extension separately and setting PYOPTSPARSE_IMPORT_SNOPT_FROM=/path/to/snopt-bind.

If your trial package is library-only and not source, the current pyOptSparse docs do not give a simple setup.py edit path anymore. I would ask pyOptSparse/SNOPT support with the exact files you received, because the source-copy instructions may not apply to that package.

Refs:

pyOptSparse SNOPT install notes: https://mdolab-pyoptsparse.readthedocs-hosted.com/en/latest/optimizers/SNOPT.html
pyOptSparse README licensing note: https://github.com/mdolab/pyoptsparse#optimizers

[Ananth-Narayan-IITM]

I think I found the solution that works for me (#998 (comment)), as of now I didn't get any problem in using SNOPT. Thanks for the code to verify if SNOPT is properly installed.

[Ananth-Narayan-IITM]

I think I have found the solution and is working for me. For those in future, try these steps and let me know in case of any issues.

• Installing pySNOPT in docker

  • Open the base docker image as, (don't use --rm, not sure if this will affect this installation)

    docker run -it -u dafoamuser --mount "type=bind,src=$(pwd),target=/home/dafoamuser/mount" -w /home/dafoamuser/mount dafoam/opt-packages:v5.0.0 bash

  • Get pyOptSparse v2.10.1 and extract them as

    cd ~/dafoam/repos
    wget https://github.com/mdolab/pyoptsparse/archive/refs/tags/v2.10.1.tar.gz
    tar -xzf v2.10.1.tar.gz
    rm -rf v2.10.1.tar.gz

  • Clone build_pyoptsparse in ~/dafoam/repo

    git clone https://github.com/OpenMDAO/build_pyoptsparse.git
    python -m pip install ./build_pyoptsparse
    python -m build_pyoptsparse.snopt_module --help

    python -m build_pyoptsparse.snopt_module --help should give output, means repository is installed
  • Meanwhile request the 3 months Trial version, where these files will be provided. These will be located under libsnopt7/

    ./libsnopt7.a
    ./libsnopt7.so
    ./snopt_cwrap.h
    ./snopt.h
    ./sqopt_cwrap.h
    ./sqopt.h
    ./snopt.lic

  • Now copy the libsnopt7/ into ~/dafoam/repos/pyoptsparse-2.10.1/pyoptsparse/pySNOPT/source. Make sure to have (from ccom.ucsd.edu) folder libsnopt7/ in /mount/

    cp -r ~/mount/libsnopt7/ .

  • Compile the libsnopt7.so as

    python -m build_pyoptsparse.snopt_module --snopt-lib libsnopt7/libsnopt7.so

  • Verify that ls ~/dafoam/packages/miniconda3/lib/python3.10/site-packages/pyoptsparse/pySNOPT/ has snopt.cpython-310-x86_64-linux-gnu.so
  • Run these commands to compile the helper functions,

    gfortran -fPIC -c openunit.f
    gfortran -fPIC -c closeunit.f
    
    gfortran -shared \
    -o libsnopt_helpers.so \
    openunit.o \
    closeunit.o

  • Upon nm -D libsnopt_helpers.so | grep unit you should see T openunit_, T closeunit_
  • Now export the LD_PRELOAD and license path also make sure to write this in ~/.bashrc and source ~/.bashrc

    export LD_PRELOAD=~/dafoam/repos/pyoptsparse-2.10.1/pyoptsparse/pySNOPT/source/libsnopt_helpers.so
    export SNOPT_LICENSE=~/dafoam/repos/pyoptsparse-2.10.1/pyoptsparse/pySNOPT/source/libsnopt7/snopt7.lic
    export SNOPT_LICENSE_FILE=$SNOPT_LICENSE

  • To verify if everything is correct, run these two commands

    python - <<'PY'
    import pyoptsparse.pySNOPT.snopt as snopt
    print(snopt.sntitle()) # Output: b'S N O P T  7.7.7    (Feb 2021)'
    PY
    
    python - <<'PY'
    from pyoptsparse import OPT
    opt = OPT('SNOPT')
    print(opt) # Output: <pyoptsparse.pySNOPT.pySNOPT.SNOPT object at 0x7fbea7e7bc10>
    
    PY

  • Commit the docker container in different terminal as

    docker commit <container_id> dafoam-snopt:SNOPT_working

    You can find container id, inside the current docker container as, dafoamuser@<container_id>
  • Open docker run -it -u dafoamuser --mount "type=bind,src=$(pwd),target=/home/dafoamuser/mount" -w /home/dafoamuser/mount dafoam-snopt:SNOPT_working bash in different terminal and verify Channel_TopoCHT/ tutorial is working.

Thank you
Ananth Narayan

[friedenhe]

@Ananth-Narayan-IITM Thanks for sharing!!