Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interface to GiNaC for Simplification #3088

Merged
merged 116 commits into from
May 9, 2024
Merged
Show file tree
Hide file tree
Changes from 55 commits
Commits
Show all changes
116 commits
Select commit Hold shift + click to select a range
8d1e68e
working on simplification contrib package
michaelbynum Nov 8, 2023
8015d7e
working on simplification contrib package
michaelbynum Nov 8, 2023
56e8ac8
working on ginac interface for simplification
michaelbynum Nov 20, 2023
932d3d6
ginac interface improvements
michaelbynum Nov 20, 2023
208a5da
simplification interface
michaelbynum Nov 20, 2023
6ef8914
bugs
michaelbynum Nov 21, 2023
af47ef7
simplification tests
michaelbynum Nov 21, 2023
475ec06
simplification tests
michaelbynum Jan 10, 2024
491db9f
update GHA to install ginac
michaelbynum Jan 10, 2024
b3a1ff9
run black
michaelbynum Jan 10, 2024
de8743a
syntax
michaelbynum Jan 10, 2024
ee9f830
install ginac in GHA
michaelbynum Jan 10, 2024
36cfd63
install ginac in GHA
michaelbynum Jan 10, 2024
8269539
install ginac in GHA
michaelbynum Jan 10, 2024
7bb0ff5
install ginac in GHA
michaelbynum Jan 10, 2024
546dad1
install ginac in GHA
michaelbynum Jan 10, 2024
37a955b
install ginac in GHA
michaelbynum Jan 10, 2024
05134ce
install ginac in GHA
michaelbynum Jan 10, 2024
1151927
install ginac in GHA
michaelbynum Jan 11, 2024
2c4fdbe
skip tests when dependencies are not available
michaelbynum Jan 11, 2024
9ebd79b
install ginac in GHA
michaelbynum Jan 11, 2024
0bd1563
update simplification tests
michaelbynum Jan 11, 2024
26007ac
run black
michaelbynum Jan 11, 2024
fc36411
add pytest marker for simplification
michaelbynum Jan 11, 2024
2007de4
Merge branch 'main' into ginac
michaelbynum Jan 11, 2024
5ba03e2
update GHA
michaelbynum Jan 11, 2024
90cdeba
update GHA
michaelbynum Jan 11, 2024
17cb11d
debugging GHA
michaelbynum Jan 11, 2024
d1fe244
test simplification with ginac and sympy
michaelbynum Jan 11, 2024
9159c3c
test simplification with ginac and sympy
michaelbynum Jan 11, 2024
457f2b3
fixing simplification tests
michaelbynum Jan 11, 2024
1e7c3f1
fixing simplification tests
michaelbynum Jan 11, 2024
b2d969f
fixing simplification tests
michaelbynum Jan 11, 2024
da2fe3d
fixing simplification tests
michaelbynum Jan 11, 2024
d37b304
Merge remote-tracking branch 'main_fork/main' into ginac
michaelbynum Jan 22, 2024
2b1596b
fixing simplification tests
michaelbynum Jan 22, 2024
d75c5f8
fixing simplification tests
michaelbynum Jan 22, 2024
dae0205
run black
michaelbynum Jan 22, 2024
313108d
fixing simplification tests
michaelbynum Jan 22, 2024
5ff4d4d
fixing simplification tests
michaelbynum Jan 22, 2024
010a997
fixing simplification tests
michaelbynum Jan 22, 2024
2c59b29
fixing simplification tests
michaelbynum Jan 22, 2024
d67d90d
fixing simplification tests
michaelbynum Jan 22, 2024
3fcec35
fixing simplification tests
michaelbynum Jan 22, 2024
b5550fe
fixing simplification tests
michaelbynum Jan 22, 2024
8984389
fixing simplification tests
michaelbynum Jan 22, 2024
125b7c7
Merge branch 'main' into ginac
michaelbynum Jan 24, 2024
92163f2
cleanup
michaelbynum Feb 21, 2024
0dff80f
cleanup
michaelbynum Feb 21, 2024
c49c2df
cleanup
michaelbynum Feb 21, 2024
2c42772
Merge remote-tracking branch 'michaelbynum/ginac' into ginac
michaelbynum Feb 21, 2024
ce308d9
Merge branch 'main' into ginac
michaelbynum Feb 21, 2024
29d6a19
cleanup
michaelbynum Feb 21, 2024
350f3d0
Merge remote-tracking branch 'main_fork/main' into ginac
michaelbynum Feb 21, 2024
9cb26c7
NFC: Fix year in copyright assertion comment
blnicho Feb 28, 2024
1e609c0
run black
michaelbynum Mar 4, 2024
e076cbf
Merge remote-tracking branch 'michaelbynum/ginac' into ginac
michaelbynum Mar 4, 2024
3dcaa1d
Merge branch 'main' into ginac
michaelbynum Mar 5, 2024
637e277
Merge branch 'main' into ginac
michaelbynum Apr 23, 2024
79f7189
Merge remote-tracking branch 'michaelbynum/ginac' into ginac
michaelbynum Apr 23, 2024
f4e989f
Add fileutils patch so find_library returns absolute path on Linux
jsiirola Apr 29, 2024
ebb4d07
bugfix: add missing import
jsiirola Apr 29, 2024
16d49e1
NFC: clarify ginac builder exception message
jsiirola Apr 29, 2024
bd3299d
Register the GiNaC interface builder with the ExtensionBuilder
jsiirola Apr 29, 2024
29b2072
Rework GiNaC interface builder (in development - testing several things)
jsiirola Apr 29, 2024
0e42033
Merge branch 'main' into ginac
michaelbynum Apr 29, 2024
1b1f944
bugfix
michaelbynum Apr 29, 2024
77fff61
Merge remote-tracking branch 'michael/ginac' into ginac
jsiirola Apr 30, 2024
74052ee
Disable local build of GiNaC
jsiirola Apr 30, 2024
6bb0f7f
NFC: apply black
jsiirola Apr 30, 2024
39643b3
remove repeated code
jsiirola Apr 30, 2024
89ace9b
Add support for download tar archives to theFileDownloader
jsiirola Apr 30, 2024
bb274ff
Switch GiNaC interface builder to use TempfileManager
jsiirola Apr 30, 2024
adaefbc
Add function for downloading and installing GiNaC and CLN
jsiirola Apr 30, 2024
c7a8f8e
Hook GiNaC builder into pyomo command
jsiirola Apr 30, 2024
a29bb3e
Remove simplification test marker
jsiirola Apr 30, 2024
c475fe7
Switching output to sys.stdout, adding debugging
jsiirola Apr 30, 2024
7053690
Support walking up the directory tree looking for ginac headers (this
jsiirola Apr 30, 2024
be69224
Fix several typos / include search logic
jsiirola Apr 30, 2024
4ac04a6
NFC: apply black
jsiirola Apr 30, 2024
abe5f8b
Resync GHA workflows, remove ginac build code
jsiirola Apr 30, 2024
54bd3d3
Move the ginac interface sources to a subdirectory
jsiirola May 5, 2024
8a37c8b
Update builder to use argparse, clean up output
jsiirola May 5, 2024
c2c63bc
Run sympy tests any time sympy is installed
jsiirola May 5, 2024
b53bbfc
Define NamedIntEnum
jsiirola May 5, 2024
e7540f2
Rework Simplifier so we can force the backend mode
jsiirola May 5, 2024
9c220b0
Improve robustness of tar filter
jsiirola May 5, 2024
e597bb5
Ensure tar file is closed
jsiirola May 5, 2024
70166ff
test get_tar_archive()
jsiirola May 5, 2024
591514d
Merge pull request #10 from jsiirola/ginac
michaelbynum May 7, 2024
4446d36
fix tests
michaelbynum May 7, 2024
90b1783
Update tar filter to handle ValueError from commonpath()
jsiirola May 7, 2024
5aae459
keep mutable parameters in sympy conversion
michaelbynum May 7, 2024
fa84fb7
Merge remote-tracking branch 'michaelbynum/ginac' into ginac
michaelbynum May 7, 2024
ae5ebd3
update defaults for mutable parameters when using sympy
michaelbynum May 7, 2024
2b3bd4e
update tests
michaelbynum May 7, 2024
82dfda1
Ensure the same output is logged on Windows and other platforms
jsiirola May 8, 2024
e132fb0
Merge remote-tracking branch 'main_fork/main' into ginac
michaelbynum May 8, 2024
3572c44
ginac cleanup
michaelbynum May 8, 2024
4b340bf
Merge remote-tracking branch 'michaelbynum/ginac' into ginac
michaelbynum May 8, 2024
a53c6f7
Add 'builders' marker for testing custom library builds (currently ju…
jsiirola May 8, 2024
8f33eed
Support multiple markers (categories) in jenkins driver
jsiirola May 8, 2024
4dc4e89
Additional (debugging) output in ginac_interface builder
jsiirola May 8, 2024
27ac6dc
Merge remote-tracking branch 'refs/remotes/michael/ginac' into ginac
jsiirola May 8, 2024
e847f10
NFC: fix typo
jsiirola May 8, 2024
8b11a1c
NFC: resyncing test_branches and test_pr_and_main
jsiirola May 8, 2024
fa2cc31
improve handling of category quotation
jsiirola May 8, 2024
bd5f10c
Improve conftest.py efficiency
jsiirola May 8, 2024
354feb2
Ensure that all unmarked tests are marked with the implicit markers
jsiirola May 8, 2024
92edcc5
NFC: apply black
jsiirola May 8, 2024
7ae7275
Prevent APPSI / GiNaC interfaces from exposing module symbols globally
jsiirola May 8, 2024
1409aa2
Fix bug in Jenkins driver
jsiirola May 9, 2024
aa75601
Fix typo in Jenkins driver
jsiirola May 9, 2024
cdaff17
Add info to the build log
jsiirola May 9, 2024
76d53de
Explicitly call out CWD when running configure
jsiirola May 9, 2024
35b71f8
Removing singletest from test_branches
jsiirola May 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/test_branches.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,14 @@ jobs:
PYENV: conda
PACKAGES: mpi4py

- os: ubuntu-latest
python: 3.11
other: /singletest
category: "-m 'neos or importtest'"
skip_doctest: 1
TARGET: linux
PYENV: pip

- os: ubuntu-latest
python: '3.10'
other: /cython
Expand Down Expand Up @@ -149,6 +157,24 @@ jobs:
# path: cache/os
# key: pkg-${{env.CACHE_VER}}.0-${{runner.os}}

- name: install GiNaC
if: matrix.other == '/singletest'
run: |
cd ..
curl https://www.ginac.de/CLN/cln-1.3.7.tar.bz2 >cln-1.3.7.tar.bz2
tar -xvf cln-1.3.7.tar.bz2
cd cln-1.3.7
./configure
make -j 2
sudo make install
cd ..
curl https://www.ginac.de/ginac-1.8.7.tar.bz2 >ginac-1.8.7.tar.bz2
tar -xvf ginac-1.8.7.tar.bz2
cd ginac-1.8.7
./configure
make -j 2
sudo make install

- name: TPL package download cache
uses: actions/cache@v4
if: ${{ ! matrix.slim }}
Expand Down Expand Up @@ -608,13 +634,26 @@ jobs:
echo ""
pyomo build-extensions --parallel 2

- name: Install GiNaC Interface
if: matrix.other == '/singletest'
run: |
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
echo "LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH" >> $GITHUB_ENV
cd pyomo/contrib/simplification/
$PYTHON_EXE build.py --inplace

- name: Report pyomo plugin information
run: |
echo "$PATH"
pyomo help --solvers || exit 1
pyomo help --transformations || exit 1
pyomo help --writers || exit 1

- name: Run Simplification Tests
if: matrix.other == '/singletest'
run: |
pytest -v -m 'simplification' pyomo/contrib/simplification/tests/test_simplification.py --junitxml="TEST-pyomo-simplify.xml"

- name: Run Pyomo tests
if: matrix.mpi == 0
run: |
Expand Down
31 changes: 31 additions & 0 deletions .github/workflows/test_pr_and_main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,24 @@ jobs:
# path: cache/os
# key: pkg-${{env.CACHE_VER}}.0-${{runner.os}}

- name: install GiNaC
if: matrix.other == '/singletest'
run: |
cd ..
curl https://www.ginac.de/CLN/cln-1.3.7.tar.bz2 >cln-1.3.7.tar.bz2
tar -xvf cln-1.3.7.tar.bz2
cd cln-1.3.7
./configure
make -j 2
sudo make install
cd ..
curl https://www.ginac.de/ginac-1.8.7.tar.bz2 >ginac-1.8.7.tar.bz2
tar -xvf ginac-1.8.7.tar.bz2
cd ginac-1.8.7
./configure
make -j 2
sudo make install

- name: TPL package download cache
uses: actions/cache@v4
if: ${{ ! matrix.slim }}
Expand Down Expand Up @@ -638,13 +656,26 @@ jobs:
echo ""
pyomo build-extensions --parallel 2

- name: Install GiNaC Interface
if: matrix.other == '/singletest'
run: |
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
echo "LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH" >> $GITHUB_ENV
cd pyomo/contrib/simplification/
$PYTHON_EXE build.py --inplace

- name: Report pyomo plugin information
run: |
echo "$PATH"
pyomo help --solvers || exit 1
pyomo help --transformations || exit 1
pyomo help --writers || exit 1

- name: Run Simplification Tests
if: matrix.other == '/singletest'
run: |
pytest -v -m 'simplification' pyomo/contrib/simplification/tests/test_simplification.py --junitxml="TEST-pyomo-simplify.xml"

- name: Run Pyomo tests
if: matrix.mpi == 0
run: |
Expand Down
12 changes: 12 additions & 0 deletions pyomo/contrib/simplification/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# ___________________________________________________________________________
#
# Pyomo: Python Optimization Modeling Objects
# Copyright (c) 2008-2024
# National Technology and Engineering Solutions of Sandia, LLC
# Under the terms of Contract DE-NA0003525 with National Technology and
# Engineering Solutions of Sandia, LLC, the U.S. Government retains certain
# rights in this software.
# This software is distributed under the 3-clause BSD License.
# ___________________________________________________________________________

from .simplify import Simplifier
michaelbynum marked this conversation as resolved.
Show resolved Hide resolved
108 changes: 108 additions & 0 deletions pyomo/contrib/simplification/build.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# ___________________________________________________________________________
#
# Pyomo: Python Optimization Modeling Objects
# Copyright (c) 2008-2024
# National Technology and Engineering Solutions of Sandia, LLC
# Under the terms of Contract DE-NA0003525 with National Technology and
# Engineering Solutions of Sandia, LLC, the U.S. Government retains certain
# rights in this software.
# This software is distributed under the 3-clause BSD License.
# ___________________________________________________________________________

import glob
import os
import shutil
import sys
import tempfile
from distutils.dist import Distribution

from pybind11.setup_helpers import Pybind11Extension, build_ext
from pyomo.common.envvar import PYOMO_CONFIG_DIR
from pyomo.common.fileutils import find_library, this_file_dir


def build_ginac_interface(args=None):
if args is None:
args = list()
dname = this_file_dir()
_sources = ['ginac_interface.cpp']
sources = list()
for fname in _sources:
sources.append(os.path.join(dname, fname))

ginac_lib = find_library('ginac')
if ginac_lib is None:
raise RuntimeError(
'could not find GiNaC library; please make sure it is in the LD_LIBRARY_PATH environment variable'
)
ginac_lib_dir = os.path.dirname(ginac_lib)
ginac_build_dir = os.path.dirname(ginac_lib_dir)
ginac_include_dir = os.path.join(ginac_build_dir, 'include')
if not os.path.exists(os.path.join(ginac_include_dir, 'ginac', 'ginac.h')):
raise RuntimeError('could not find GiNaC include directory')

cln_lib = find_library('cln')
if cln_lib is None:
raise RuntimeError(
'could not find CLN library; please make sure it is in the LD_LIBRARY_PATH environment variable'
)
cln_lib_dir = os.path.dirname(cln_lib)
cln_build_dir = os.path.dirname(cln_lib_dir)
cln_include_dir = os.path.join(cln_build_dir, 'include')
if not os.path.exists(os.path.join(cln_include_dir, 'cln', 'cln.h')):
raise RuntimeError('could not find CLN include directory')

extra_args = ['-std=c++11']
ext = Pybind11Extension(
'ginac_interface',
sources=sources,
language='c++',
include_dirs=[cln_include_dir, ginac_include_dir],
library_dirs=[cln_lib_dir, ginac_lib_dir],
libraries=['cln', 'ginac'],
extra_compile_args=extra_args,
)

class ginacBuildExt(build_ext):
def run(self):
basedir = os.path.abspath(os.path.curdir)
if self.inplace:
tmpdir = this_file_dir()
else:
tmpdir = os.path.abspath(tempfile.mkdtemp())
print("Building in '%s'" % tmpdir)
os.chdir(tmpdir)
try:
super(ginacBuildExt, self).run()
if not self.inplace:
library = glob.glob("build/*/ginac_interface.*")[0]
target = os.path.join(
PYOMO_CONFIG_DIR,
'lib',
'python%s.%s' % sys.version_info[:2],
'site-packages',
'.',
)
if not os.path.exists(target):
os.makedirs(target)
shutil.copy(library, target)
finally:
os.chdir(basedir)
if not self.inplace:
shutil.rmtree(tmpdir, onerror=handleReadonly)

package_config = {
'name': 'ginac_interface',
'packages': [],
'ext_modules': [ext],
'cmdclass': {"build_ext": ginacBuildExt},
}

dist = Distribution(package_config)
dist.script_args = ['build_ext'] + args
dist.parse_command_line()
dist.run_command('build_ext')


if __name__ == '__main__':
build_ginac_interface(sys.argv[1:])
Loading