Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Link Python module to Cantera shared library #1429

Merged
merged 28 commits into from Feb 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
d3d5d62
Move factory function definitions out of .h files
speth Jan 24, 2023
a31d1f3
Fix incorrect virtual specifier on FalloffRate::evalFromStruct
speth Jan 24, 2023
4e6ad9d
Move some function definitions out of header files
speth Jan 26, 2023
9341f34
Fix compilation failures with PCH and Eigen
speth Jan 29, 2023
523be4e
Add "using" directives for common types
speth Jan 26, 2023
caed5c0
[SCons] Automatically export symbols for DLL
speth Jan 28, 2023
35d4cf2
Eliminate obsolete CANTERA_USE_INTERNAL
speth Jan 28, 2023
fb5948f
[SCons] Compile using C++17 standard
speth Jan 29, 2023
783129b
Introduce cantera_python shim and load only when needed
speth Jan 27, 2023
09db742
[SCons] Link Python module to Cantera shared library
speth Jan 26, 2023
8748dc0
Handle case where extension manager comes from existing Python module
speth Jan 27, 2023
13da7c4
Bump vendored fmt library to version 9.1.0
speth Jan 28, 2023
bcd1a90
[SCons] Link tests and samples to shared library on Windows
speth Jan 28, 2023
196d2a0
Bump vendored Eigen to version 3.4.0
speth Jan 28, 2023
ffcd3e4
[SCons] Enforce minimum MACOS_DEPLOYMENT_TARGET
speth Jan 29, 2023
71c85e3
[SCons] Restore functionality of buildutils.which
speth Jan 29, 2023
b61e957
[SCons] Link to shared standard libraries with MinGW
speth Jan 29, 2023
a6ae255
[CI] Upload wheel from MinGW to enable debugging
speth Jan 30, 2023
fd80340
[CI] Disable mucking around with MinGW libraries
speth Jan 30, 2023
1d7d4e0
[CI] Upload MinGW libraries and gtest binaries for debugging
speth Jan 30, 2023
9194a61
[CI] Use newer Python/Windows for MinGW builds
speth Jan 30, 2023
7688562
[CI] Show timing info for brew install commands
speth Jan 31, 2023
3c5881e
Fix export of registration function when using MinGW
speth Jan 31, 2023
5f4eb5a
Add function to distinguish shared vs static linking at runtime
speth Jan 31, 2023
d6f1a79
Check for match between shared lib and Python module
speth Feb 1, 2023
f0d6c9f
[SCons] Consolidate addition of build/lib to search path
speth Feb 1, 2023
3ec02f2
Handle multiple versions of Python extensions
speth Feb 1, 2023
3dd702d
[SCons] Trim set of symbols exported in cantera_shared.dll
speth Feb 2, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
41 changes: 29 additions & 12 deletions .github/workflows/main.yml
Expand Up @@ -133,14 +133,14 @@ jobs:
python-version: 3.11
if: matrix.python-version == '3.11'
- name: Install Brew dependencies
run: brew install boost libomp hdf5
run: brew install --display-times boost libomp hdf5
- name: Setup Homebrew Python
# This path should work for future Python versions as well
if: matrix.python-version != '3.11'
run: |
brew install python@${{ matrix.python-version }}
brew install --display-times python@${{ matrix.python-version }}
brew link --force --overwrite python@${{ matrix.python-version }}
brew install scons
brew install --display-times scons
- name: Upgrade pip
run: $PYTHON_CMD -m pip install -U pip 'setuptools>=47.0.0,<48' wheel
- name: Install Python dependencies
Expand Down Expand Up @@ -384,6 +384,7 @@ jobs:
- name: Run the examples
# See https://unix.stackexchange.com/a/392973 for an explanation of the -exec part
run: |
export LD_LIBRARY_PATH=build/lib
find samples/python -type f -iname "*.py" \
-exec sh -c 'for n; do echo "$n" | tee -a results.txt && python3 "$n" >> results.txt || exit 1; done' sh {} +
env:
Expand Down Expand Up @@ -562,7 +563,7 @@ jobs:
id: cache-boost
with:
path: ${{env.BOOST_ROOT}}
key: boost
key: boost-178-win

- name: Install Boost Headers
if: steps.cache-boost.outputs.cache-hit != 'true'
Expand Down Expand Up @@ -680,9 +681,9 @@ jobs:
python3 `which scons` test show_long_tests=yes verbose_tests=yes --debug=time

windows-mingw:
name: mingw on Windows, Python 3.8
runs-on: windows-2019
timeout-minutes: 60
name: mingw on Windows, Python 3.10
runs-on: windows-2022
timeout-minutes: 120 # MinGW is slooooow
env:
BOOST_ROOT: ${{github.workspace}}/3rdparty/boost
BOOST_URL: https://boostorg.jfrog.io/artifactory/main/release/1.78.0/source/boost_1_78_0.7z
Expand All @@ -694,22 +695,23 @@ jobs:
- name: Set Up Python
uses: actions/setup-python@v4
with:
python-version: 3.8
python-version: "3.10"
architecture: x64
- name: Install Python dependencies
run: |
python -m pip install -U pip 'setuptools>=47.0.0,<48' wheel
python -m pip install -U pip setuptools wheel
python -m pip install scons pypiwin32 numpy ruamel.yaml cython h5py pandas pytest pytest-github-actions-annotate-failures
- name: Restore Boost cache
uses: actions/cache@v3
id: cache-boost
with:
path: ${{env.BOOST_ROOT}}
key: boost
key: boost-178-win
- name: Set up MinGW
uses: egor-tensin/setup-mingw@v2
with:
platform: x64
static: false
- name: Install Boost Headers
if: steps.cache-boost.outputs.cache-hit != 'true'
run: |
Expand All @@ -722,11 +724,26 @@ jobs:
shell: bash
- name: Build Cantera
run: scons build -j2 boost_inc_dir=%BOOST_ROOT% debug=n logging=debug
python_package=full env_vars=PYTHONPATH,GITHUB_ACTIONS
python_package=full env_vars=USERPROFILE,PYTHONPATH,GITHUB_ACTIONS
toolchain=mingw f90_interface=n --debug=time
shell: cmd
- name: Upload Wheel
uses: actions/upload-artifact@v3
with:
path: build\python\dist\Cantera*.whl
name: Cantera-win_amd64.whl
retention-days: 2
- name: Test Cantera
run: scons test show_long_tests=yes verbose_tests=yes --debug=time
- name: Upload Test binaries
if: always()
uses: actions/upload-artifact@v3
with:
path: |
build/test/**/*.exe
build/lib/*.dll
name: mingw-gtest-binaries
retention-days: 2

dotnet:
name: .NET on ${{ matrix.os }}
Expand Down Expand Up @@ -755,7 +772,7 @@ jobs:
shell: pwsh
if: matrix.os == 'windows-2022'
- name: Install Brew dependencies (macOS)
run: brew install hdf5
run: brew install --display-times hdf5
if: matrix.os == 'macos-11'
- name: Install Apt dependencies (Ubuntu)
run: sudo apt install libhdf5-dev
Expand Down
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Expand Up @@ -79,8 +79,8 @@
* Class names use `InitialCapsNames`
* Methods use `camelCaseNames`
* Do not indent the contents of namespaces
* Code should follow the C++14 standard, with minimum required compiler versions
GCC 6.1, Clang 3.4, MSVC 14.1 (2017) and Intel 17.0.
* Code should follow the C++17 standard, with minimum required compiler versions
GCC 7.0, Clang 4.0, MSVC 14.14 (Visual Studio 2017 version 15.7) and Intel 19.0.
* Avoid manual memory management (that is, `new` and `delete`), preferring to use
standard library containers, as well as `std::unique_ptr` and
`std::shared_ptr` when dynamic allocation is required.
Expand Down
34 changes: 26 additions & 8 deletions SConstruct
Expand Up @@ -219,8 +219,8 @@ config_options = [
options with spaces, for example, "cxx_flags='-g -Wextra -O3 --std=c++14'"
""",
{
"cl": "/EHsc",
"default": "-std=c++14"
"cl": "/EHsc /std:c++17",
"default": "-std=c++17"
}),
Option(
"CC",
Expand Down Expand Up @@ -1131,9 +1131,6 @@ if env['coverage']:
logger.error("Coverage testing is only available with GCC.")
exit(0)

if env['toolchain'] == 'mingw':
env.Append(LINKFLAGS=['-static-libgcc', '-static-libstdc++'])

def config_error(message):
if env["logging"].lower() == "debug":
logger.error(message)
Expand Down Expand Up @@ -1900,13 +1897,20 @@ if env["python_package"] == "full" and env["OS"] == "Darwin":
# the name of the wheel file for the Python module. If this is not specified by the
# MACOSX_DEPLOYMENT_TARGET environment variable, get the value from the Python
# installation and use that.
if not env["ENV"].get("MACOSX_DEPLOYMENT_TARGET", False):
mac_target = env["ENV"].get("MACOSX_DEPLOYMENT_TARGET", None)
if not mac_target:
info = get_command_output(
env["python_cmd"],
"-c",
"import sysconfig; print(sysconfig.get_platform())"
)
env["ENV"]["MACOSX_DEPLOYMENT_TARGET"] = info.split("-")[1]
mac_target = info.split("-")[1]
if parse_version(mac_target) < parse_version('10.15'):
# macOS 10.15 is the minimum version with C++17 support
mac_target = '10.15'

env["ENV"]["MACOSX_DEPLOYMENT_TARGET"] = mac_target
logger.info(f"MACOSX_DEPLOYMENT_TARGET = {mac_target}")

# Matlab Toolbox settings
if env["matlab_path"] != "" and env["matlab_toolbox"] == "default":
Expand Down Expand Up @@ -2087,6 +2091,9 @@ else:
for loc in locations:
env[f"inst_{loc}"] = env[f"ct_{loc}"].replace(env["ct_installroot"], instRoot)

if env['use_rpath_linkage']:
env.Append(RPATH=env['ct_libdir'])

# **************************************
# *** Set options needed in config.h ***
# **************************************
Expand Down Expand Up @@ -2189,7 +2196,15 @@ def install(*args, **kwargs):
env.SConsignFile()

env.Prepend(CPPPATH=[],
LIBPATH=[Dir('build/lib')])
LIBPATH=[Dir('build/lib')])

# Add build/lib to library search path in order to find Cantera shared library
if env['OS'] == 'Windows':
env.PrependENVPath('PATH', Dir('#build/lib').abspath)
elif env['OS'] == 'Darwin':
env.PrependENVPath('DYLD_LIBRARY_PATH', Dir('#build/lib').abspath)
else:
env.PrependENVPath('LD_LIBRARY_PATH', Dir('#build/lib').abspath)

for yaml in multi_glob(env, "data", "yaml"):
dest = Path() / "build" / "data" / yaml.name
Expand Down Expand Up @@ -2219,6 +2234,9 @@ else:
env["external_libs"] = []
env["external_libs"].extend(env["sundials_libs"])

if env["OS"] == "Linux":
env["external_libs"].append("dl")

if env["use_hdf5"]:
env["external_libs"].append("hdf5")

Expand Down
3 changes: 1 addition & 2 deletions ext/SConscript
Expand Up @@ -40,8 +40,7 @@ if not env['system_fmt']:
license_files["fmtlib"] = File("#ext/fmt/LICENSE.rst")
localenv = prep_default(env)
localenv.Prepend(CPPPATH=Dir('#ext/fmt/include'))
libraryTargets.extend(
localenv.SharedObject(multi_glob(localenv, 'fmt/src', 'cc')))
libraryTargets.extend(localenv.SharedObject(['fmt/src/format.cc', 'fmt/src/os.cc']))
for name in ('format.h', 'ostream.h', 'printf.h', 'core.h', 'format-inl.h'):
ext_copies.extend(
copyenv.Command("#include/cantera/ext/fmt/" + name,
Expand Down
2 changes: 1 addition & 1 deletion ext/eigen
Submodule eigen updated from 21ae2a to 314739
2 changes: 1 addition & 1 deletion ext/fmt
Submodule fmt updated 135 files
25 changes: 25 additions & 0 deletions include/cantera/base/ExtensionManager.h
Expand Up @@ -44,6 +44,31 @@ class ExtensionManager
throw NotImplementedError("ExtensionManager::registerRateBuilders");
};

//! Register a user-defined ReactionRate implementation with ReactionRateFactory
//! @param extensionName The name of the library/module containing the user-defined
//! rate. For example, the module name for rates implemented in Python.
//! @param className The name of the rate in the user's code. For example, the
//! Python class name
//! @param rateName The name used to construct a rate of this type using
//! the newReactionRate() function or from a YAML input file
virtual void registerRateBuilder(const string& extensionName,
const string& className, const string& rateName)
{
throw NotImplementedError("ExtensionManager::registerRateBuilder");
}

//! Register a user-defined ReactionData implementation
//! @param extensionName The name of the library/module containing the user-defined
//! type. For example, the module name for rates implemented in Python.
//! @param className The name of the data object in the user's code. For example,
//! the Python class name
//! @param rateName The name of the corresponding reaction rate type
virtual void registerRateDataBuilder(const string& extensionName,
const string& className, const string& rateName)
{
throw NotImplementedError("ExtensionManager::registerRateDataBuilder");
}

//! Create an object in an external language that wraps the specified ReactionData
//! object
//!
Expand Down
2 changes: 1 addition & 1 deletion include/cantera/base/ExtensionManagerFactory.h
Expand Up @@ -26,11 +26,11 @@ class ExtensionManagerFactory : public Factory<ExtensionManager>
//! Delete the static instance of this factory
virtual void deleteFactory();

private:
//! Static function that returns the static instance of the factory, creating it
//! if necessary.
static ExtensionManagerFactory& factory();

private:
//! static member of the single factory instance
static ExtensionManagerFactory* s_factory;

Expand Down
18 changes: 13 additions & 5 deletions include/cantera/base/ct_defs.h
Expand Up @@ -22,9 +22,11 @@
#include <cstdlib>
#include <vector>
#include <map>
#include <set>
#include <string>
#include <algorithm>
#include <memory>
#include <functional>

/**
* Namespace for the Cantera kernel.
Expand All @@ -36,6 +38,12 @@ using std::shared_ptr;
using std::make_shared;
using std::unique_ptr;
using std::isnan; // workaround for bug in libstdc++ 4.8
using std::string;
using std::vector;
using std::map;
using std::set;
using std::function;
using std::pair;

/*!
* All physical constants are stored here.
Expand Down Expand Up @@ -159,20 +167,20 @@ const double Tiny = 1.e-20;
/*!
* This is used mostly to assign concentrations and mole fractions to species.
*/
typedef std::map<std::string, double> compositionMap;
typedef map<string, double> compositionMap;

//! Map from string names to doubles. Used for defining species mole/mass
//! fractions, elemental compositions, and reaction stoichiometries.
typedef std::map<std::string, double> Composition;
typedef map<string, double> Composition;

//! Turn on the use of stl vectors for the basic array type within cantera
//! Vector of doubles.
typedef std::vector<double> vector_fp;
typedef vector<double> vector_fp;
//! Vector of ints
typedef std::vector<int> vector_int;
typedef vector<int> vector_int;

//! A grouplist is a vector of groups of species
typedef std::vector<std::vector<size_t> > grouplist_t;
typedef vector<vector<size_t>> grouplist_t;

//! index returned by functions to indicate "no position"
const size_t npos = static_cast<size_t>(-1);
Expand Down
13 changes: 13 additions & 0 deletions include/cantera/base/global.h
Expand Up @@ -87,6 +87,14 @@ void loadExtension(const std::string& extType, const std::string& name);
//! @since New in Cantera 3.0
void loadExtensions(const AnyMap& node);

//! @copydoc Application::searchPythonVersions
void searchPythonVersions(const string& versions);

//! Returns `true` if Cantera was loaded as a shared library in the current
//! application. Returns `false` if it was statically linked.
//! @since New in Cantera 3.0
bool usingSharedLibrary();
speth marked this conversation as resolved.
Show resolved Hide resolved

//! Delete and free all memory associated with the application
/*!
* Delete all global data. It should be called at the end of the
Expand All @@ -97,6 +105,11 @@ void appdelete();
//! @copydoc Application::thread_complete
void thread_complete();

//! Returns the Cantera version. This function when needing to access the version from a
//! library, rather than the `CANTERA_VERSION` macro that is available at compile time.
//! @since New in Cantera 3.0
string version();

//! Returns the hash of the git commit from which Cantera was compiled, if known
std::string gitCommit();

Expand Down
19 changes: 4 additions & 15 deletions include/cantera/clib/clib_defs.h
Expand Up @@ -11,21 +11,10 @@
#include "cantera/base/config.h"
#include <stdlib.h>

#ifdef _WIN32
// Windows (MSVC or MinGW)
# ifdef CANTERA_USE_INTERNAL
# define CANTERA_CAPI extern __declspec(dllexport)
# else
# define CANTERA_CAPI extern __declspec(dllimport)
# endif
#else
// Non-Windows platform
# ifdef CANTERA_USE_INTERNAL
# define CANTERA_CAPI extern
# else
# define CANTERA_CAPI
# endif
#endif
// Legacy attribute applied to clib functions. Currently, used only to identify
// functions that should be considered by the 'sourcegen' parser for inclusion in the
// C# interface.
#define CANTERA_CAPI

// Values returned for error conditions
#ifndef ERR
Expand Down
5 changes: 1 addition & 4 deletions include/cantera/cython/funcWrapper.h
Expand Up @@ -7,9 +7,6 @@
#include "cantera/numerics/Func1.h"
#include "cantera/base/ctexceptions.h"
#include <stdexcept>

#define CANTERA_USE_INTERNAL
#include "cantera/clib/clib_defs.h"
#include "Python.h"

typedef double(*callback_wrapper)(double, void*, void**);
Expand Down Expand Up @@ -166,7 +163,7 @@ class Func1Py : public Cantera::Func1
};

extern "C" {
CANTERA_CAPI PyObject* pyCanteraError;
extern PyObject* pyCanteraError;
}


Expand Down