Combine funtofem, pyfuntofem folder into funtofem #183

sean-engelstad · 2023-05-20T00:10:41Z

Simplify the funtofem package by moving all python and cython code from funtofem and pyfuntofem folders into the funtofem folder for consistency. This way the python module installed by anaconda is called funtofem which matches the conda package name and the github repo name.
Import TransferScheme and MELD classes for mphys with from funtofem import TransferScheme and from funtofem.mphys import MeldBuilder, etc.
Previous classes and interfaces which were imported with pyfuntofem such as from pyfuntofem.model import FUNtoFEMmodel or from pyfuntofem.driver import FUNtoFEMnlbgs are imported now with from funtofem.model import FUNtoFEMmodel and from funtofem.driver import FUNtoFEMnlbgs

sean-engelstad · 2023-05-20T00:22:01Z

This PR addresses the Issue, #140, in which the TransferScheme is stored in a separate folder and are not part of pyfuntofem package so they can't be imported directly from conda. I also moved mphys from funtofem folder into the pyfuntofem/interface/mphys folder now and they are automatically imported all the way up the top pyfuntofem level. I asked @A-CGray to review this since you use mphys.

A-CGray · 2023-05-20T16:41:38Z

So is the idea that my existing MPhys scripts run fine as-is with this import:

from funtofem.mphys import MeldBuilder

Or do I need to change something?

sean-engelstad · 2023-05-20T16:47:15Z

You need to change it to from pyfuntofem import MeldBuilder now

sean-engelstad · 2023-05-25T15:30:28Z

@A-CGray is that okay with you?

bburke38

Just need @A-CGray to approve since this directly affects you.

A-CGray · 2023-05-25T17:00:13Z

I'm getting this error when running make interface:

pip install -e .
Obtaining file:///home/ali/repos/funtofem
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build editable did not run successfully.
  │ exit code: 1
  ╰─> [19 lines of output]
      Traceback (most recent call last):
        File "/home/ali/.pyenv/versions/3.10.9/envs/Aviation2023/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/ali/.pyenv/versions/3.10.9/envs/Aviation2023/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/ali/.pyenv/versions/3.10.9/envs/Aviation2023/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 132, in get_requires_for_build_editable
          return hook(config_settings)
        File "/tmp/pip-build-env-mwtpa9nn/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 450, in get_requires_for_build_editable
          return self.get_requires_for_build_wheel(config_settings)
        File "/tmp/pip-build-env-mwtpa9nn/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-mwtpa9nn/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-mwtpa9nn/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 487, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/tmp/pip-build-env-mwtpa9nn/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in run_setup
          exec(code, locals())
        File "<string>", line 6, in <module>
      ModuleNotFoundError: No module named 'numpy'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build editable did not run successfully.
│ exit code: 1
╰─> See above for output.

I definitely have numpy installed

sean-engelstad · 2023-05-26T18:00:42Z

@A-CGray I recreated your error by starting a new conda environment in python 3.8 and python 3.10. Both versions gave me the same numpy error as you. Here's what I did to fix it, main thing was conda installing numpy. Can you try again?

conda create -n F2F python=3.8
conda activate F2F
conda install -c conda-forge numpy mpi4py cython
conda install gxx_linux-64=9.3.0 -q -y
cd ~/git/funtofem
make clean
make
make interface

…ntofem_only

sean-engelstad · 2023-05-28T01:52:27Z

@A-CGray I added the pyproject.toml to this pull request. I submitted a PR to mphys OpenMDAO/mphys#152, and waiting on word back from that.

timryanb · 2023-05-29T11:47:36Z

@sean-engelstad can we preserve the mphys subpackage in the import path i.e. from pyfuntofem.mphys import MeldBuilder. This keeps consistent with the other tools in the mphys library.

Also it sounds like this will be a breaking change, which means it should follow with a version bump

timryanb · 2023-05-29T11:51:30Z

It's probably worth mentioning this on the mphys call so the other users are aware of this as well

sean-engelstad · 2023-05-29T13:11:47Z

@timryanb I just moved the mphys folder from pyfuntofem.interface.mphys to pyfuntofem.mphys like you requested. The import will be from pyfuntofem.mphys import MeldBuilder, etc. I also updated the mphys PR for the imports there.

kejacobson · 2023-05-29T13:18:30Z

pyproject.toml

+#pyproject.toml
+[build-system]
+# Minimum requirements for the build system to execute.
+requires = ['setuptools>=45.0', 'wheel', 'cython>=0.29', 'oldest-supported-numpy', 'mpi4py==3.1.1']


Can you make this 'mpi4py>=3.1.1'? I have 3.1.4 and it tried to install 3.1.1 unnecessarily

pip will still likely install a temporary version of mpi4py during the build regardless @kejacobson. This is how pip handles building cython packages. In general we pin against older versions of mpi4py because they are always forward compatible if the users system mpi4py is newer. I'd recommend keeping the mpi4py version number as was, since we've seen issues in the past with pip trying to build codes against newer versions mpi4py than the user is running. This is the same reason we specify 'numpy-oldest' in the build section

Ok I changed this back to mpi4py==3.1.1 then. I thought there was a reason that was the convention in TACS

Interesting, it didn't seem forward compatible when I tried to build it. My pip failed when building mpi4py 3.1.1 with ==, but found my existing 3.1.4 with the >= change.

Weird. Do you have the failure message? Also what version of pip are you using?

sean-engelstad · 2023-05-29T13:23:52Z

@kejacobson I've changed the pyproject.toml file to mpi4py>=3.1.1 now.

timryanb · 2023-05-29T14:41:10Z

@timryanb I just moved the mphys folder from pyfuntofem.interface.mphys to pyfuntofem.mphys like you requested. The import will be from pyfuntofem.mphys import MeldBuilder, etc. I also updated the mphys PR for the imports there.

You can also accomplish this by adding from interface import mphys to pyfuntofem/__init__.py if you still want to house the mphys directory under interfaces

sean-engelstad · 2023-05-29T16:54:24Z

I decided to leave the mphys folder in the directory pyfuntofem/mphys since I was having trouble getting that to work @timryanb. It allowed me to do from pyfuntofem import mphys but not from pyfuntofem.mphys import MeldBuilder since there was no mphys submodule in that directory. Importing it into there didn't seem to work. I also tried setting up a mphys.py submodule file, but that doens't work you need another mphys folder in that place.

sean-engelstad · 2023-05-30T14:18:19Z

Brian and I have decided to instead combine all the python and cython code into the funtofem folder not the pyfuntofem folder. This way the conda package funtofem installs the python module funtofem which is consistent with the conda package and repo name (otherwise with pyfuntofem it is confusing and mismatching). This is a non-backwards compatible change so we will be releasing a new version after this.

pyproject.toml

bburke38

@sean-engelstad Can you change the first topic in the conversation to indicate the new import name?

sean-engelstad · 2023-05-31T00:05:38Z

@bburke38 I updated the main message block of the PR to say that we are merging into funtofem now and how the imports have changed.

sean-engelstad added 2 commits May 19, 2023 20:01

move TransferScheme cython classes to pyfuntofem

cc05e1d

black reformat

8ba3168

sean-engelstad added the enhancement New feature or request label May 20, 2023

improved mphys python imports

86acf4d

sean-engelstad linked an issue May 20, 2023 that may be closed by this pull request

Transfer schemes are not part of pyFUNtoFEM package #140

Closed

sean-engelstad requested review from bburke38 and A-CGray May 20, 2023 00:16

sean-engelstad removed the enhancement New feature or request label May 20, 2023

remove in_github_workflow overwrite

f0bfa08

bburke38 approved these changes May 25, 2023

View reviewed changes

sean-engelstad changed the title ~~Move funtofem folder into pyfuntofem~~ Move funtofem and mphys folder into pyfuntofem May 26, 2023

sean-engelstad added 2 commits May 27, 2023 20:12

Merge branch 'master' of github.com:sean-engelstad/funtofem into pyfu…

d02bc0e

…ntofem_only

added pyproject toml for easier install

ed971a6

sean-engelstad mentioned this pull request May 28, 2023

Modify Funtofem Mphys Imports OpenMDAO/mphys#152

Closed

sean-engelstad linked an issue May 28, 2023 that may be closed by this pull request

Can't find numpy package build error #186

Closed

sean-engelstad added 2 commits May 29, 2023 09:02

moved mphys to pyfuntofem.mphys directory

24cdcba

updated __init__.py files for mphys

b5d483d

kejacobson reviewed May 29, 2023

View reviewed changes

changed to mpi4py>=3.1.1 in pyproject.toml

6531417

switched back to mpi4py==3.1.1

b9c2907

change pyfuntofem folder to funtofem to be consistent with conda

02637d0

sean-engelstad changed the title ~~Move funtofem and mphys folder into pyfuntofem~~ Combine funtofem,pyfuntofem,mphys folder into funtofem May 30, 2023

sean-engelstad changed the title ~~Combine funtofem,pyfuntofem,mphys folder into funtofem~~ Combine funtofem, pyfuntofem, mphys folder into funtofem May 30, 2023

timryanb reviewed May 30, 2023

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

Update pyproject.toml to be python 3.11 compatible

1296403

sean-engelstad changed the title ~~Combine funtofem, pyfuntofem, mphys folder into funtofem~~ Combine funtofem, pyfuntofemfolder into funtofem May 30, 2023

sean-engelstad changed the title ~~Combine funtofem, pyfuntofemfolder into funtofem~~ Combine funtofem, pyfuntofem folder into funtofem May 30, 2023

bburke38 self-requested a review May 30, 2023 23:55

bburke38 requested changes May 30, 2023

View reviewed changes

bburke38 self-requested a review May 31, 2023 15:36

bburke38 approved these changes May 31, 2023

View reviewed changes

sean-engelstad merged commit e256b5f into smdogroup:master May 31, 2023
6 checks passed

sean-engelstad deleted the pyfuntofem_only branch August 23, 2023 19:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Combine funtofem, pyfuntofem folder into funtofem #183

Combine funtofem, pyfuntofem folder into funtofem #183

sean-engelstad commented May 20, 2023 •

edited

Loading

sean-engelstad commented May 20, 2023 •

edited

Loading

A-CGray commented May 20, 2023

sean-engelstad commented May 20, 2023

sean-engelstad commented May 25, 2023

bburke38 left a comment

A-CGray commented May 25, 2023

sean-engelstad commented May 26, 2023 •

edited

Loading

sean-engelstad commented May 28, 2023 •

edited

Loading

timryanb commented May 29, 2023

timryanb commented May 29, 2023

sean-engelstad commented May 29, 2023 •

edited

Loading

kejacobson May 29, 2023

timryanb May 29, 2023 •

edited

Loading

sean-engelstad May 29, 2023

kejacobson May 29, 2023

timryanb May 29, 2023 •

edited

Loading

sean-engelstad commented May 29, 2023

timryanb commented May 29, 2023

sean-engelstad commented May 29, 2023

sean-engelstad commented May 30, 2023

bburke38 left a comment

sean-engelstad commented May 31, 2023

Combine funtofem, pyfuntofem folder into funtofem #183

Combine funtofem, pyfuntofem folder into funtofem #183

Conversation

sean-engelstad commented May 20, 2023 • edited Loading

sean-engelstad commented May 20, 2023 • edited Loading

A-CGray commented May 20, 2023

sean-engelstad commented May 20, 2023

sean-engelstad commented May 25, 2023

bburke38 left a comment

Choose a reason for hiding this comment

A-CGray commented May 25, 2023

sean-engelstad commented May 26, 2023 • edited Loading

sean-engelstad commented May 28, 2023 • edited Loading

timryanb commented May 29, 2023

timryanb commented May 29, 2023

sean-engelstad commented May 29, 2023 • edited Loading

kejacobson May 29, 2023

Choose a reason for hiding this comment

timryanb May 29, 2023 • edited Loading

Choose a reason for hiding this comment

sean-engelstad May 29, 2023

Choose a reason for hiding this comment

kejacobson May 29, 2023

Choose a reason for hiding this comment

timryanb May 29, 2023 • edited Loading

Choose a reason for hiding this comment

sean-engelstad commented May 29, 2023

timryanb commented May 29, 2023

sean-engelstad commented May 29, 2023

sean-engelstad commented May 30, 2023

bburke38 left a comment

Choose a reason for hiding this comment

sean-engelstad commented May 31, 2023

sean-engelstad commented May 20, 2023 •

edited

Loading

sean-engelstad commented May 20, 2023 •

edited

Loading

sean-engelstad commented May 26, 2023 •

edited

Loading

sean-engelstad commented May 28, 2023 •

edited

Loading

sean-engelstad commented May 29, 2023 •

edited

Loading

timryanb May 29, 2023 •

edited

Loading

timryanb May 29, 2023 •

edited

Loading