BUG: f2py - Invalid command line options do not warn or error on Python 3.12 #24874
Comments
|
I also note that |
|
After installing these, I replace the line 140 of shutil.copy(source, bdir)by but then compilation fails FAILED: _interface.cpython-312-x86_64-linux-gnu.so.p/interface.f90.o
gfortran -I_interface.cpython-312-x86_64-linux-gnu.so.p -I. -I.. -I/home/alex/Python/lib/python3.12/site-packages/numpy/core/include -I/home/alex/Python/lib/python3.12/site-packages/numpy/f2py/src -I/home/alex/Python/include/python3.12 -fvisibility=hidden -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -O3 -fPIC -J_interface.cpython-312-x86_64-linux-gnu.so.p -o _interface.cpython-312-x86_64-linux-gnu.so.p/interface.f90.o -c ../interface.f90
../interface.f90:8:7:
8 | use typedef, only: &
| 1
Fatal Error: Cannot open module file ‘typedef.mod’ for reading at (1): No such file or directory
```shell
The Meson build system
Version: 1.2.2
Source dir: /home/alex/python/source/multistar/_build
Build dir: /home/alex/python/source/multistar/_build/bbdir
Build type: native build
Project name: _interface
Project version: 0.1
Fortran compiler for the host machine: gfortran (gcc 13.2.1 "GNU Fortran (GCC) 13.2.1 20230728 (Red Hat 13.2.1-1)")
Fortran linker for the host machine: gfortran ld.bfd 2.39-9
C compiler for the host machine: cc (gcc 13.2.1 "cc (GCC) 13.2.1 20230728 (Red Hat 13.2.1-1)")
C linker for the host machine: cc ld.bfd 2.39-9
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python3 found: YES (/home/alex/Python/bin/python3.12)
Found pkg-config: /usr/bin/pkg-config (1.8.0)
Run-time dependency python found: YES 3.12
Build targets in project: 1
Found ninja-1.11.1.git.kitware.jobserver-1 at /home/alex/Python/bin/ninja
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /home/alex/Python/bin/ninja -C /home/alex/python/source/multistar/_build/bbdir
ninja: Entering directory `/home/alex/python/source/multistar/_build/bbdir'
[3/7] Compiling Fortran object _interface.cpython-312-x86_64-linux-gnu.so.p/interface.f90.o
FAILED: _interface.cpython-312-x86_64-linux-gnu.so.p/interface.f90.o
gfortran -I_interface.cpython-312-x86_64-linux-gnu.so.p -I. -I.. -I/home/alex/Python/lib/python3.12/site-packages/numpy/core/include -I/home/alex/Python/lib/python3.12/site-packages/numpy/f2py/src -I/home/alex/Python/include/python3.12 -fvisibility=hidden -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -O3 -fPIC -J_interface.cpython-312-x86_64-linux-gnu.so.p -o _interface.cpython-312-x86_64-linux-gnu.so.p/interface.f90.o -c ../interface.f90
../interface.f90:8:7:
8 | use typedef, only: &
| 1
Fatal Error: Cannot open module file ‘typedef.mod’ for reading at (1): No such file or directory
compilation terminated.
[4/7] Compiling Fortran object _interface.cpython-312-x86_64-linux-gnu.so.p/_interface-f2pywrappers2.f90.o
FAILED: _interface.cpython-312-x86_64-linux-gnu.so.p/_interface-f2pywrappers2.f90.o
gfortran -I_interface.cpython-312-x86_64-linux-gnu.so.p -I. -I.. -I/home/alex/Python/lib/python3.12/site-packages/numpy/core/include -I/home/alex/Python/lib/python3.12/site-packages/numpy/f2py/src -I/home/alex/Python/include/python3.12 -fvisibility=hidden -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -O3 -fPIC -J_interface.cpython-312-x86_64-linux-gnu.so.p -o _interface.cpython-312-x86_64-linux-gnu.so.p/_interface-f2pywrappers2.f90.o -c ../_interface-f2pywrappers2.f90
../_interface-f2pywrappers2.f90:7:21:
7 | use typedef, only: real64
| 1
Fatal Error: Cannot open module file ‘typedef.mod’ for reading at (1): No such file or directory
compilation terminated.
[6/7] Compiling C object _interface.cpython-312-x86_64-linux-gnu.so.p/_interfacemodule.c.o
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/home/alex/Python/bin/f2py3", line 8, in <module>
sys.exit(main())
^^^^^^
File "/home/alex/Python/lib/python3.12/site-packages/numpy/f2py/f2py2e.py", line 732, in main
run_compile()
File "/home/alex/Python/lib/python3.12/site-packages/numpy/f2py/f2py2e.py", line 705, in run_compile
builder.compile()
File "/home/alex/Python/lib/python3.12/site-packages/numpy/f2py/_backends/_meson.py", line 131, in compile
self.run_meson(self.build_dir)
File "/home/alex/Python/lib/python3.12/site-packages/numpy/f2py/_backends/_meson.py", line 124, in run_meson
raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['meson', 'compile', '-C', 'bbdir']' returned non-zero exit status 1.Apparently, the Happy to help with debugging. |
|
I try to add the include file manually, adding |
|
another error seems to be [6/6] Linking target _interface.cpython-312-x86_64-linux-gnu.so
Traceback (most recent call last):
File "/home/alex/Python/bin/f2py3", line 8, in <module>
sys.exit(main())
^^^^^^
File "/home/alex/Python/lib/python3.12/site-packages/numpy/f2py/f2py2e.py", line 732, in main
run_compile()
File "/home/alex/Python/lib/python3.12/site-packages/numpy/f2py/f2py2e.py", line 705, in run_compile
builder.compile()
File "/home/alex/Python/lib/python3.12/site-packages/numpy/f2py/_backends/_meson.py", line 132, in compile
self._move_exec_to_root(self.build_dir)
File "/home/alex/Python/lib/python3.12/site-packages/numpy/f2py/_backends/_meson.py", line 88, in _move_exec_to_root
shutil.move(path_object, Path.cwd())
File "/home/alex/Python/lib/python3.12/shutil.py", line 860, in move
raise Error("Destination path '%s' already exists" % real_dst)
shutil.Error: Destination path '/home/alex/python/source/multistar/_interface.cpython-312-x86_64-linux-gnu.so' already exists |
|
here, in def _move_exec_to_root(self, build_dir: Path):
walk_dir = Path(build_dir) / self.meson_build_dir
path_objects = walk_dir.glob(f"{self.modulename}*.so")
for path_object in path_objects:
shutil.move(path_object, Path.cwd())needs to be replaced by def _move_exec_to_root(self, build_dir: Path):
walk_dir = Path(build_dir) / self.meson_build_dir
path_objects = walk_dir.glob(f"{self.modulename}*.so")
for path_object in path_objects:
if (file := (Path.cwd() / path_object.name)).exists:
file.unlink()
shutil.move(path_object, Path.cwd()) |
|
... after some more wiggling I get it to produce a binary module, however, however, I can't import stuff from it. Since compiler flags are not passed on, optimisations and special settings are lost. This requires a fix. The object library |
|
It seems |
|
Some modifications that seem to work, except it does not allow to build in its own directory, the the specified include directory for fortran template module from __future__ import annotations
import errno
import shutil
import subprocess
from pathlib import Path
from ._backend import Backend
from string import Template
import warnings
class MesonTemplate:
"""Template meson build file generation class."""
def __init__(
self,
modulename: str,
sources: list[Path],
deps: list[str],
object_files: list[Path],
linker_args: list[str],
c_args: list[str],
build_type: str,
extra_objects: list[str],
):
self.modulename = modulename
self.build_template_path = (
Path(__file__).parent.absolute() / "meson.build.template"
)
self.sources = sources
self.deps = deps
self.substitutions = {}
self.objects = object_files
self.extra_objects = extra_objects
self.pipeline = [
self.initialize_template,
self.sources_substitution,
self.deps_substitution,
self.extra_substitution,
]
self.build_type = build_type
def meson_build_template(self) -> str:
if not self.build_template_path.is_file():
raise FileNotFoundError(
errno.ENOENT,
"Meson build template"
f" {self.build_template_path.absolute()}"
" does not exist.",
)
return self.build_template_path.read_text()
def initialize_template(self) -> None:
self.substitutions["modulename"] = self.modulename
self.substitutions["buildtype"] = self.build_type
def sources_substitution(self) -> None:
indent = " " * 21
self.substitutions["source_list"] = f",\n{indent}".join(
[f"'{source}'" for source in self.sources]
)
def deps_substitution(self) -> None:
indent = " " * 21
self.substitutions["dep_list"] = f",\n{indent}".join(
[f"dependency('{dep}')" for dep in self.deps]
)
def extra_substitution(self) -> None:
indent = " " * 21
self.substitutions["extra_objects"] = f",\n{indent}".join(
[f"'{extra}'" for extra in self.extra_objects]
)
def generate_meson_build(self):
for node in self.pipeline:
node()
template = Template(self.meson_build_template())
return template.substitute(self.substitutions)
class MesonBackend(Backend):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.dependencies = self.extra_dat.get("dependencies", [])
self.meson_build_dir = "bbdir"
self.build_type = (
"debug" if any("debug" in flag for flag in self.fc_flags) else "release"
)
def _move_exec_to_root(self, build_dir: Path):
walk_dir = Path(build_dir) / self.meson_build_dir
path_objects = walk_dir.glob(f"{self.modulename}*.so")
for path_object in path_objects:
if (file := (Path.cwd() / path_object.name)).exists:
file.unlink()
shutil.move(path_object, Path.cwd())
def _get_build_command(self):
return [
"meson",
"setup",
self.meson_build_dir,
]
def write_meson_build(self, build_dir: Path) -> None:
"""Writes the meson build file at specified location"""
meson_template = MesonTemplate(
self.modulename,
self.sources,
self.dependencies,
self.extra_objects,
self.flib_flags,
self.fc_flags,
self.build_type,
self.extra_objects,
)
src = meson_template.generate_meson_build()
Path(build_dir).mkdir(parents=True, exist_ok=True)
meson_build_file = Path(build_dir) / "meson.build"
meson_build_file.write_text(src)
return meson_build_file
def run_meson(self, build_dir: Path):
completed_process = subprocess.run(self._get_build_command(), cwd=build_dir)
if completed_process.returncode != 0:
raise subprocess.CalledProcessError(
completed_process.returncode, completed_process.args
)
completed_process = subprocess.run(
["meson", "compile", "-C", self.meson_build_dir], cwd=build_dir
)
if completed_process.returncode != 0:
raise subprocess.CalledProcessError(
completed_process.returncode, completed_process.args
)
def compile(self) -> None:
self.sources = _prepare_sources(self.modulename, self.sources, self.build_dir)
self.write_meson_build(self.build_dir)
self.run_meson(self.build_dir)
self._move_exec_to_root(self.build_dir)
def _prepare_sources(mname, sources, bdir):
extended_sources = sources.copy()
Path(bdir).mkdir(parents=True, exist_ok=True)
# Copy sources
for source in sources:
try:
shutil.copy(source, bdir)
except shutil.SameFileError:
pass
generated_sources = [
Path(f"{mname}module.c"),
Path(f"{mname}-f2pywrappers2.f90"),
Path(f"{mname}-f2pywrappers.f"),
]
bdir = Path(bdir)
for generated_source in generated_sources:
if generated_source.exists():
shutil.copy(generated_source, bdir / generated_source.name)
extended_sources.append(generated_source.name)
generated_source.unlink()
extended_sources = [
Path(source).name
for source in extended_sources
if not Path(source).suffix == ".pyf"
]
return extended_sources |
|
Now a version that seems to work for my use case, also allowing to pass parameters to the fortran compiler. Obviously, this can be system-dependent and care is on the user for now. But t least it can be done.
from __future__ import annotations
import errno
import shutil
import subprocess
from pathlib import Path
from ._backend import Backend
from string import Template
import warnings
class MesonTemplate:
"""Template meson build file generation class."""
def __init__(
self,
modulename: str,
sources: list[Path],
deps: list[str],
object_files: list[Path],
linker_args: list[str],
c_args: list[str],
build_type: str,
extra_objects: list[str],
fc_flags: list[str],
):
self.modulename = modulename
self.build_template_path = (
Path(__file__).parent.absolute() / "meson.build.template"
)
self.sources = sources
self.deps = deps
self.substitutions = {}
self.objects = object_files
self.extra_objects = extra_objects
self.fc_flags = fc_flags
self.sources = sources
self.pipeline = [
self.initialize_template,
self.sources_substitution,
self.deps_substitution,
self.extra_substitution,
self.determine_f90,
self.fc_flags_substitution,
]
self.build_type = build_type
def meson_build_template(self) -> str:
if not self.build_template_path.is_file():
raise FileNotFoundError(
errno.ENOENT,
"Meson build template"
f" {self.build_template_path.absolute()}"
" does not exist.",
)
return self.build_template_path.read_text()
def initialize_template(self) -> None:
self.substitutions["modulename"] = self.modulename
self.substitutions["buildtype"] = self.build_type
def sources_substitution(self) -> None:
indent = " " * 21
self.substitutions["source_list"] = f",\n{indent}".join(
[f"'{source}'" for source in self.sources]
)
def deps_substitution(self) -> None:
indent = " " * 21
self.substitutions["dep_list"] = f",\n{indent}".join(
[f"dependency('{dep}')" for dep in self.deps]
)
def extra_substitution(self) -> None:
indent = " " * 21
self.substitutions["extra_objects"] = f",\n{indent}".join(
[f"'{extra}'" for extra in self.extra_objects]
)
def determine_f90(self) -> None:
for source in self.sources:
if (source.endswith('f2pywrappers.f90') or
source.endswith('f2pywrappers2.f90')):
self.f90 = True
break
else:
self.f90 = False
def fc_flags_substitution(self) -> None:
# TODO - options should be filteretd and split up
indent = " " * 21
if self.f90:
select = '--f90flags='
else:
select = '--f77flags='
for flags in self.fc_flags:
if flags.startswith(select):
fortran_flags = flags[11:].split()
break
else:
fortrtran_flags = list()
self.substitutions["fortran_args"] = f",\n{indent}".join(
[f"'{flag}'" for flag in fortran_flags]
)
def generate_meson_build(self):
for node in self.pipeline:
node()
template = Template(self.meson_build_template())
return template.substitute(self.substitutions)
class MesonBackend(Backend):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.dependencies = self.extra_dat.get("dependencies", [])
self.meson_build_dir = "bbdir"
self.build_type = (
"debug" if any("debug" in flag for flag in self.fc_flags) else "release"
)
def _move_exec_to_root(self, build_dir: Path):
walk_dir = Path(build_dir) / self.meson_build_dir
path_objects = walk_dir.glob(f"{self.modulename}*.so")
for path_object in path_objects:
if (file := (Path.cwd() / path_object.name)).exists:
file.unlink()
shutil.move(path_object, Path.cwd())
def _get_build_command(self):
return [
"meson",
"setup",
self.meson_build_dir,
]
def write_meson_build(self, build_dir: Path) -> None:
"""Writes the meson build file at specified location"""
meson_template = MesonTemplate(
self.modulename,
self.sources,
self.dependencies,
self.extra_objects,
self.flib_flags,
self.fc_flags,
self.build_type,
self.extra_objects,
self.fc_flags,
)
src = meson_template.generate_meson_build()
Path(build_dir).mkdir(parents=True, exist_ok=True)
meson_build_file = Path(build_dir) / "meson.build"
meson_build_file.write_text(src)
return meson_build_file
def run_meson(self, build_dir: Path):
completed_process = subprocess.run(self._get_build_command(), cwd=build_dir)
if completed_process.returncode != 0:
raise subprocess.CalledProcessError(
completed_process.returncode, completed_process.args
)
completed_process = subprocess.run(
["meson", "compile", "-v", "-C", self.meson_build_dir], cwd=build_dir
)
if completed_process.returncode != 0:
raise subprocess.CalledProcessError(
completed_process.returncode, completed_process.args
)
def compile(self) -> None:
self.sources = _prepare_sources(self.modulename, self.sources, self.build_dir)
self.write_meson_build(self.build_dir)
self.run_meson(self.build_dir)
self._move_exec_to_root(self.build_dir)
def _prepare_sources(mname, sources, bdir):
extended_sources = sources.copy()
Path(bdir).mkdir(parents=True, exist_ok=True)
# Copy sources
for source in sources:
try:
shutil.copy(source, bdir)
except shutil.SameFileError:
pass
generated_sources = [
Path(f"{mname}module.c"),
Path(f"{mname}-f2pywrappers2.f90"),
Path(f"{mname}-f2pywrappers.f"),
]
bdir = Path(bdir)
for generated_source in generated_sources:
if generated_source.exists():
shutil.copy(generated_source, bdir / generated_source.name)
extended_sources.append(generated_source.name)
generated_source.unlink()
extended_sources = [
Path(source).name
for source in extended_sources
if not Path(source).suffix == ".pyf"
]
return extended_sourcesIf there is interest, I can make a pull request of it. |
|
Thanks for going over the new module and apologies for not noticing this earlier (feel free to ping me on f2py issues generally). In general, none of the old flags to pass parameters will work, and the idea is to use It should have worked by just populating |
|
To me, personally, I would rather document this better and let users rework their build systems to |
Tracked in #24670 |
|
@HaoZeke Tanks for the clarification. Yes, I had to know what I wanted in the Would it be ok to add these flange for some modest out-of-the-box backwards compatibility? No one has to use these for new projects or at all. I would make a pull request. My background is that I use these in a python script that links f2py with external library also built by the script, but using an old-fashioned (extended) makefile. Ongoing scientific code development, not using |
That's a good point, sorry I should have read the issue more carefully. I think Also (not well documented) there was a e.g. For LAPACK: This also supports external libraries with a
I would be quite strongly against this. The reason being that the plan is to have
I understand, and in general backwards compatibility is a major concern but, NumPy 2.0 requires users to come and look at the documentation anyway since it is a major version change with a lot of breaking changes. In the same vein, It would be really great if you could add an example showing the translation from pre-2.0 ^- I understand this is a more annoying request than accepting the PR for the changes which work for your code (and will generally, for
So the python process runs F2PY as a shell command? (via subprocess or something?) That should be a good fit for setting environment variables in the dependent shell as well, via modifications of |
|
@HaoZeke I trigger f2py using a subprocess call. os.environ does not work for this purpose, retains environment python was called with. At least, this is what it was s few years back. |
|
I understand that you want to offload responsibility for the link and compile stage. My proposal aims at adding more basic functionality to allow people a quick start for simple project.
If there are settings which cannot be passed as environment variables and are not amenable to being upstreamed I could see this but I don't think I understand which options would require this (yet).
os.environ does not work for this purpose, retains environment python was called with. At least, this is what it was s few years back.
It does, but it returns a dictionary so the common use case is (honoring environment variables)
cenv = os.environ
cenv['FFLAGS'] = cenv.get('FFLAGS', '') + ' -O2'
result = subprocess.run(['whatever'], text=True, capture_output=True, env=cenv)
Or to use a "blank" environment one could call subprocess.run with any dictionary, but I prefer this approach.
…On Oct 20 2023, at 3:30 pm, Alexander Heger ***@***.***> wrote:
@HaoZeke ***@***.***/0?redirect=https%3A%2F%2Fgithub.com%2FHaoZeke&recipient=cmVwbHkrYWJiY3V0Mjd4MnMzYzRubTJwM29lYWdkaDdvMnBldmJuaGhoZ2cyYnVlQHJlcGx5LmdpdGh1Yi5jb20%3D)
I understand that you want to offload responsibility for the link and compile stage. My proposal aims at adding more basic functionality to allow people a quick start for simple project.
I trigger f2py using a subprocess call.
os.environ does not work for this purpose, retains environment python was called with. At least, this is what it was s few years back.
—
Reply to this email directly, view it on GitHub ***@***.***/1?redirect=https%3A%2F%2Fgithub.com%2Fnumpy%2Fnumpy%2Fissues%2F24874%23issuecomment-1772958923&recipient=cmVwbHkrYWJiY3V0Mjd4MnMzYzRubTJwM29lYWdkaDdvMnBldmJuaGhoZ2cyYnVlQHJlcGx5LmdpdGh1Yi5jb20%3D), or unsubscribe ***@***.***/2?redirect=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABBCUT767IS6J5TAENOWV43YAKKJPAVCNFSM6AAAAAA5WTXUCSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZSHE2TQOJSGM&recipient=cmVwbHkrYWJiY3V0Mjd4MnMzYzRubTJwM29lYWdkaDdvMnBldmJuaGhoZ2cyYnVlQHJlcGx5LmdpdGh1Yi5jb20%3D).
You are receiving this because you were mentioned.
|
|
@HaoZeke Ah, yes, now I recall, I wanted to use environment variables in a different way, access them in a library compiled/linked with f2py and these set before loading the module - or used on the module in general. I had not found a way to change that in this case. I will give your suggestion of workaround a try in the next few days or week. (Still waiting for the port of vtk before transitioning to Python 3.12) |
Makes sense, I think for that there would need to be changes to the C code generated by F2PY (to capture the environment and pass it through). I will open an enhancement issue for it.
Wonderful, thanks a ton. |
HaoZeke
changed the title
Could this be documented somewhere? I just lost several hours to this, wondering why none of my arguments are being picked up. |
Sorry about that, it was in the release notes, but not prominently listed. For now, examples are still in #24532, but the documentation has been updated in https://github.com/numpy/numpy/pull/25124/files if you'd like to take a look. I'm leaving this open because I want to at some point have a "cheatsheet" to translate between common |
|
@HaoZeke def _move_exec_to_root(self, build_dir: Path):
walk_dir = Path(build_dir) / self.meson_build_dir
path_objects = walk_dir.glob(f"{self.modulename}*.so")
for path_object in path_objects:
shutil.move(path_object, Path.cwd())by def _move_exec_to_root(self, build_dir: Path):
walk_dir = Path(build_dir) / self.meson_build_dir
path_objects = walk_dir.glob(f"{self.modulename}*.so")
for path_object in path_objects:
if (file := (Path.cwd() / path_object.name)).exists:
file.unlink()
shutil.move(path_object, Path.cwd()) |
|
And also def _prepare_sources(mname, sources, bdir):
extended_sources = sources.copy()
Path(bdir).mkdir(parents=True, exist_ok=True)
# Copy sources
for source in sources:
shutil.copy(source, bdir)by def _prepare_sources(mname, sources, bdir):
extended_sources = sources.copy()
Path(bdir).mkdir(parents=True, exist_ok=True)
# Copy sources
for source in sources:
if Path(source).parent == Path(bdir):
continue
shutil.copy(source, bdir) |
This is related to in tree builds right? I am not sure it's a good idea to unconditionally remove pre existing files, though it is annoying in development environments.. |
|
@HaoZeke |
|
@HaoZeke |
I also spend more time on development and find it annoying... However it is the default behavior, even signature files are not overwritten by default. A flag would be a reasonable solution but also seems a little convoluted (would have to store state and make sure it works for distutils as well) for what is basically a single line externally, i.e: Perhaps a note in the documentation? I'm not actually against a flag as a PR, it might even be useful in some CI situations, but it would need to account for the different naming schemes on windows and stuff like that... |
|
@HaoZeke This leaves the issue with the move to same file I mentioned first. |
This makes more sense but it still adds an assumption of state. So you might be left with files which aren't updated because of the continue clause. Which would again be an edge case someone can run into. The current behavior is clear and unambiguous IMO.. Since what's happening is that we copy the files to a new folder and run meson. Trying to handle the case of copying files back is kind of problematic to deal with. It back to the problem of in place builds, which I guess is best not supported. This is also the stance taken by meson. The easiest solution is to just use the build directory which can be easily removed when needed without worrying about changes made to the source files. |
|
@HaoZeke |
|
The harm here is that it is then silenced, and a user would not know why their files aren't being updated. The current code is verbose, and lets the user know exactly what went wrong. In general, as mentioned earlier, this will only mostly happen either in development environments or when trying to use the same directory for build-dir (which might as well be disallowed perhaps).
…On 11/30/23 4:49 AM, Alexander Heger ***@***.***> wrote:
@HaoZeke <https://github.com/HaoZeke>
My point is that I believe that the current code does not make sense in
any circumstance, trying to move a file onto itself, and then failing to
do so. Maybe, usually, this should not occur as usually the directories
differ, but what is the harm in the code not failing if it does occur?
—
Reply to this email directly, view it on GitHub
<#24874 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABBCUT2DQOILSWWGEZRGWDLYHAF6JAVCNFSM6AAAAAA5WTXUCSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZTGEYDGOJYHA>.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
@HaoZeke I found the |
So the problem is when you have a file being copied onto itself either:
This seems like a trivial thing to document, and since the That is, given that this: cat << EOF > fib.f
SUBROUTINE FIB(A,N)
C
C CALCULATE FIRST N FIBONACCI NUMBERS
C
INTEGER N
REAL*8 A(N)
DO I=1,N
IF (I.EQ.1) THEN
A(I) = 0.0D0
ELSEIF (I.EQ.2) THEN
A(I) = 1.0D0
ELSE
A(I) = A(I-1) + A(I-2)
ENDIF
ENDDO
END
EOF
# No errors no matter how many times it is run, overwrites automatically
python -m numpy.f2py -c fib.f -m fib --backend distutils
# Will error out if the .so is present
python -m numpy.f2py -c fib.f -m fib --backend mesonSo it is OK to remove the file to be consistent with Which seems so trivially simple I wouldn't have normally bothered adding automatic file overwriting (which the user will then need to remember). For example, I forgot
I'm sorry it seems this way, but the bottleneck is really in the frontend. All of these would be trivial if adding flags was much easier, but it isn't because of #25179. The migration from Personally, I'm always glad there's engagement from users like yourself, and the distribution based slant in development is just because for the most part (still) the biggest consumer is At the end of the day its about how much additional effort it takes to support each workflow at the user level. For example, if we change to unconditionally overwrite the Also pinging @melissawm for another F2PY maintainer perspective :) |
|
@HaoZeke I probably should have made a PR from the original post, but then was unavailable for a few weeks due to an accident, hence the long silence. My use case is that I have a detailed ... I am sure I will find a way around in the end. Thanks for all your support. |
I'm very sorry to hear about that, I hope you are in better health now.
Would this be calling |
|
@2sn, we've restored the previous behavior, i.e. |
|
@HaoZeke I think this error had resulted from the As for calling As for making my packages available on github/PyPI, there are too many small and interconnected packages I wrote over the years to make this an easy task, and splitting them up in the independent way they should be, makes handling dozens of git repositories quite a pain compared to just one private one, considering that development changes usually involve several packages. There must be a better way. |
FWIW, I think that that is the expected behavior. It matches what any other build steps/systems do for generated files when the source file changes. |
yes, but not how |
Describe the issue:
In python 3.12 there seems to be an issue with directory handling that is not present in 3.11. In the very least it does behave differently. It seems to be ignoring some path specifications as well and created files in directories where there were none before, ignoring path specifications. Same numpy version, only difference is Python 3.12 instead of 3.11.6.
Likely related to
mesonbackend, and that I use a runtime parameter combination that may not have been used/tested before/otherwise.Maybe this can be fixed easily by someone familiar with the meson backend.
(or advise what I am doing wrong - that said, it does work with Python 3.11 and compatibility may be preferable)
Reproduce the code example:
Runtime information:
1.26.0
3.12.0 (main, Oct 7 2023, 10:56:13) [GCC 13.2.1 20230728 (Red Hat 13.2.1-1)]
Python 3.12built from source on Fedora 38,numpyinstalled frompip.Context for the issue:
My f2py module no longer compiles. Can't use Python 3.12.
The text was updated successfully, but these errors were encountered: