Skip to content

Use TemporaryFile for test code compilation #39646

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 6, 2025

Conversation

tobiasdiez
Copy link
Contributor

Use tempfile.TemporaryFile instead of Sage's own temporary file handler for the compilation of the test cython file.

This fixes errors during test collection with pytest:

conftest.py:129: in collect
    for test in finder.find(module, module.__name__):
../../../miniconda3/envs/sage-dev/lib/python3.12/doctest.py:948: in find
    self._find(tests, obj, name, module, source_lines, globs, {})
conftest.py:82: in _find
    super()._find(  # type:ignore[misc]
../../../miniconda3/envs/sage-dev/lib/python3.12/doctest.py:1022: in _find
    self._find(tests, val, valname, module, source_lines,
conftest.py:82: in _find
    super()._find(  # type:ignore[misc]
../../../miniconda3/envs/sage-dev/lib/python3.12/doctest.py:1010: in _find
    test = self._get_test(obj, name, module, globs, source_lines)
../../../miniconda3/envs/sage-dev/lib/python3.12/doctest.py:1092: in _get_test
    return self._parser.get_doctest(docstring, globs, name,
../../../miniconda3/envs/sage-dev/lib/python3.12/doctest.py:684: in get_doctest
    return DocTest(self.get_examples(string, name), globs,
../../../miniconda3/envs/sage-dev/lib/python3.12/doctest.py:698: in get_examples
    return [x for x in self.parse(string, name)
../../../miniconda3/envs/sage-dev/lib/python3.12/site-packages/sage/doctest/parsing.py:1111: in parse
    elif tag not in available_software:
../../../miniconda3/envs/sage-dev/lib/python3.12/site-packages/sage/doctest/external.py:489: in __contains__
    elif feature.is_present():
../../../miniconda3/envs/sage-dev/lib/python3.12/site-packages/sage/features/__init__.py:211: in is_present
    res = self._is_present()
../../../miniconda3/envs/sage-dev/lib/python3.12/site-packages/sage/features/__init__.py:924: in _is_present
    with open(tmp_filename(ext='.pyx'), 'w') as pyx:
../../../miniconda3/envs/sage-dev/lib/python3.12/site-packages/sage/misc/temporary_file.py:126: in tmp_filename
    handle, tmp = tempfile.mkstemp(prefix=name,
../../../miniconda3/envs/sage-dev/lib/python3.12/tempfile.py:357: in mkstemp
    return _mkstemp_inner(dir, prefix, suffix, flags, output_type)
../../../miniconda3/envs/sage-dev/lib/python3.12/tempfile.py:256: in _mkstemp_inner
    fd = _os.open(file, flags, 0o600)
E   FileNotFoundError: [Errno 2] No such file or directory: '/tmp/sage__ig56pe8/tmp_tjklyklm.pyx'

https://github.com/sagemath/sage/actions/runs/13718450041/job/38368348037?pr=39641#step:13:2369

📝 Checklist

  • The title is concise and informative.
  • The description explains in detail what this PR is about.
  • I have linked a relevant issue or discussion.
  • I have created tests covering the changes.
  • I have updated the documentation and checked the documentation preview.

⌛ Dependencies

Copy link

github-actions bot commented Mar 7, 2025

Documentation preview for this PR (built with commit 30a213f; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

@tobiasdiez tobiasdiez requested a review from orlitzky March 13, 2025 20:38
cython_import(pyx.name, verbose=-1)
with TemporaryFile(suffix='.pyx') as pyx:
pyx.write(self.test_code)
cython_import(pyx.name, verbose=-1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this will work on POSIX systems. The docs say,

The returned object is a true file object on POSIX platforms. On other platforms, it is a file-like object whose file attribute is the underlying true file object.

What it does on my machine:

>>> import tempfile
>>> with tempfile.TemporaryFile() as f:
...     print(f.name)
... 
3

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, it should have used NamedTemporaryFile. Strange thing is that on CI the Cython features bliss and sage.misc.cython were nevertheless correctly recognized.

sage: z
sage: from tempfile import NamedTemporaryFile
sage: context = { "z": 1}
sage: with NamedTemporaryFile(mode='w', suffix='.py') as file:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are the only one able to test it, but I would err on the side of caution and clean up the file explicitly here (i.e. set delete=False, and unlink it when done). The docs say that reopening an open file by name is tricky on Windows:

Opening the temporary file again by its name while it is still open works as follows... On Windows, make sure that at least one of the following conditions are fulfilled:

  • delete is false
  • additional open shares delete access (e.g. by calling os.open() with the flag O_TEMPORARY)
  • delete is true but delete_on_close is false. Note, that in this case the additional opens that do not share delete access (e.g. created via builtin open()) must be closed before exiting the context manager, else the os.unlink() call on context manager exit will fail with a PermissionError.

So ideally what we want to do is set delete_on_close=False... but the delete_on_close parameter was only added in python-3.12.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, that's a good point.

So ideally what we want to do is set delete_on_close=False... but the delete_on_close parameter was only added in python-3.12.

We can drop support for 3.11 this autumn and I would prefer to wait until then with improving windows support.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dropping 3.11 would be nice as well to enable the use of nanobind (the successor of pybind11)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but that would have to wait a few more months if we want to adhere to spec 0.

@tobiasdiez tobiasdiez requested a review from dimpase May 27, 2025 03:40
@dimpase
Copy link
Member

dimpase commented Jun 27, 2025

could you rebase this?

@tobiasdiez
Copy link
Contributor Author

could you rebase this?

Sure, done!

Copy link
Member

@dimpase dimpase left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@tobiasdiez
Copy link
Contributor Author

Thanks!

@vbraun vbraun merged commit 182a02d into sagemath:develop Jul 6, 2025
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants