FileNotFoundError - building 9.4.beta1

[strogdon]

 * Using python3.9 in global scope
 * python3_9: running distutils-r1_run_phase python_compile_all
/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/rings/padics/padic_base_leaves.py:1104: DeprecationWarning: invalid escape sequence \Z
  """
/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/rings/padics/padic_base_leaves.py:1139: DeprecationWarning: invalid escape sequence \Q
  """
 * ACCESS DENIED:  unlinkat:     /usr/share/sage/knotinfo
rm: cannot remove '/usr/share/sage/knotinfo': Permission denied
 * ACCESS DENIED:  mkdir:        /usr/share/sage/knotinfo
Traceback (most recent call last):
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/databases/knotinfo_db.py", line 420, in demo_version
    self._num_knots =  load(num_knots_file)
  File "sage/misc/persist.pyx", line 187, in sage.misc.persist.load (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/misc/persist.c:2807)
    with open(filename, 'rb') as fobj:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/share/sage/knotinfo/num_knots.sobj'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/sage_docbuild/__main__.py", line 1, in <module>
    from sage_docbuild import main
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/sage_docbuild/__init__.py", line 57, in <module>
    import sage.all
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/all.py", line 215, in <module>
    from sage.knots.all import *
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/knots/all.py", line 6, in <module>
    if DatabaseKnotInfo().is_present():
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/features/__init__.py", line 162, in is_present
    res = self._is_present()
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/features/__init__.py", line 823, in _is_present
    importlib.import_module(self.name)
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/knots/knotinfo.py", line 239, in <module>
    from sage.databases.knotinfo_db import KnotInfoColumns, db
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/databases/knotinfo_db.py", line 815, in <module>
    dc = db.columns()
  File "sage/misc/cachefunc.pyx", line 2310, in sage.misc.cachefunc.CachedMethodCallerNoArgs.__call__ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/misc/cachefunc.c:12918)
    self.cache = f(self._instance)
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/databases/knotinfo_db.py", line 594, in columns
    column_dict = self.read_column_dict()
  File "sage/misc/cachefunc.pyx", line 2310, in sage.misc.cachefunc.CachedMethodCallerNoArgs.__call__ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/misc/cachefunc.c:12918)
    self.cache = f(self._instance)
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/databases/knotinfo_db.py", line 617, in read_column_dict
    if self.demo_version():
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/databases/knotinfo_db.py", line 422, in demo_version
    self.reset_filecache()
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/databases/knotinfo_db.py", line 391, in reset_filecache
    sage_makedirs(sobj_path)
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/misc/misc.py", line 93, in sage_makedirs
    os.makedirs(dirname)
  File "/usr/lib/python3.9/os.py", line 225, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/usr/share/sage/knotinfo'
 * ERROR: sci-mathematics/sage-9999::sage-on-gentoo failed (compile phase):
 *   failed to produce html doc
 * 
 * Call stack:
 *     ebuild.sh, line  125:  Called src_compile
 *   environment, line 4717:  Called distutils-r1_src_compile
 *   environment, line 1844:  Called _distutils-r1_run_common_phase 'python_compile_all'
 *   environment, line  608:  Called multibuild_foreach_variant 'distutils-r1_run_phase' 'python_compile_all'
 *   environment, line 3609:  Called _multibuild_run 'distutils-r1_run_phase' 'python_compile_all'
 *   environment, line 3607:  Called distutils-r1_run_phase 'python_compile_all'
 *   environment, line 1832:  Called python_compile_all
 *   environment, line 3958:  Called die
 * The specific snippet of code:
 *           "${PYTHON}" sage_docbuild/__main__.py --no-pdf-links all html || die "failed to produce html doc";
 * 
 * If you need support, post the output of `emerge --info '=sci-mathematics/sage-9999::sage-on-gentoo'`,
 * the complete build log and the output of `emerge -pqv '=sci-mathematics/sage-9999::sage-on-gentoo'`.
 * The complete build log is located at '/var/tmp/portage/sci-mathematics/sage-9999/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/sci-mathematics/sage-9999/temp/environment'.
 * Working directory: '/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9'
 * S: '/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src'
 * --------------------------- ACCESS VIOLATION SUMMARY ---------------------------
 * LOG FILE: "/var/tmp/portage/sci-mathematics/sage-9999/temp/sandbox.log"
 * 
VERSION 1.0
FORMAT: F - Function called
FORMAT: S - Access Status
FORMAT: P - Path as passed to function
FORMAT: A - Absolute Path (not canonical)
FORMAT: R - Canonical Path
FORMAT: C - Command Line

F: unlinkat
S: deny
P: /usr/share/sage/knotinfo
A: /usr/share/sage/knotinfo
R: /usr/share/sage/knotinfo
C: rm -rf /usr/share/sage/knotinfo 

F: mkdir
S: deny
P: /usr/share/sage/knotinfo
A: /usr/share/sage/knotinfo
R: /usr/share/sage/knotinfo
C: /usr/bin/python3.9 sage_docbuild/__main__.py --no-pdf-links all html 

[kiwifb]

I am currently writing an email to sage-release about that.

[strogdon]

Of course, Prefix just chews merrily through this with no error. Who knows what the result will be - it's still building.

[kiwifb]

That's why it doesn't have visibility in vanilla.

[kiwifb]

https://groups.google.com/g/sage-release/c/wkObkoZQpe4/m/paCllwrKAQAJ

[kiwifb]

https://github.com/sagemath/sage/blob/9c85a6c1cfea365260c67ea69717c5831fa47980/src/sage/features/databases.py#L90 and after should have used a StaticFile structure not a PythonModule.

[kiwifb]

Experimenting with a fix.

[kiwifb]

Should take care of things. Now I have troubles with gap during doc building. Scanning more tickets...

[strogdon]

Without this issue's commit doc building finishes on Prefix.

[kiwifb]

Well, that's something. My machine is brand new and that's really the first build of sage-9999 and the doc on it. I wonder if I forgot to set some stuff in limits.

[strogdon]

Also try #629 to see if that is an issue?

[kiwifb]

It looks quite distinct

[categorie]  from /var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/doc/en/reference/categories/sage/categories/finite_coxeter_groups.rst:
[categorie] Traceback (most recent call last):
[categorie]   File "/usr/lib/python3.9/site-packages/matplotlib/sphinxext/plot_directive.py", line 483, in run_code
[categorie]     exec(code, ns)
[categorie]   File "<string>", line 1, in <module>
[categorie]   File "sage/misc/lazy_import.pyx", line 362, in sage.misc.lazy_import.LazyImport.__call__ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/misc/lazy_import.c:4041)
[categorie]     return self.get_object()(*args, **kwds)
[categorie]   File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/combinat/root_system/coxeter_group.py", line 135, in CoxeterGroup
[categorie]     return CoxeterMatrixGroup(cartan_type, base_ring, index_set)
[categorie]   File "sage/misc/classcall_metaclass.pyx", line 322, in sage.misc.classcall_metaclass.ClasscallMetaclass.__call__ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/misc/classcall_metaclass.c:1743)
[categorie]     return cls.classcall(cls, *args, **kwds)
[categorie]   File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/groups/matrix_gps/coxeter_group.py", line 230, in __classcall_private__
[categorie]     return super(CoxeterMatrixGroup, cls).__classcall__(cls,
[categorie]   File "sage/misc/cachefunc.pyx", line 1001, in sage.misc.cachefunc.CachedFunction.__call__ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/misc/cachefunc.c:6087)
[categorie]     w = self.f(*args, **kwds)
[categorie]   File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/structure/unique_representation.py", line 1007, in __classcall__
[categorie]     instance = typecall(cls, *args, **options)
[categorie]   File "sage/misc/classcall_metaclass.pyx", line 486, in sage.misc.classcall_metaclass.typecall (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/misc/classcall_metaclass.c:2207)
[categorie]     return (<PyTypeObject*>type).tp_call(cls, args, kwds)
[categorie]   File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/groups/matrix_gps/coxeter_group.py", line 273, in __init__
[categorie]     one = MS.one()
[categorie]   File "sage/misc/cachefunc.pyx", line 2310, in sage.misc.cachefunc.CachedMethodCallerNoArgs.__call__ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/misc/cachefunc.c:12918)
[categorie]     self.cache = f(self._instance)
[categorie]   File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/matrix/matrix_space.py", line 1608, in identity_matrix
[categorie]     A = self.zero_matrix().__copy__()
[categorie]   File "sage/misc/cachefunc.pyx", line 2310, in sage.misc.cachefunc.CachedMethodCallerNoArgs.__call__ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/misc/cachefunc.c:12918)
[categorie]     self.cache = f(self._instance)
[categorie]   File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/matrix/matrix_space.py", line 1766, in zero_matrix
[categorie]     zero = self.base_ring().zero()
[categorie]   File "sage/misc/cachefunc.pyx", line 2310, in sage.misc.cachefunc.CachedMethodCallerNoArgs.__call__ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/misc/cachefunc.c:12918)
[categorie]     self.cache = f(self._instance)
[categorie]   File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/rings/universal_cyclotomic_field.py", line 1397, in zero
[categorie]     return self.element_class(self, libgap.zero())
[categorie]   File "sage/libs/gap/libgap.pyx", line 618, in sage.libs.gap.libgap.Gap.zero (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/gap/libgap.c:6277)
[categorie]     return self(0)
[categorie]   File "sage/structure/parent.pyx", line 898, in sage.structure.parent.Parent.__call__ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/structure/parent.c:9339)
[categorie]     return mor._call_(x)
[categorie]   File "sage/structure/coerce_maps.pyx", line 161, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/structure/coerce_maps.c:4628)
[categorie]     raise
[categorie]   File "sage/structure/coerce_maps.pyx", line 156, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/structure/coerce_maps.c:4520)
[categorie]     return C._element_constructor(x)
[categorie]   File "sage/libs/gap/libgap.pyx", line 298, in sage.libs.gap.libgap.Gap._element_constructor_ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/gap/libgap.c:3391)
[categorie]     initialize()
[categorie]   File "sage/libs/gap/util.pyx", line 314, in sage.libs.gap.util.initialize (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/gap/util.c:5897)
[categorie]     with atomic_write(workspace) as f:
[categorie]   File "sage/libs/gap/util.pyx", line 316, in sage.libs.gap.util.initialize (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/gap/util.c:5852)
[categorie]     gap_eval('SaveWorkspace("{0}")'.format(f.name))
[categorie]   File "sage/libs/gap/util.pyx", line 398, in sage.libs.gap.util.gap_eval (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/gap/util.c:6068)
[categorie]     GAP_Enter()
[categorie] sage.libs.gap.util.GAPError: Error, reached the pre-set memory limit
[categorie] (change it with the -o command line option)
[function_] building [inventory]: targets for 11 source files that are out of date
[function_] updating environment: [new config] 11 added, 0 changed, 0 removed
[graphs   ] The inventory files are in ../../build_doc/inventory/en/reference/graphs.
Build finished. The built documents can be found in /var/tmp/portage/sci-mathematics/sage-9999/work/build_doc/inventory/en/reference/graphs
[functions] building [inventory]: targets for 20 source files that are out of date
[functions] updating environment: [new config] 20 added, 0 changed, 0 removed
[finite_ri] The inventory files are in ../../build_doc/inventory/en/reference/finite_rings.
Build finished. The built documents can be found in /var/tmp/portage/sci-mathematics/sage-9999/work/build_doc/inventory/en/reference/finite_rings
[function_] The inventory files are in ../../build_doc/inventory/en/reference/function_fields.
Build finished. The built documents can be found in /var/tmp/portage/sci-mathematics/sage-9999/work/build_doc/inventory/en/reference/function_fields
[game_theo] building [inventory]: targets for 6 source files that are out of date
[game_theo] updating environment: [new config] 6 added, 0 changed, 0 removed
[game_theo] The inventory files are in ../../build_doc/inventory/en/reference/game_theory.
Build finished. The built documents can be found in /var/tmp/portage/sci-mathematics/sage-9999/work/build_doc/inventory/en/reference/game_theory
[games    ] building [inventory]: targets for 5 source files that are out of date
[games    ] updating environment: [new config] 5 added, 0 changed, 0 removed
/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/symbolic/expression_conversions.py:946: DeprecationWarning: invalid escape sequence \|
  """
[categorie] The inventory files are in ../../build_doc/inventory/en/reference/categories.
Error building the documentation.
Traceback (most recent call last):
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/sage_docbuild/__main__.py", line 2, in <module>
    main()
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/sage_docbuild/__init__.py", line 1762, in main
    builder()
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/sage_docbuild/__init__.py", line 345, in _wrapper
    getattr(get_builder(document), 'inventory')(*args, **kwds)
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/sage_docbuild/__init__.py", line 577, in _wrapper
    self._build_everything_except_bibliography(lang, format, *args, **kwds)
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/sage_docbuild/__init__.py", line 563, in _build_everything_except_bibliography
    build_many(build_ref_doc, non_references)
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/sage_docbuild/__init__.py", line 297, in build_many
    _build_many(target, args, processes=processes)
  File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/sage_docbuild/utils.py", line 289, in build_many
    raise worker_exc.original_exception
OSError: /var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/categories/finite_coxeter_groups.py:docstring of sage.categories.finite_coxeter_groups.FiniteCoxeterGroups.ParentMethods.permutahedron:50: WARNING: Exception occurred in plotting finite_coxeter_groups-1

with sage.libs.gap.util.GAPError: Error, reached the pre-set memory limit being the root cause.

[kiwifb]

https://trac.sagemath.org/ticket/31205 and https://trac.sagemath.org/ticket/14779 are relevant to my case. Also https://trac.sagemath.org/ticket/29314.

[strogdon]

doc building completes here on Gentoo (with this issue's fix).

[strogdon]

Perhaps not important but I have dev-gap/GAPDoc-1.6.3 installed on computers where the docs build.

[kiwifb]

Could have an impact, I have done quite a bit of stuff in prep for gap 4.11.1 and a number of gap packages are updated as a consequence on this machine.

[kiwifb]

Fails in a different spot. Still related to gap. This is very interesting how this breaks in spite of what it says in the various PackageInfo.g files from the various gap packages that things only need very low versions of gap.

[kiwifb]

Reverting the update to primgrp and ctbllib did the trick. Not sure if one or the other was sufficient but pinning the gap minimum packages for a given version of gap may be a good idea.

[kiwifb]

This is now https://trac.sagemath.org/ticket/31921

[kiwifb]

Patch updated to reflect the new understanding of the problem, align it with the branch on trac and be usable with the new database_knotinfo ebuild I pushed a few minutes ago.

[strogdon]

I suspect this can be closed, Trac #31921 has been merged.

[kiwifb]

Yes indeed.