Modularization of sagelib: Break out separate packages sagemath-objects, sagemath-categories

[mkoeppe]

This ticket creates two experimental packages https://pypi.org/project/sagemath-objects/ and https://pypi.org/project/sagemath-categories/, outlined in Meta-ticket #29705, that consist of small subsets of the modules of sagelib, making Sage objects, the element/parent framework, categories, the coercion system and the related metaclasses available.

To build an sdist:

./sage -sh build/pkgs/sagemath_objects/spkg-src
./sage -sh build/pkgs/sagemath_categories/spkg-src

This creates a pip-installable source distribution sagemath-objects-VERSION.tar.gz (or sagemath-categories-VERSION.tar.gz) in upstream/.

The distributions depend only on Cython, pkgconfig, cysignals, gmpy2. The latter, of course, requires various headers and libraries, so they are not quite "purely" pip-installable packages.

The goal is for all modules of sagemath-objects and sagemath-categories to be importable even when sagelib is not installed. This can be tested (with tox) as follows:

./bootstrap && ./sage -sh -c '(cd build/pkgs/sagemath_objects/src && tox -v -v -e py39)'
./bootstrap && ./sage -sh -c '(cd build/pkgs/sagemath_categories/src && tox -v -v -e py39)'

sagemath-categories is intended to have sagemath-objects as a prerequisite. However, currently, sagemath-categories is an independent distribution that provides a superset of the modules of sagemath-objects.

Recombining it with the rest of sagelib via namespace package magic as outlined in #29705 is not implemented on this ticket. This will be done in follow-up tickets.

Depends on #29039
Depends on #32933

CC: @videlec @tscrim @orlitzky @nthiery @timokau @fchapoton @tobiasdiez @kwankyu @vbraun

Component: refactoring

Keywords: sd111

Author: Matthias Koeppe

Branch/Commit: 36d6b09

Reviewer: Kwankyu Lee

Issue created by migration from https://trac.sagemath.org/ticket/29865

[mkoeppe]

Branch: u/mkoeppe/t/29865-modularizarion-sage-objects

[sagetrac-git]

Commit: dd3dc15

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

889f0c6	sage.env.sage_include_directories: Do not fail if numpy cannot be imported
dd3dc15	build/pkgs/sage_objects: Add dependencies, get minimal tox working

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

4a99c1c	Add comment
b72264e	WIP: More stuff

[sagetrac-git]

Changed commit from dd3dc15 to b72264e

[sagetrac-git]

Changed commit from b72264e to 23aaec1

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

23aaec1

Version with sage.structure, sage.categories that builds OK; uses gmpy2

[mkoeppe]

Changed dependencies from #29864 to #29864, #29869

[sagetrac-git]

Changed commit from 23aaec1 to 29e30a4

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

cbc5555	sage.misc.misc, sage.misc.persist, sage.structure.formal_sum: Make some imports local to functions
0b7e11f	On the way to get sage.structure.all importing
19f86dc	sage.env: Do not die if DOT_SAGE is None
a795d56	sage.structure.factorization: Make import of sage.rings.integer.Integer local to methods
0ce9916	sage.structure.sequence: Make import of sage.misc.latex local to the latex method
fa22e19	sage.structure.all: Import .formal_sum lazily to avoid pulling in sage.modules
acf1ba3	sage.structure.factorization: Make import of sage.misc.misc_c.prod local to methods
14b1e55	sage.structure.formal_sum, .parent_gens: Fixup: Remove imports
a5453bf	Fixup: Add src/sage/misc/call.py
29e30a4	Merge branch 't/29869/move_attrcall_and_friends_from_sage_misc_misc_to_new_module_sage_misc_call' into t/29865-modularizarion-sage-objects

[mkoeppe]

comment:11

import sage.structure.all succeeds now

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

c4a68e3

build/pkgs/sage_objects/src/tox.ini: Add comments, test more

[sagetrac-git]

Changed commit from 29e30a4 to c4a68e3

[mkoeppe]

comment:13

Some of the Python modules in sage.categories seem to be sensitive to import order.

$ build/pkgs/sage_objects/src/.tox/python/bin/python -c 'import sage.categories.map'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "sage/categories/map.pyx", line 1, in init sage.categories.map (build/cythonized/sage/categories/map.c:16980)
  File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/build/pkgs/sage_objects/src/.tox/python/lib/python3.7/site-packages/sage/structure/__init__.py", line 2, in <module>
    import sage.structure.element
  File "sage/structure/parent.pxd", line 12, in init sage.structure.element (build/cythonized/sage/structure/element.c:34583)
  File "sage/categories/map.pxd", line 4, in init sage.structure.parent (build/cythonized/sage/structure/parent.c:27689)
AttributeError: module 'sage.categories.map' has no attribute 'Map'

vs.

$ build/pkgs/sage_objects/src/.tox/python/bin/python -c 'import sage.structure.all, sage.categories.map'
# succeeds

[sagetrac-git]

Changed commit from c4a68e3 to 7cca294

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

1963567	sage.categories.crystals: Make import of sage.misc.latex local to a method
c557714	sage.categories: Make imports from sage.rings, .sets, .combinat, .plot, .matrix local to methods
7cca294	Now we can import sage.categories.all

[mkoeppe]

comment:15

$ build/pkgs/sage_objects/src/.tox/python/bin/python -c 'import sage.structure.all, sage.categories.all'
# succeeds

but does not succeed without importing sage.structure.all first...

[mkoeppe]

comment:16

build/pkgs/sage_objects/src/.tox/python/bin/python -c 'import sage.structure.all, sage.categories.all, sage.categories.facade_sets; sage.categories.facade_sets.FacadeSets().example()'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "sage/misc/classcall_metaclass.pyx", line 322, in sage.misc.classcall_metaclass.ClasscallMetaclass.__call__ (build/cythonized/sage/misc/classcall_metaclass.c:1717)
  File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/build/pkgs/sage_objects/src/.tox/python/lib/python3.7/site-packages/sage/categories/category_with_axiom.py", line 1993, in __classcall__
    (base_category_class, axiom) = cls._base_category_class_and_axiom
  File "sage/misc/lazy_attribute.pyx", line 699, in sage.misc.lazy_attribute.lazy_class_attribute.__get__ (build/cythonized/sage/misc/lazy_attribute.c:2894)
  File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/build/pkgs/sage_objects/src/.tox/python/lib/python3.7/site-packages/sage/categories/category_with_axiom.py", line 1933, in _base_category_class_and_axiom
    base_category_class, axiom = base_category_class_and_axiom(cls)
  File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/build/pkgs/sage_objects/src/.tox/python/lib/python3.7/site-packages/sage/categories/category_with_axiom.py", line 1790, in base_category_class_and_axiom
    assert getattr(base_category_class, axiom, None) is cls, \
  File "sage/misc/lazy_import.pyx", line 498, in sage.misc.lazy_import.LazyImport.__get__ (build/cythonized/sage/misc/lazy_import.c:4236)
  File "sage/misc/lazy_import.pyx", line 188, in sage.misc.lazy_import.LazyImport.get_object (build/cythonized/sage/misc/lazy_import.c:2340)
  File "sage/misc/lazy_import.pyx", line 217, in sage.misc.lazy_import.LazyImport._get_object (build/cythonized/sage/misc/lazy_import.c:2482)
RuntimeError: resolving lazy import FacadeSets during startup

[sagetrac-git]

Changed commit from 7cca294 to 2c2ddb7

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

2c2ddb7

build/pkgs/sage_objects/src/setup.py: Remove unused imports