Add spkg-legacy-uninstall scripts for most standard packages #25140

embray · 2018-04-10T17:42:39Z

This is a followup to #25139 and represents work that was originally done in #22510 to convert most packages to use spkg-legacy-uninstall and remove uninstall-related commands from their spkg-install scripts.

This work was brought over from an older branch and still needs some adjustment: It should incorporate the changes from the #24024 metaticket which conflict with some of these changes. There are also some additional optional and experimental packages (and maybe a couple standard packages) that still need to be updated as well.

Probably won't bother with that until and unless #25139 is approved of in some form that still includes the spkg-legacy-uninstall feature.

To summarize what this is about: Part of the goal in implementing the new installation/uninstallation system for packages is that spkg-install should, ideally, not modify $SAGE_LOCAL in any way, at least for packages that support DESTDIR-based installation. This allows more separation from the building of packages from their runtime installation target.

However, many packages' spkg-install or spkg-build include commands (mostly in the form of rm -rf's) to clean up files from previous installs of those packages. This is no longer necessary for new-style package installations that track exactly what files are installed by a package. But it is still necessary when upgrading from old installs of packages that don't have a file manifest. This is what the spkg-legacy-uninstall script is for. It's just taking the rm -rf's from spkg-install and moving it into a separate script that is called by the uninstaller if it doesn't have a file manifest for that package.

So this ticket is just creating the spkg-legacy-uninstall for many packages that already have DESTDIR support.

Component: build

Keywords: uninstall

Author: Erik Bray

Branch/Commit: u/embray/build/spkg-uninstall/legacy-uninstallers @ e2d2b01

Reviewer: Julian Rüth

Issue created by migration from https://trac.sagemath.org/ticket/25140

The text was updated successfully, but these errors were encountered:

embray · 2018-04-10T17:49:32Z

Changed keywords from none to uninstall

embray · 2018-07-10T16:58:24Z

Changed dependencies from #25139 to none

sagetrac-git · 2018-07-10T17:22:35Z

Changed commit from 219150c to 58ad74e

sagetrac-git · 2018-07-10T17:22:35Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`58ad74e`	`Fix spkg-installs of packages with legacy uninstall commands; moving those commands into spkg-legacy-uninstall scripts`

embray · 2018-07-10T17:24:37Z

comment:5

As a reminder, the goal of this is to make it so that spkg-install scripts do not modify the contents of $SAGE_LOCAL directly.

sagetrac-git · 2018-07-11T16:46:32Z

Changed commit from 58ad74e to e6f189e

sagetrac-git · 2018-07-11T16:46:32Z

Branch pushed to git repo; I updated commit sha1. New commits:

`e6f189e`	`add spkg-legacy-uninstall for a few more packages that I missed`

embray · 2018-07-11T16:48:35Z

comment:7

Added spkg-legacy-uninstall for a few packages that could use it.

This is now done for at least most, if not all packages that have DESTDIR support from #24024. It doesn't make sense to do for packages that haven't been converted to the new install/uninstall method yet. However, for any remaining packages that need to be converted, they should also get an spkg-legacy-uninstall at the same time, where appropriate.

saraedum · 2018-07-17T15:28:12Z

comment:8

Feel free to set this to positive review if you are confident that this still works.

saraedum · 2018-07-17T15:28:12Z

Reviewer: Julian Rüth

embray · 2018-07-17T16:35:30Z

comment:9

The pyflakes plugin found an actual bug in the elliptic_curves package; it wasn't otherwise caught since I didn't bump the package versions for this (it isn't really necessary to).

I think I should fix that, and test doing a forced-reinstall of all these packages. I know I did that a long time ago back when this was part of #22510, but that was then...

sagetrac-git · 2018-07-17T16:37:01Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`b4e6851`	`Fix spkg-installs of packages with legacy uninstall commands; moving those commands into spkg-legacy-uninstall scripts`
`d45b4af`	`add spkg-legacy-uninstall for a few more packages that I missed`

sagetrac-git · 2018-07-17T16:37:01Z

Changed commit from e6f189e to d45b4af

embray · 2018-07-18T11:47:03Z

comment:11

I believe this issue can reasonably be addressed for Sage 8.4.

jdemeyer · 2018-08-07T12:24:17Z

comment:13

First of all, there are merge conflicts.

Second, do we really need this? I understand that you want to clean up the spkg-install scripts, but we can do that gradually as we upgrade packages anyway. I don't really see the point of adding a whole new feature only for backwards compatibility in cases where it almost never matters (typically, it's not a problem to skip the uninstall).

I'm not actively against this ticket, it's just that I don't see the point.

vbraun · 2018-08-12T09:05:45Z

Changed commit from bafe689 to none

vbraun · 2018-08-12T09:05:45Z

comment:19

When reinstalling an already installed elliptic_curves:

$ ./sage -p elliptic_curves
[...]
No record that 'elliptic_curves' was ever installed; skipping uninstall
Creating database /mnt/disk/home/release/Sage/local/share/cremona/cremona_mini.db
Traceback (most recent call last):
  File "spkg-install.py", line 89, in <module>
    install_cremona()
  File "spkg-install.py", line 31, in install_cremona
    con.execute('CREATE TABLE t_class(rank INTEGER, class TEXT PRIMARY KEY,'
sqlite3.OperationalError: table t_class already exists

real	0m0.161s
user	0m0.053s
sys	0m0.028s
************************************************************************
Error installing package elliptic_curves-0.8.p0
************************************************************************

This breaks incremental rebuilds...

vbraun · 2018-08-12T09:06:53Z

comment:20

Please check that all packages can be installed twice before settings back to positive review.

embray · 2018-08-14T15:45:02Z

New commits:

`801a2c3`	`Fix spkg-installs of packages with legacy uninstall commands; moving those commands into spkg-legacy-uninstall scripts`
`bafe689`	`add spkg-legacy-uninstall for a few more packages that I missed`

embray · 2018-08-14T15:45:02Z

Changed branch from bafe689 to u/embray/build/spkg-uninstall/legacy-uninstallers

embray · 2018-08-14T15:45:02Z

Commit: bafe689

sagetrac-git · 2018-08-14T16:05:43Z

Changed commit from bafe689 to 6f82064

sagetrac-git · 2018-08-14T16:05:43Z

Branch pushed to git repo; I updated commit sha1. New commits:

`6f82064`	`revert changes to elliptic_curves; it will be handled in a separate ticket`

embray · 2018-08-14T16:06:36Z

comment:23

Yes, there were still several problems with the spkg-install for elliptic_curves; I think it should just be updated in a separate ticket. It should not have been included in this ticket in the first place since it was only intending to update packages that already had working DESTDIR support.

New commits:

`6f82064`	`revert changes to elliptic_curves; it will be handled in a separate ticket`

embray · 2018-12-28T14:10:15Z

comment:25

Retargeting some of my tickets (somewhat optimistically for now).

sagetrac-git · 2019-01-09T17:45:48Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`e2d2b01`	`Fix spkg-installs of packages with legacy uninstall commands; moving those commands into spkg-legacy-uninstall scripts`

sagetrac-git · 2019-01-09T17:45:48Z

Changed commit from 6f82064 to e2d2b01

embray · 2019-01-09T17:46:56Z

comment:27

Rebased this, though some of these are now handled in other tickets so I need to clean it up a bit more (focus only on packages that already otherwise have been fixed as part of #24024).

embray · 2019-03-25T10:44:36Z

comment:28

Removing most of the rest of my open tickets out of the 8.7 milestone, which should be closed.

mkoeppe · 2023-12-08T03:20:56Z

Different versions of the spkg-legacy-uninstall scripts were added to Sage in

cb965d2 for boost_cropped
6a2596e for ecl
8635c52 for gfan
87c694f for jmol
6c88682 for libgd
0f3dab9 for numpy
0157c38 for python3

@embray

Most normal SPKGs are installed by staging in `DESTDIR`. When copying to the final install location in `SAGE_LOCAL`, an installation record is created, which is used later in package uninstallation. However, when SPKG installation was switched to using staging in `DESTDIR` (Meta-ticket sagemath#24024), the parts of `spkg-install` scripts that used to be responsible for removing an old version of the installed package were either kept in place or moved to `spkg-legacy-uninstall` scripts. This was done to enable incremental builds from older installations. By passage of time, this is no longer needed. Some of the removals are a partial cherry-pick from sagemath#25140 by @embray. We also switch `frobby` to `DESTDIR` staging and *add* an `spkg-legacy- uninstall` script. Resolves sagemath#25140. Resolves sagemath#30480.    ### 📝 Checklist     - [x] The title is concise, informative, and self-explanatory. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies  - Depends on sagemath#36778 (merged here to resolve merge conflict)  URL: sagemath#36839 Reported by: Matthias Köppe Reviewer(s): Dima Pasechnik

embray added this to the sage-8.2 milestone Apr 10, 2018

embray added the p: major / 3 label Apr 10, 2018

embray added the c: build label Apr 10, 2018

embray modified the milestones: sage-8.2, sage-8.3 Apr 26, 2018

embray added the s: needs review label Jul 10, 2018

embray added s: needs work and removed s: needs review labels Jul 17, 2018

embray modified the milestones: sage-8.3, sage-8.4 Jul 18, 2018

This comment has been minimized.

Sign in to view

embray added s: needs review and removed s: needs work labels Jul 24, 2018

jdemeyer added s: needs work and removed s: needs review labels Aug 7, 2018

vbraun closed this as completed Aug 11, 2018

vbraun reopened this Aug 12, 2018

embray modified the milestones: sage-8.4, sage-8.5 Oct 28, 2018

embray modified the milestones: sage-8.5, sage-8.7 Dec 28, 2018

embray self-assigned this Jan 9, 2019

embray removed this from the sage-8.7 milestone Mar 25, 2019

embray added the pending label Mar 25, 2019

This was referenced Jan 9, 2019

Enhanced package uninstallation for Sage spkgs #22510

Closed

Add sage-spkg-uninstall script and use it when possible to remove packages #25139

Closed

mkoeppe mentioned this issue Dec 8, 2023

Remove most spkg-legacy-uninstall scripts #36839

Merged

5 tasks

vbraun closed this as completed in #36839 Dec 19, 2023

mkoeppe added this to the sage-10.3 milestone Dec 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add spkg-legacy-uninstall scripts for most standard packages #25140

Add spkg-legacy-uninstall scripts for most standard packages #25140

embray commented Apr 10, 2018

embray commented Apr 10, 2018

embray commented Jul 10, 2018

sagetrac-git mannequin commented Jul 10, 2018

sagetrac-git mannequin commented Jul 10, 2018

embray commented Jul 10, 2018

sagetrac-git mannequin commented Jul 11, 2018

sagetrac-git mannequin commented Jul 11, 2018

embray commented Jul 11, 2018

saraedum commented Jul 17, 2018

saraedum commented Jul 17, 2018

embray commented Jul 17, 2018

sagetrac-git mannequin commented Jul 17, 2018

sagetrac-git mannequin commented Jul 17, 2018

embray commented Jul 18, 2018

This comment has been minimized.

jdemeyer commented Aug 7, 2018

vbraun commented Aug 12, 2018

vbraun commented Aug 12, 2018

vbraun commented Aug 12, 2018

embray commented Aug 14, 2018

embray commented Aug 14, 2018

embray commented Aug 14, 2018

sagetrac-git mannequin commented Aug 14, 2018

sagetrac-git mannequin commented Aug 14, 2018

embray commented Aug 14, 2018

embray commented Dec 28, 2018

sagetrac-git mannequin commented Jan 9, 2019

sagetrac-git mannequin commented Jan 9, 2019

embray commented Jan 9, 2019

embray commented Mar 25, 2019

mkoeppe commented Dec 8, 2023

Add spkg-legacy-uninstall scripts for most standard packages #25140

Add spkg-legacy-uninstall scripts for most standard packages #25140

Comments

embray commented Apr 10, 2018

embray commented Apr 10, 2018

embray commented Jul 10, 2018

sagetrac-git mannequin commented Jul 10, 2018

sagetrac-git mannequin commented Jul 10, 2018

embray commented Jul 10, 2018

sagetrac-git mannequin commented Jul 11, 2018

sagetrac-git mannequin commented Jul 11, 2018

embray commented Jul 11, 2018

saraedum commented Jul 17, 2018

saraedum commented Jul 17, 2018

embray commented Jul 17, 2018

sagetrac-git mannequin commented Jul 17, 2018

sagetrac-git mannequin commented Jul 17, 2018

embray commented Jul 18, 2018

This comment has been minimized.

jdemeyer commented Aug 7, 2018

vbraun commented Aug 12, 2018

vbraun commented Aug 12, 2018

vbraun commented Aug 12, 2018

embray commented Aug 14, 2018

embray commented Aug 14, 2018

embray commented Aug 14, 2018

sagetrac-git mannequin commented Aug 14, 2018

sagetrac-git mannequin commented Aug 14, 2018

embray commented Aug 14, 2018

embray commented Dec 28, 2018

sagetrac-git mannequin commented Jan 9, 2019

sagetrac-git mannequin commented Jan 9, 2019

embray commented Jan 9, 2019

embray commented Mar 25, 2019

mkoeppe commented Dec 8, 2023