Metaticket: revamp workflow / switch to git #13015

kini · 2012-05-25T20:12:53Z

We plan to revamp the development workflow for Sage, involving a few separate (big) changes:

switch to git
consolidate our repositories into a single repository
simplified development interface

We break down the steps needed for this transition in the following tasks/tickets:

switch sage to the new directory layout #14480 - switch to new directory layout and update build system
document the new workflow #14481 - write a new developer manual; see https://github.com/sagemath/git-developer-guide
create the new development scripts/library #14482 - write development scripts
Make sure the patchbot works with the new layout (this seems to be okay).
Make sure the buildbot works with the new layout (VB: works now)
make sure all optional packages work in the git repository #14962 - Make sure optional spkgs work with the new layout.
Add editor- and os-specific temporary files to .gitignore #15241 - ignore various temporary files
"make download" target for sage #15222 - make download to download all tarballs
Sage should re-build packages when versions change. #15221 - always rebuild packages if the version number changes
Build git without gettext if necessary, spkg-check sanity #15401 - Build git without gettext if necessary
Borken git doctests. #15420 - Borken git doctests.
Enable cython caching #15430 - Enable cython caching
Cksum uses tabs instead of spaces, breaking sage-spkg's regex #15436 - Cksum uses tabs instead of spaces, breaking sage-spkg's regex
"make distclean" fails on Solaris #15458 - "make distclean" fails on Solaris
Allow using Sage without dev/git #15465 - Allow using Sage without git repository
update to latest upstream git #15468 - update to latest upstream git
Symmetrica spkg-check fails due to whitespace #15469 - Symmetrica spkg-check fails due to whitespace
Fixes for the source tarball generation (sage -sdist) #15472 - Fixes for the source tarball generation (sage -sdist)
Store actual tarball name in pkg metadata. #14968 - Store actual tarball name in pkg metadata.
Run sage-upgrade *outside* of the Sage shell #15517 - Run sage-upgrade outside of the Sage shell

Instructions for using the new workflow: QuickStartSageGit

Depends on #15517
Depends on #14968
Depends on #15472
Depends on #15468
Depends on #15469
Depends on #15465
Depends on #15458
Depends on #15436
Depends on #15430
Depends on #15420
Depends on #15401
Depends on #14480
Depends on #14481
Depends on #14482
Depends on #14962
Depends on #15241
Depends on #15222
Depends on #15221

CC: @koffie @jdemeyer @roed314 @ohanar @robertwb @burcin @ppurka @saraedum @nexttime @dimpase

Component: misc

Keywords: git workflow, sd51

Branch/Commit: u/hivert/master @ 68680e9

Issue created by migration from https://trac.sagemath.org/ticket/13015

The text was updated successfully, but these errors were encountered:

ohanar · 2012-05-31T08:19:31Z

Dependencies: #9906, #11080

jdemeyer · 2012-06-01T22:20:42Z

Changed dependencies from #9906, #11080 to #12938, #9906, #11080

ohanar · 2012-06-01T22:52:41Z

comment:8

Those four spkgs are treated differently in the git layout, so the current (or future) spkg-install scripts are irrelevant.

ohanar · 2012-06-01T22:52:41Z

Changed dependencies from #12938, #9906, #11080 to #9906, #11080

ohanar · 2012-06-16T08:56:29Z

Changed dependencies from #9906, #11080 to #9906, #11080, #13123

ohanar · 2012-09-06T09:05:10Z

Changed dependencies from #9906, #11080, #13123 to #9906, #11080, #13031, #13032, #13123, #13432

ohanar · 2012-09-12T00:33:00Z

Changed dependencies from #9906, #11080, #13031, #13032, #13123, #13432 to #9906, #11080, #13031, #13032, #13123, #13432, #13448

robertwb · 2013-03-28T00:56:43Z

Changed dependencies from #9906, #11080, #13031, #13032, #13123, #13432, #13448 to #9906, #11080, #13031, #13032, #13123, #13432, #13448, #14372

ohanar · 2013-03-28T17:41:35Z

Changed dependencies from #9906, #11080, #13031, #13032, #13123, #13432, #13448, #14372 to #9906, #11080, #13031, #13032, #13123, #13432, #13448, #14372, #14226, #14316, #14331

ohanar · 2013-03-28T17:41:35Z

comment:15

These are the patches that I'm importing that are not merged in 5.9.beta1: #14226, #13031, #14316, #14331.

So those are the most vital.

ohanar · 2013-04-24T05:56:11Z

Changed dependencies from #9906, #11080, #13031, #13032, #13123, #13432, #13448, #14372, #14226, #14316, #14331 to #9906, #11080, #13031, #13032, #13123, #13432, #14226, #14316, #14331, #14480, #14481, #14482

tkluck · 2013-07-23T09:25:23Z

Changed dependencies from #9906, #11080, #13031, #13032, #13123, #13432, #14226, #14316, #14331, #14480, #14481, #14482 to #14480, #14481, #14482, #14330, #14273, #14954

tkluck · 2013-07-23T09:25:23Z

comment:19

Updating dependencies field to remove circular dependencies etc.

tkluck · 2013-07-23T10:07:15Z

Changed keywords from git workflow to git workflow, sd51

tkluck · 2013-07-23T10:07:15Z

comment:20

I'm adding a branch to this that is just the merge of all dependencies.

tkluck · 2013-07-23T10:07:15Z

Branch: u/tkluck/master

tkluck · 2013-07-24T05:59:44Z

Changed branch from u/tkluck/master to public/sage-git/master

tkluck · 2013-07-24T12:14:53Z

Changed dependencies from #14480, #14481, #14482, #14330, #14273, #14954 to #14480, #14481, #14482, #14330, #14273, #14954, #14962

sagetrac-git · 2013-09-03T14:03:19Z

Branch pushed to git repo; I updated commit sha1. New commits:

`[changeset:d8713eb]`	`Merge remote-tracking branch 'origin/build_system' into public/sage-git/master`

sagetrac-git · 2013-09-03T14:03:19Z

Commit: d8713eb

sagetrac-git · 2013-09-06T16:33:06Z

Branch pushed to git repo; I updated commit sha1. New commits:

`[changeset:b890215]`	`Merge branch 'ticket/14482' into public/sage-git/master`
`[changeset:6bb1502]`	`Fixing doctests and adding some coverage`
`[changeset:90d7a64]`	`Fix some doctest problems`
`[changeset:393d8bb]`	`Fix a bug in unstash`
`[changeset:4281985]`	`Merge branch 'u/saraedum/ticket/14482' of ssh://trac.sagemath.org:2222/sage into ticket/14482`
`[changeset:1183a8b]`	`Fixing doctest errors, adding functions for changing status of tickets, making e-mail notifications happen`
`[changeset:348e0a0]`	`"dependencies" is not a valid name for a branch since this is a special keyword for merge().`
`[changeset:e793b41]`	`Implemented 'dependencies' in merge()`
`[changeset:9569608]`	`Several trivial doctest fixes in sage dev scripts.`
`[changeset:fa96127]`	`Delete dependencies when abandoning a ticket.`
`[changeset:736295c]`	`Stashes were printed twice by unstash()`
`[changeset:7191281]`	`Merge branch 'u/roed/ticket/14482' of ssh://trac.sagemath.org:2222/sage into ticket/14482`
`[changeset:d7b6620]`	`Changed the default behaviour for merge() with respect to remote branches.`
`[changeset:cfc472d]`	`Fixed _is_remote_branch_name when exists!=None.`
`[changeset:c0648c7]`	`Update documentation of import_patch and download_patch`
`[changeset:b38fe65]`	`Merge branch 'stash/1' into ticket/14482`
`[changeset:8b48c3c]`	`Fix some typos`
`[changeset:aa642da]`	`Merge branch 'u/saraedum/ticket/14482' of ssh://trac.sagemath.org:2222/sage into ticket/14482`
`[changeset:4aebcfa]`	`Changes stashed by reset_to_clean_working_directory()`
`[changeset:fdaa995]`	`More error checking on ticket edits, add a set_attributes function`
`[changeset:51c04ac]`	`Use --work-tree in git_interface.`
`[changeset:a36c492]`	`Fixed an optional internet doctest in sagedev.`
`[changeset:38636b0]`	`Fix typo in unstash`
`[changeset:4828a9a]`	`Revert "Make sure we do not touch files outside of sage in the dev scripts."`
`[changeset:ae2fde5]`	`Fixed edit_ticket().`
`[changeset:03cd3fd]`	`Fixed import_patch()`
`[changeset:15d9e77]`	`Fixing typos, making small changes`
`[changeset:9e3a961]`	`Fixed a question in import_patch()`
`[changeset:0b04731]`	`Fixed a typo in import_patch()`
`[changeset:a625996]`	`Fixed a typo in import_patch()`
`[changeset:f63d614]`	`Removed a reference to /dev/shm/ from sagedev.py`
`[changeset:167fe88]`	`Moved hg patch related methods to patch.py`
`[changeset:76b09f0]`	`Make download_patch and import_patch download and apply patches like the patchbot does.`
`[changeset:73410f3]`	`Fixed two typos in docstrings/comments`
`[changeset:6061772]`	`fixed a TracConnectionError() call`
`[changeset:f95ece6]`	`Fixed two optional: internet tests`
`[changeset:6df1b96]`	`Merge branch 'u/roed/ticket/14482' of ssh://trac.sagemath.org:2222/sage into ticket/14482`
`[changeset:4072ade]`	`download() did not behave as advertised in the docstring`
`[changeset:5f85692]`	`Fixed an incorrect help string in upload()`
`[changeset:453486f]`	`Fixed an incorrect help string in download()`
`[changeset:979fcd4]`	`Merge branch 'u/saraedum/ticket/14482' of ssh://trac.sagemath.org:2222/sage into ticket/14482`
`[changeset:c575cca]`	`Added capability to view ticket info and comments via dev scripts`
`[changeset:8052e27]`	`Make sure we do not touch files outside of sage in the dev scripts.`
`[changeset:f43d853]`	`Corrected the _src directory in the git interface`
`[changeset:6b64c54]`	`switch_ticket()/create_ticket() works with an unclean working directory in some cases.`
`[changeset:4275bbd]`	`Fixed doctests if no user.name and user.email had been set in git`
`[changeset:e078628]`	`Add a warning in upload() to not erase dependencies.`
`[changeset:51e2179]`	`merge() fails when merging a ticket into itself`
`[changeset:ed8ed9d]`	`Fixed an infinite recursion in download_patch().`
`[changeset:0f30605]`	`Fixed a doctest for download_patch()`
`[changeset:d9d7fe4]`	`Merge branch 'stash/1' into ticket/14482`
`[changeset:7c934d2]`	`Changes stashed by reset_to_clean_working_directory()`
`[changeset:debc6ed]`	`Added a doctest to sagedev which shows how to start afresh when working on a ticket.`
`[changeset:c68940a]`	`Delete branch -> ticket association when abandon()ing a branch in sagedev.`
`[changeset:bc5ce5a]`	`Moved much of the code of create_ticket() to switch_ticket()`
`[changeset:2167665]`	`Make user interface for sagedev more tolerant.`
`[changeset:859987e]`	`Forbid None, True, False as branch names`
`[changeset:7b0a041]`	`Fall back to http if ssl is not available`
`[changeset:07dfcef]`	`Be more verbose about whether a commit has been created or not in sagedev`
`[changeset:b8c78f2]`	`Hide git debug messages per default in sagedev.`
`[changeset:1ba2a7a]`	`fixed a typo in sagedev.py`
`[changeset:a9c8a91]`	`Merge branch 'build_system' into dev_scripts`
`[changeset:5adc4a4]`	`Store the data of the SavingDicts of SageDev in .git`
`[changeset:957a415]`	`Small fix to legacy support for SavingDicts in SageDev`
`[changeset:8b0d31e]`	`Fixed a doctest in trac_interface.py`
`[changeset:b8dccf1]`	`SavingDicts for SageDev are now stored in the working tree`
`[changeset:c66bdfd]`	`fixed two problems in SageDev.commit()`
`[changeset:a773f54]`	`Merge branch 'build_system' into ticket/14482`

sagetrac-git · 2013-09-06T16:33:06Z

Changed commit from d8713eb to b890215

vbraun · 2013-11-19T03:44:28Z

Changed dependencies from #15430, #15420, #15401, #14480, #14481, #14482, #14330, #14273, #14962, #15241, #15222, #15221 to #15436, #15430, #15420, #15401, #14480, #14481, #14482, #14330, #14273, #14962, #15241, #15222, #15221

vbraun · 2013-11-27T01:13:48Z

Changed dependencies from #15436, #15430, #15420, #15401, #14480, #14481, #14482, #14330, #14273, #14962, #15241, #15222, #15221 to #15458, #15436, #15430, #15420, #15401, #14480, #14481, #14482, #14330, #14273, #14962, #15241, #15222, #15221

sagetrac-git · 2013-11-27T05:20:30Z

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

`[0c6fcdf](https://github.com/sagemath/sagetrac-mirror/commit/0c6fcdf)`	`Merge in sage-5.13.beta4`
`[47cb70d](https://github.com/sagemath/sagetrac-mirror/commit/47cb70d)`	`[FIXUP] 5.13.beta4: update git specific files`
`[83ea767](https://github.com/sagemath/sagetrac-mirror/commit/83ea767)`	`Merge branch 'upstream' into build_system`
`[b27454a](https://github.com/sagemath/sagetrac-mirror/commit/b27454a)`	`Merge 'sage-5.13.beta4'`
`[d8b9438](https://github.com/sagemath/sagetrac-mirror/commit/d8b9438)`	`5.13.beta4`
`[7fa6d9b](https://github.com/sagemath/sagetrac-mirror/commit/7fa6d9b)`	`5.13.beta4`
`[46ec651](https://github.com/sagemath/sagetrac-mirror/commit/46ec651)`	`Trac #15402: fix eint1() precision`
`[6d844cf](https://github.com/sagemath/sagetrac-mirror/commit/6d844cf)`	`#14706: fix visibility of symbols for built-in libintl.`
`[79daab2](https://github.com/sagemath/sagetrac-mirror/commit/79daab2)`	`#14706: Move previous install of R on OS X.`
`[39238cd](https://github.com/sagemath/sagetrac-mirror/commit/39238cd)`	`#14706: Add $SAGE_LOCAL/lib to LDFLAGS.`

sagetrac-git · 2013-11-27T05:20:30Z

Changed commit from f36d14a to 0c6fcdf

vbraun · 2013-11-28T20:47:14Z

Changed dependencies from #15458, #15436, #15430, #15420, #15401, #14480, #14481, #14482, #14330, #14273, #14962, #15241, #15222, #15221 to #15465, #15458, #15436, #15430, #15420, #15401, #14480, #14481, #14482, #14330, #14273, #14962, #15241, #15222, #15221

vbraun · 2013-11-29T18:14:04Z

Changed dependencies from #15465, #15458, #15436, #15430, #15420, #15401, #14480, #14481, #14482, #14330, #14273, #14962, #15241, #15222, #15221 to #15468, #15469, #15465, #15458, #15436, #15430, #15420, #15401, #14480, #14481, #14482, #14330, #14273, #14962, #15241, #15222, #15221

vbraun · 2013-11-30T18:57:01Z

Changed dependencies from #15468, #15469, #15465, #15458, #15436, #15430, #15420, #15401, #14480, #14481, #14482, #14330, #14273, #14962, #15241, #15222, #15221 to #15472, #15468, #15469, #15465, #15458, #15436, #15430, #15420, #15401, #14480, #14481, #14482, #14330, #14273, #14962, #15241, #15222, #15221

vbraun · 2013-12-01T13:45:16Z

Changed dependencies from #15472, #15468, #15469, #15465, #15458, #15436, #15430, #15420, #15401, #14480, #14481, #14482, #14330, #14273, #14962, #15241, #15222, #15221 to #14968, #15472, #15468, #15469, #15465, #15458, #15436, #15430, #15420, #15401, #14480, #14481, #14482, #14330, #14273, #14962, #15241, #15222, #15221

vbraun · 2013-12-02T11:48:15Z

Changed dependencies from #14968, #15472, #15468, #15469, #15465, #15458, #15436, #15430, #15420, #15401, #14480, #14481, #14482, #14330, #14273, #14962, #15241, #15222, #15221 to #14968, #15472, #15468, #15469, #15465, #15458, #15436, #15430, #15420, #15401, #14480, #14481, #14482, #14962, #15241, #15222, #15221

sagetrac-git · 2013-12-06T18:59:59Z

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

`[1e9e230](https://github.com/sagemath/sagetrac-mirror/commit/1e9e230)`	`Merge in sage-5.13.beta5`
`[33dec52](https://github.com/sagemath/sagetrac-mirror/commit/33dec52)`	`[FIXUP] 5.13.beta5: update git specific files`
`[47de785](https://github.com/sagemath/sagetrac-mirror/commit/47de785)`	`Merge branch 'upstream' into build_system`
`[0b16085](https://github.com/sagemath/sagetrac-mirror/commit/0b16085)`	`Merge 'sage-5.13.beta5'`
`[f715110](https://github.com/sagemath/sagetrac-mirror/commit/f715110)`	`5.13.beta5`
`[dfdb305](https://github.com/sagemath/sagetrac-mirror/commit/dfdb305)`	`5.13.beta5`
`[374a99d](https://github.com/sagemath/sagetrac-mirror/commit/374a99d)`	`#14410: let ATLAS build shared libs on Cygwin.`
`[b8fc170](https://github.com/sagemath/sagetrac-mirror/commit/b8fc170)`	`Trac #15385: cleanup of hadamard matrices`
`[4f6874d](https://github.com/sagemath/sagetrac-mirror/commit/4f6874d)`	`Trac #15174: permutations multiplication order made more explicit and less of a minefield; also other fixes to permutation.py`
`[b00bb0d](https://github.com/sagemath/sagetrac-mirror/commit/b00bb0d)`	`Trac #13849: deprecate degrevlex`

sagetrac-git · 2013-12-06T18:59:59Z

Changed commit from 0c6fcdf to 1e9e230

sagetrac-git · 2013-12-12T05:59:42Z

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

`[4b0912c](https://github.com/sagemath/sagetrac-mirror/commit/4b0912c)`	`Merge in sage-5.13.rc0`
`[cd237e7](https://github.com/sagemath/sagetrac-mirror/commit/cd237e7)`	`[FIXUP] 5.13.rc0: update git specific files`
`[9fd8301](https://github.com/sagemath/sagetrac-mirror/commit/9fd8301)`	`Merge branch 'upstream' into build_system`
`[12b4002](https://github.com/sagemath/sagetrac-mirror/commit/12b4002)`	`Merge 'sage-5.13.rc0'`
`[702cbc1](https://github.com/sagemath/sagetrac-mirror/commit/702cbc1)`	`5.13.rc0`
`[fb29132](https://github.com/sagemath/sagetrac-mirror/commit/fb29132)`	`5.13.rc0`
`[0ad64ab](https://github.com/sagemath/sagetrac-mirror/commit/0ad64ab)`	Trac #15433: define `__ACCELERATE__` on OS X
`[4f31c50](https://github.com/sagemath/sagetrac-mirror/commit/4f31c50)`	`Trac #15433: don't build sqlite on OS X 10.9`
`[1b706ff](https://github.com/sagemath/sagetrac-mirror/commit/1b706ff)`	`Trac 15433: work around for incompatibility between g++ and an OS X header`
`[c31654f](https://github.com/sagemath/sagetrac-mirror/commit/c31654f)`	`Trac #15150: Implemented NCSym.`

sagetrac-git · 2013-12-12T05:59:42Z

Changed commit from 1e9e230 to 4b0912c

vbraun · 2013-12-16T14:12:19Z

Changed dependencies from #14968, #15472, #15468, #15469, #15465, #15458, #15436, #15430, #15420, #15401, #14480, #14481, #14482, #14962, #15241, #15222, #15221 to #15517, #14968, #15472, #15468, #15469, #15465, #15458, #15436, #15430, #15420, #15401, #14480, #14481, #14482, #14962, #15241, #15222, #15221

sagetrac-git · 2013-12-16T16:47:02Z

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

`[68680e9](https://github.com/sagemath/sagetrac-mirror/commit/68680e9)`	`Updated Sage version to 6.0.rc0`
`[93cd08f](https://github.com/sagemath/sagetrac-mirror/commit/93cd08f)`	`Trac #15440: Random spaces injected into Maxima pexpect output`
`[ef54138](https://github.com/sagemath/sagetrac-mirror/commit/ef54138)`	`Trac #14968: Store actual tarball name in pkg metadata.`
`[b6b4adb](https://github.com/sagemath/sagetrac-mirror/commit/b6b4adb)`	`updated git checksum`
`[a25d736](https://github.com/sagemath/sagetrac-mirror/commit/a25d736)`	`updated R checksum`
`[9d08fd5](https://github.com/sagemath/sagetrac-mirror/commit/9d08fd5)`	`Trac #15472: Fixes for the source tarball generation (sage -sdist)`
`[a73f7ca](https://github.com/sagemath/sagetrac-mirror/commit/a73f7ca)`	`Trac #15469: Symmetrica spkg-check fails due to whitespace`
`[ee17952](https://github.com/sagemath/sagetrac-mirror/commit/ee17952)`	`Trac #15468: update to latest upstream git`
`[608159c](https://github.com/sagemath/sagetrac-mirror/commit/608159c)`	`Trac #15465: Allow using Sage without dev/git`
`[e1a7d2a](https://github.com/sagemath/sagetrac-mirror/commit/e1a7d2a)`	`Trac #15458: "make distclean" fails on Solaris`

sagetrac-git · 2013-12-16T16:47:02Z

Changed commit from 4b0912c to 68680e9

vbraun · 2013-12-16T23:59:55Z

comment:63

All tickets are merged! woo-hoo ;-)

hivert · 2015-05-08T15:47:23Z

Changed branch from master to u/hivert/master

kini added the p: major / 3 label May 25, 2012

kini assigned jasongrout May 25, 2012

kini added this to the sage-6.0 milestone May 25, 2012

This comment has been minimized.

Sign in to view

vbraun closed this as completed Dec 16, 2013

kini mentioned this issue Mar 3, 2018

Metaticket: clean up SPKGs #13025

Closed

ohanar mentioned this issue Dec 16, 2013

switch sage to the new directory layout #14480

Closed

vbraun mentioned this issue Dec 29, 2022

Fix buildbot random failures #15459

Closed

Metaticket: revamp workflow / switch to git #13015

Metaticket: revamp workflow / switch to git #13015

Comments

kini commented May 25, 2012

This comment has been minimized.

ohanar commented May 31, 2012

jdemeyer commented Jun 1, 2012

ohanar commented Jun 1, 2012

ohanar commented Jun 1, 2012

ohanar commented Jun 16, 2012

ohanar commented Sep 6, 2012

This comment has been minimized.

ohanar commented Sep 12, 2012

This comment has been minimized.

robertwb commented Mar 28, 2013

ohanar commented Mar 28, 2013

ohanar commented Mar 28, 2013

ohanar commented Apr 24, 2013

This comment has been minimized.

tkluck commented Jul 23, 2013

tkluck commented Jul 23, 2013

tkluck commented Jul 23, 2013

tkluck commented Jul 23, 2013

tkluck commented Jul 23, 2013

tkluck commented Jul 24, 2013

tkluck commented Jul 24, 2013

sagetrac-git mannequin commented Sep 3, 2013

sagetrac-git mannequin commented Sep 3, 2013

This comment has been minimized.

sagetrac-git mannequin commented Sep 6, 2013

sagetrac-git mannequin commented Sep 6, 2013

vbraun commented Nov 19, 2013

This comment has been minimized.

This comment has been minimized.

vbraun commented Nov 27, 2013

This comment has been minimized.

sagetrac-git mannequin commented Nov 27, 2013

sagetrac-git mannequin commented Nov 27, 2013

This comment has been minimized.

vbraun commented Nov 28, 2013

This comment has been minimized.

vbraun commented Nov 29, 2013

This comment has been minimized.

vbraun commented Nov 30, 2013

This comment has been minimized.

vbraun commented Dec 1, 2013

vbraun commented Dec 2, 2013

sagetrac-git mannequin commented Dec 6, 2013

sagetrac-git mannequin commented Dec 6, 2013

sagetrac-git mannequin commented Dec 12, 2013

sagetrac-git mannequin commented Dec 12, 2013

vbraun commented Dec 16, 2013

This comment has been minimized.

sagetrac-git mannequin commented Dec 16, 2013

sagetrac-git mannequin commented Dec 16, 2013

vbraun commented Dec 16, 2013

hivert commented May 8, 2015