mercurial --> plain text --> mercurial

[williamstein]

Robert Bradshaw has mostly solved this:

I've looked into this some more and it looks like we can completely
reconstruct a repository from the export of all its keywords. The
trick is to use the --exact keyword when importing. This forces it to
apply the given patch to the correct parent (sometimes creating a new
head) and will also correctly import merge patches (removing heads).
Some scripts to do this are up at

http://sage.math.washington.edu/home/robertwb/hg/

I've successfully exported and re-created simple repositories (with
branching) with these scripts, and it works great and preserves all
the history. The only issue is that I can't seem to get it to work
with any repositories older than a certain date. I think the issue is
that mercurial changed the way nodeid's are calculated (and I keep
getting an error "abort: patch is damaged or loses information" which
is thrown when the newly computed nodeid does not match the one in
the patch (command.py:1632 in 0.9.5)). Matt Mackall, any suggestions
on how to cleanly get around this/get the old node-id numbers instead?

- Robert Bradshaw

But there are issues. See the complete thread here:

http://groups.google.com/group/sage-devel/browse_thread/thread/79da4852b8e20851/c4b8e87260f08f96?#c4b8e87260f08f96

---

Apply attachment: trac_3052-makefile.patch or attachment: trac_3052-makefile-rebased.patch to the Sage root repository (in $SAGE_ROOT/).

Apply attachment: trac_3052-textify.patch to the base repository (in $SAGE_ROOT/spkg/base/).

Depends on #10594

Component: distribution

Author: Keshav Kini

Reviewer: Volker Braun

Merged: sage-4.7.2.alpha3

Issue created by migration from https://trac.sagemath.org/ticket/3052

[kini]

comment:1

Is this ticket still valid? I'm not sure I understand what exactly needs to be done with this, but it seems to apply to ancient mercurial versions and sage 2.x...

[kini]

comment:2

OK, William explained it to me simply - some users (especially, say, corporate environments) might be afraid of viruses in Mercurial's binary history data. This ticket is looking for a way to make our source distribution consist entirely of text files only.

[kini]

comment:3

I've written a python program that converts Mercurial 1.0+ bundles to a JSON representation and back. This doesn't produce a series of patch files, but it is a fully 7-bit-clean text format which is human-readable (though not as easy to grasp as a git diff or unified diff). It is also a bit-identically reversible computation and preserves all node IDs. The transformation from repository to bundle is also fully reversible as it is what Mercurial itself uses for pushes and pulls.

[kini]

apply to $SAGE_ROOT

[kini]

Attachment: trac_3052-makefile.patch.gz

Attachment: trac_3052-textify.patch.gz

apply to $SAGE_ROOT/spkg/base

[kini]

comment:4

So here are a couple of patches. Apply as indicated. The python file is not doctested, but most of the functions are pretty side-effectful so I don't see how exactly to do this. It at least does fail the doctester if it can't find the correct things to import, so that much is at least safeguarded (in case Mercurial is too old or too new, for example). Speaking of which, this patch depends on #10594 (merged in 4.7.2.alpha0).

Test this patch by unpacking a source distro tarball, doing make text-expand, make text-collapse, make, and make ptestlong (in that order, of course). Works for me on sage.math.washington.edu .

[nexttime]

Dependencies: #10594

[vbraun]

Author: Keshav Kini

[vbraun]

Reviewer: Volker Braun

[vbraun]

comment:6

Sounds good, positive review.

Of course there is still a 35MB spkg containing various OSX Fortran compilers, thats a lot of binary code ;) Hopefully apple will fix their toolchain at one point...

[kini]

comment:8

Thanks for the review! :)

[nexttime]

comment:10

Looks like Jeroen's merger doesn't yet support the base repository... :(

[nexttime]

Merged: sage-4.7.2.alpha3

[jdemeyer]

Attachment: trac_3052-makefile-rebased.patch.gz

[nexttime]

comment:13

I rebased all patches with fuzz 2, but didn't bother about this one because it was so trivial (or obvious). ;-)

[kini]

comment:14

BTW this code imports stuff from Mercurial internals so it may need to be updated when we next upgrade Mercurial. Any ideas on how to make a doctest that will break when Mercurial is upgraded so that this will be noticed when the time comes? I guess there's always the trivial

>>> import mercurial.__version__.version
>>> mercurial.__version__.version
'1.8.4'

But even if we use this silly doctest, where should it go? $SAGE_ROOT/spkg/base is not doctested by make ptestlong.

[williamstein]

comment:15

Odd tests go in SAGE_ROOT/devel/sage/sage/tests.

[nexttime]

comment:16

Perhaps we could tag such tests # optional - release such they get only run by developers and the release manager(s) or release scripts. Then we could really test whether text-expand and text-collapse are still functional (and not just test for a Mercurial version, which is a bit odd).