Add OutputGzipFile, in order to create a gzip compressed file

[ChrisJefferson]

Continues #4110, as I stupidly put that branch in gap-system/gap, and shouldn't do that. All suggested fixes from that patch have been applied here.

[coveralls]

Coverage decreased (-0.0009%) to 93.724% when pulling f73982b on ChrisJefferson:raw-file into 0f8f080 on gap-system:master.

[wilfwilson]

Which PR are you referring to? #4410 doesn't seem to exist on gap-system/gap (which is here).

[ChrisJefferson]

Sorry, meant #4110 (which I've corrected in the first message, in case anyone else comes along)

[ChrisJefferson]

There was a failing test which is, I believe, unrelated to this PR but I have fixed here anyway -- ReadLine can get only a partial line when reading from a process which hasn't yet finished outputting a whole line.

[fingolfin]

Since I've run into the PackageManager issue myself now (paging @mtorpey), I've thought about this PR. First off, for PackageManager, I suggest that it directly downloads into a file when possible, sidestepping the double compression issue (see also gap-packages/PackageManager#59).

But overall, I am scared that the double compression bug will hit more people; while the automatic gzip compression is a cool feature, it's also a trap inviting regressions and weird bugs.

This PR attempts to mitigate it by at least adding a way to bypass the issue (so an opt-out solution). But this strikes me as the wrong approach: It still breaks existing code which needs to be adapted (and then the adaption means that old code won't work with old GAP versions). Shouldn't we rather return the default behavior to what it was (i.e., no automatic compression), and then add a way to opt-in to compression? E.g. via a GzipOutputStream, which then also would work with arbitrary extensions?

[fingolfin]

At the very least, we should remove the automatic compression from the stable-4.11 branch, so that if we ever make a 4.11.1 release it doesn't regress existing code; if we decide we want the regression, it should be in 4.12.0.

[ChrisJefferson]

Now significantly changed based on feedback (although the internals are similar).

Remove all 'Raw' methods. Add 'OutputGzipFile' (to match OutputTextFile), which gzips anything. OutputTextFile now never compresses.

InputTextFile still always uncompresses (as it has for many years, although now it does it more situations). Even if this was disabled, you still couldn't read gzipped files with InputTextFile, because we have always (and we have no way to disable currently) rewriten line endings in windows, which would break binary files.

[fingolfin]

Hmm, wait, isn't this supposed to have been changed with this?

[ChrisJefferson]

Sorry, your message is too brief for me to guess the problem?

My aim is now OutputTextFile has it's 4.10 behaviour -- it never gzips, and you use OutputGzipFile explicitly if you want to output with gzip compression.

[ChrisJefferson]

I see the problem, yes, docs are still wrong.

[ChrisJefferson]

Also, I noticed I was still compressing things like OpenLog. Maybe we want that, but for now let's do the minimal.

The only thing I'm now compressing if it ends .gz without prompting is SaveWorkspace, because it's just a binary file most people wouldn't understand anyway, and it compresses really well.

[ChrisJefferson]

Sorry, just cleaning up and pushing some changes. I am having one consistent problem, which is this patch seems to stop HPCGAP from even getting to the gap prompt, which is strange and I'm investigating.

[ChrisJefferson]

EDIT: Found the issue, was simple (needed to fix HPC-GAP's console code to use new functions)

[fingolfin]

LGTM!
The testmanuals check failed to download the bootstrap archive; I've restarted the checks to make sure it passes then (unfortunately one can only restart all checks at once...)