Problems with GAP standard tests

[olexandr-konovalov]

I will appreciate further help from @cdwensley and @james-d-mitchell.

First, groupoids 1.57 has been picked up on Tuesday Aug 28, 2018 12:45:36 PM. It happened that that test{install,standard,bugix} from the standard GAP test suite passed after that on Linux with gcc 4.8 (where Semigroups can not be compiled), but were failing on macOS where Semigroups was in working order. These are examples of what happened:

########> Diff in /Volumes/hudson-fs/workspace/GAP-stable-test/GAPCOPTS/64buil\
d/GAPTARGET/install/label/fruitloop/GAP-stable-snapshot/tst/testinstall/ctblmo\
li.tst:31
# Input is:
MolienSeries( NaturalCharacter( G ) );
# Expected output:
( 1 ) / ( (1-z^12)*(1-z^9)*(1-z^6)*(1-z^5)*(1-z^3)*(1-z) )
# But found:
Error, Semigroups: SEMIGROUPS.DeterministicNormalizer: usage,
the second arg must be a semigroup of transformations,
partial perms or bipartitions,
########

########> Diff in /Volumes/hudson-fs/workspace/GAP-stable-test/GAPCOPTS/64buil\
d/GAPTARGET/install/label/fruitloop/GAP-stable-snapshot/tst/testinstall/fpmon.\
tst:18
# Input is:
map := IsomorphismFpSemigroup(S);;
# Expected output:
# But found:
Error, recursion depth trap (5000)
########
########> Diff in /Volumes/hudson-fs/workspace/GAP-stable-test/GAPCOPTS/64buil\
d/GAPTARGET/install/label/fruitloop/GAP-stable-snapshot/tst/testinstall/fpmon.\
tst:20
# Input is:
ForAll(S, x -> (x ^ map) ^ inv = x);
# Expected output:
true
# But found:
Error, recursion depth trap (5000)
########

########> Diff in /Volumes/hudson-fs/workspace/GAP-stable-test/GAPCOPTS/64buil\
d/GAPTARGET/install/label/fruitloop/GAP-stable-snapshot/tst/testinstall/grp/ba\
sic.tst:204
# Input is:
IsomorphismGroups(Stabilizer(MathieuGroup(23),23), MathieuGroup(22)) <> fail;
# Expected output:
true
# But found:
Error, Semigroups: SEMIGROUPS.DeterministicNormalizer: usage,
the second arg must be a semigroup of transformations,
partial perms or bipartitions,
########
     486 ms (334 ms GC) and 7.64MB allocated for grp/basic.tst
testing: /Volumes/hudson-fs/workspace/GAP-stable-test/GAPCOPTS/64build/GAPTARG\
ET/install/label/fruitloop/GAP-stable-snapshot/tst/testinstall/grp/classic-G.t\
st
########> Diff in /Volumes/hudson-fs/workspace/GAP-stable-test/GAPCOPTS/64buil\
d/GAPTARGET/install/label/fruitloop/GAP-stable-snapshot/tst/testinstall/grp/cl\
assic-G.tst:49
# Input is:
GO(IsPermGroup,-1,4,9);
# Expected output:
Perm_GO(-1,4,9)
# But found:
Error, reached the pre-set memory limit
(change it with the -o command line option)
########
########> Diff in /Volumes/hudson-fs/workspace/GAP-stable-test/GAPCOPTS/64buil\
d/GAPTARGET/install/label/fruitloop/GAP-stable-snapshot/tst/testinstall/grp/cl\
assic-G.tst:81
# Input is:
GammaL(IsPermGroup,3,9);
# Expected output:
Perm_GammaL(3,9)
# But found:
Error, reached the pre-set memory limit
(change it with the -o command line option)
########

Also testinstall/grpmat.tst run out of memory.

########> Diff in /Volumes/hudson-fs/workspace/GAP-stable-test/GAPCOPTS/64buil\
d/GAPTARGET/standard/label/fruitloop/GAP-stable-snapshot/tst/teststandard/dire\
ct_factors.tst:109
# Input is:
DirectFactorsOfGroup(SymmetricGroup(5));
# Expected output:
[ Sym( [ 1 .. 5 ] ) ]
# But found:
Error, Semigroups: SEMIGROUPS.DeterministicNormalizer: usage,
the second arg must be a semigroup of transformations,
partial perms or bipartitions,
########

########> Diff in /Volumes/hudson-fs/workspace/GAP-stable-test/GAPCOPTS/64buil\
d/GAPTARGET/bugfix/label/fruitloop/GAP-stable-snapshot/tst/testbugfix/00006.ts\
t:3
# Input is:
aut:=AutomorphismGroup(g);;
# Expected output:
# But found:
Error, Semigroups: SEMIGROUPS.DeterministicNormalizer: usage,
the second arg must be a semigroup of transformations,
partial perms or bipartitions,
########

########> Diff in /Volumes/hudson-fs/workspace/GAP-stable-test/GAPCOPTS/64buil\
d/GAPTARGET/bugfix/label/fruitloop/GAP-stable-snapshot/tst/testbugfix/2005-12-\
22-t00139.tst:13
# Input is:
IdGroup(dp);
# Expected output:
[ 960, 5746 ]
# But found:
Error, recursion depth trap (5000)
########
########> Diff in /Volumes/hudson-fs/workspace/GAP-stable-test/GAPCOPTS/64buil\
d/GAPTARGET/bugfix/label/fruitloop/GAP-stable-snapshot/tst/testbugfix/2005-12-\
22-t00139.tst:15
# Input is:
IdGroup(Image(Projection(dp,2)));
# Expected output:
[ 120, 5 ]
# But found:
Error, recursion depth trap (5000)
########

Semigroups 3.0.17 has been picked up later on Thursday Aug 30, 2018 9:00:00 AM. Its previous version was not working in the GAP master branch. As soon as version 3.0.17 has been picked up, the same problems started to happen in the GAP master branch (there they may be worsened by the incompatibility of Semigroups with recent MatrixObj merge, so I suggest to debug this problem in stable-4.9 branch).

I have tried to backtrack only one of the two packages, but it seems to me that either update causes the problem (need to double-check). So I now replaced both - groupoids by its stable release 1.55, and Semigroups - by its stable release 3.0.16. Now tests are back to normal in stable-4.9 branch (and I expect them to pass in the master branch, but of course Semigroups will not be usable there).

It may be hard to reproduce the problem - try make testinstall to run the test in full to see if you can reproduce the problem before trying to localise it further.

[cdwensley]

Not sure what I'm supposed to be doing, but happy to help if I can.
Have been running 4.9.2 with both groupoids and semigroups-3.0.17 loaded.
Running install.g gives a number of diffs - in ffeconway, basic, classic_PG and meataxe - but with none of the examples listed above. No doubt these diffs are due to differences between 4.9.2 and 4.9-stable. Now running teststandard.g, but no new diffs so far.
I was pleasantly surprised to find that semigroups-3.0.17 compiled on my iMac.
Nearly all the semigroups tests passed, but semirms.tst ran out of memory.
Searching for "Semigroups" in groupoids produces lots of operations involving "SemigroupsWithObjects", but I cannot imagine that they could cause any problem.
The only non-WithObjects operation is the attribute InverseOfIsomorphismFpSemigroup.

[cdwensley]

Since this issue is now on the critical list for GAP 4.10 I need to know what, if anything, I should be doing. Running testinstall.g in GAP 4.9.3 with semigroups-3.0.18 and groupoids-1.57dev loaded I did not get any of the errors listed above.
The only interaction between the two packages that I can imagine is that groupoids installed an attribute InverseOfIsomorphismFpSemigroup. It turns out that this function is no longer needed and it has been removed from groupoids version 1.58 released this afternoon.
Since new versions of semigroups have also been released, do errors still appear in standard tests?

[james-d-mitchell]

Sorry I somehow over looked this issue previously. I will investigate as soon as I can and get back to you.

[james-d-mitchell]

@alex-konovalov can you confirm in which branch of GAP this error occurred? The line numbers in the errors don't seem to correspond to the lines in the test files in the stable-4.10 branch. For example,
tst/testinstall/grp/basic.tst:204 is <fp group of size 8 on the generators [ r, s ]>

[james-d-mitchell]

There is a method in Semigroups for Normalizer(IsPermGroup, IsSemigroup) is it possible that somehow the rank of IsSemigroup is increased to equal or exceed that of IsPermGroup so that the Semigroups package method is selected instead of the one in Groupoids? I can't immediately get Groupoids to load, it seems to have lots of dependencies.

[cdwensley]

@james-d-mitchell - more detail please:
(1) are you finding the errors that Alex reported? I cannot reproduce any.
(2) Please elaborate on the previous comment. Groupoids only requires Fga and Utils - hardly lots of dependencies - although Semigroups was reinstalled as a suggested package in yesterday's release of version 1.58. The operation 'Normalizer' does not appear in the groupoids library - though of course might be called by another function - but there is certainly no method.

[james-d-mitchell]

Sorry, I miswrote, I meant the Normalizer method in GAP not groupoids.

About the dependencies: Utils requires Polycyclic (for some reason), and Polycyclic seems to have lots of dependencies, and I didn't find time yesterday to install them all. So, no I haven't been able to reproduce the errors @alex-konovalov reported. I'll try again when I have more time.

[cdwensley]

Ah - yes. This can be fixed. Utils required Polycyclic while some functions were being transferred, but this should now be removed. And Groupoids may no longer need Utils - will check.

[cdwensley]

Functions transferred to Utils from Polycyclic still require Polycyclic to be loaded - this is a matter to be sorted out for Utils. However Groupoids no longer requires Utils - in version 1.59 just released.

[olexandr-konovalov]

New version of groupoids picked up - I will test it to see if the problem is resolved: gap-system/gap-distribution@f8c82fa

[fingolfin]

@james-d-mitchell Polycyclic has exactly two dependencies: autpgrp and alnuth; neither of which has further dependencies. Both of these are also on GitHub, and don't require compiled code (although alnuth is an interface for Pari/GP, so ideally it wants that installed; but 99% of polycyclic is perfectly usable w/o Pari/GP installed; certainly nothing relevant for this issue needs it).

[james-d-mitchell]

Thanks @fingolfin, I got as far as realising I needed alnuth, and then ran out of time. This wasn't a complaint BTW, I just didn't have a copy of all the necessary packages, and didn't have sufficient time to install them.

[olexandr-konovalov]

Nice - problems disappeared. I will close this, and will open another issue in case of anything else. Thanks!!!