Fix crash when comparing recursive data structures

[fingolfin]

This fixes crashes when comparing recursive data structures (see #1150). With this PR, we get these errors (instead of crashes):

gap> [~] < [~];
Error, recursion depth trap (5000)

not in any function at *stdin*:1
you may 'return;'
brk>

and

gap> rec(a:=~) = rec(a:=~);
Error, recursion depth trap (5000)

not in any function at *stdin*:1
you may 'return;'
brk>

etc. etc.

Note that the user can still choose to continue the recursion, and doing so often enough will still crash. That seems acceptable to me, but if desired, we could also produce a variant of this code which prevents the user from returning here.

Also, we should add test cases for these (former) crashes.

Finally, I wonder if other function share similar problems? E.g. are there lists/record types which are not plists / precords and which can also be recursive?

[codecov]

Codecov Report

Merging #1151 into master will decrease coverage by -0.03%.
The diff coverage is 97.05%.

@@            Coverage Diff             @@
##           master    #1151      +/-   ##
==========================================
- Coverage   59.27%   59.24%   -0.03%     
==========================================
  Files         434      435       +1     
  Lines      231096   231106      +10     
==========================================
- Hits       136972   136920      -52     
- Misses      94124    94186      +62

Impacted Files	Coverage Δ
src/funcs.h	100% <100%> (ø)
src/plist.c	91.69% <100%> (+0.02%)	✅
src/funcs.c	86.72% <100%> (+0.98%)	✅
src/precord.c	89.2% <93.33%> (+0.56%)	✅
src/permutat.c	55.21% <ø> (-2.92%)	❌
lib/tietze.gi	41.85% <ø> (-0.74%)	❌
lib/grpmat.gi	70.89% <ø> (-0.56%)	❌
lib/straight.gi	25.28% <ø> (-0.3%)	❌
src/compiler.c	65.45% <ø> (-0.05%)	❌
... and 2 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f90e2ad...5ededb1. Read the comment docs.

[ChrisJefferson]

I would add some tests. There are in my mind two types to add:

1. Some tests with some ~ in
2. Some tests where you just do a bunch of comparision of some nested lists and records in a loop of 10,000 iterations or so, where the code has to do some recursion, to make sure we are incrementing/decrementing the recursiondepth correctly.

However, the general design is fine.

[ChrisJefferson]

Can I be picky and ask for another pair of brackets here? I should know what the priority order of != and || is, but I'm afraid I can't remember. However, I agree the rearranged code is fine.

[ChrisJefferson]

Pop TLS RecusionDepth?

[fingolfin]

Absolutely. This is one place where I'd love we were in C++ and could manage this via RAII

[fingolfin]

(Note that you commented the old version of the code, not the new, hence the fix does not show up here)

[fingolfin]

@ChrisJefferson I am not quite sure I understand what you want to do with your second test... But I suggest you add it yourself once this PR has been merged?

[ChrisJefferson]

@fingolfin Will do. Apart from that, I'm now happy with the PR.