Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CTblLib: make sure that tests pass #1619

Closed
olexandr-konovalov opened this issue Aug 23, 2017 · 8 comments
Closed

CTblLib: make sure that tests pass #1619

olexandr-konovalov opened this issue Aug 23, 2017 · 8 comments
Labels
kind: enhancement Label for issues suggesting enhancements; and for pull requests implementing enhancements topic: packages issues or PRs related to package handling, or specific to a package (for packages w/o issue tracker)

Comments

@olexandr-konovalov
Copy link
Member

Below is the output of CTblLib tests with GAP 4.8.8 release candidate when GAP is loaded with default packages. It will be nice to clean up the diffs, which are of several kinds:

In addition, it's not possible to detect whether tests passes or not automatically. There is an easy fix for that: PackageInfo.g specifies the testfile tst/testauto.g, but that testfile only calls one test tst/docxpl.tst. In this case, one could just specify tst/docxpl.tst in PackageInfo.g instead of tst/testauto.g. Otherwise, for the compliance with automated failure detection one could follow #1617 or use TestDirectory.

============================OUTPUT START==============================

#I  RunPackageTests("ctbllib", "1.2.2", "tst/testauto.g", auto);
 ���������������������������   GAP 4.8.8, 20-Aug-2017, build of 2017-08-20 19:40:04 (BST)
 ���  GAP  ���   https://www.gap-system.org
 ���������������������������   Architecture: x86_64-pc-linux-gnu-gcc-default64
 Libs used:  gmp, readline
 Loaded workspace: wsp.g
 Components: trans 1.0, prim 2.1, small* 1.0, id* 1.0
 Packages:   AClib 1.2, Alnuth 3.0.0, AtlasRep 1.5.1, AutPGrp 1.8, 
             Browse 1.8.7, CRISP 1.4.4, Cryst 4.1.12, CrystCat 1.1.6, 
             CTblLib 1.2.2, FactInt 1.5.4, FGA 1.3.1, GAPDoc 1.6, IO 4.4.6, 
             IRREDSOL 1.4, LAGUNA 3.7.0, Polenta 1.3.7, Polycyclic 2.11, 
             RadiRoot 2.7, ResClasses 4.6.0, Sophus 1.23, SpinSym 1.5, 
             TomLib 1.2.6, Utils 0.46
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'
########> Diff in /data/gap-jenkins/workspace/GAP-minor-release-test/GAPCOPTS/\
64build/GAPGMP/gmp/GAPTARGET/packages/label/kovacs/gap4r8/pkg/ctbllib/tst/docx\
pl.tst, line 34:
# Input is:
AllCharacterTableNames( Size, 120 );
# Expected output:
[ "2.A5", "2.A6M2", "2xA5", "A5.2", "A6.2_1M3", "D120", "L2(25)M3" ]
# But found:
[ "2.A5", "2.A6M2", "2.Alt(5)", "2xA5", "A5.2", "A6.2_1M3", "D120", 
  "L2(25)M3", "Sym(5)" ]
########
########> Diff in /data/gap-jenkins/workspace/GAP-minor-release-test/GAPCOPTS/\
64build/GAPGMP/gmp/GAPTARGET/packages/label/kovacs/gap4r8/pkg/ctbllib/tst/docx\
pl.tst, line 137:
# Input is:
AllCharacterTableNames( IsSimple, true, IsDuplicateTable, false,
                           isambivalent, true );
# Expected output:
[ "3D4(2)", "A10", "A14", "A5", "A6", "J1", "J2", "L2(101)", 
  "L2(109)", "L2(113)", "L2(121)", "L2(125)", "L2(13)", "L2(16)", 
  "L2(17)", "L2(25)", "L2(29)", "L2(32)", "L2(37)", "L2(41)", 
  "L2(49)", "L2(53)", "L2(61)", "L2(64)", "L2(73)", "L2(8)", 
  "L2(81)", "L2(89)", "L2(97)", "O7(5)", "O8+(2)", "O8+(3)", 
  "O8+(7)", "O8-(2)", "O8-(3)", "O9(3)", "S10(2)", "S12(2)", "S4(4)", 
  "S4(5)", "S4(8)", "S4(9)", "S6(2)", "S6(4)", "S6(5)", "S8(2)" ]
# But found:
[ "2.Alt(2)", "3D4(2)", "A10", "A14", "A5", "A6", "Alt(10)", 
  "Alt(14)", "Alt(5)", "Alt(6)", "J1", "J2", "L2(101)", "L2(109)", 
  "L2(113)", "L2(121)", "L2(125)", "L2(13)", "L2(16)", "L2(17)", 
  "L2(25)", "L2(29)", "L2(32)", "L2(37)", "L2(41)", "L2(49)", 
  "L2(53)", "L2(61)", "L2(64)", "L2(73)", "L2(8)", "L2(81)", 
  "L2(89)", "L2(97)", "O7(5)", "O8+(2)", "O8+(3)", "O8+(7)", 
  "O8-(2)", "O8-(3)", "O9(3)", "S10(2)", "S12(2)", "S4(4)", "S4(5)", 
  "S4(8)", "S4(9)", "S6(2)", "S6(4)", "S6(5)", "S8(2)", "Sym(2)" ]
########
########> Diff in /data/gap-jenkins/workspace/GAP-minor-release-test/GAPCOPTS/\
64build/GAPGMP/gmp/GAPTARGET/packages/label/kovacs/gap4r8/pkg/ctbllib/tst/docx\
pl.tst, line 153:
# Input is:
for i in [ 2, 3, 5, 7, 11, 13 ] do
     Print( i, "\n",
       AllCharacterTableNames( IsSimple, true, IsAbelian, false,
           IsDuplicateTable, false, isppure( i ), true ),
       "\n" );
   od;
# Expected output:
2
[ "A5", "A6", "L2(16)", "L2(17)", "L2(31)", "L2(32)", "L2(64)", 
  "L2(8)", "L3(2)", "L3(4)", "Sz(32)", "Sz(8)" ]
3
[ "A5", "A6", "L2(17)", "L2(19)", "L2(27)", "L2(53)", "L2(8)", 
  "L2(81)", "L3(2)", "L3(4)" ]
5
[ "A5", "A6", "A7", "L2(11)", "L2(125)", "L2(25)", "L2(49)", "L3(4)", 
  "M11", "M22", "S4(7)", "Sz(32)", "Sz(8)", "U4(2)", "U4(3)" ]
7
[ "A7", "A8", "A9", "G2(3)", "HS", "J1", "J2", "L2(13)", "L2(49)", 
  "L2(8)", "L2(97)", "L3(2)", "L3(4)", "M22", "O8+(2)", "S6(2)", 
  "Sz(8)", "U3(3)", "U3(5)", "U4(3)", "U6(2)" ]
11
[ "A11", "A12", "A13", "Co2", "HS", "J1", "L2(11)", "L2(121)", 
  "L2(23)", "L5(3)", "M11", "M12", "M22", "M23", "M24", "McL", "ON", 
  "Suz", "U5(2)", "U6(2)" ]
13
[ "2E6(2)", "2F4(2)'", "3D4(2)", "A13", "A14", "A15", "F4(2)", 
  "Fi22", "G2(3)", "G2(4)", "L2(13)", "L2(25)", "L2(27)", "L3(3)", 
  "L4(3)", "O7(3)", "O8+(3)", "S4(5)", "S6(3)", "Suz", "Sz(8)", 
  "U3(4)" ]
# But found:
2
[ "A5", "A6", "Alt(5)", "Alt(6)", "L2(16)", "L2(17)", "L2(31)", 
  "L2(32)", "L2(64)", "L2(8)", "L3(2)", "L3(4)", "Sz(32)", "Sz(8)" ]
3
[ "A5", "A6", "Alt(5)", "Alt(6)", "L2(17)", "L2(19)", "L2(27)", 
  "L2(53)", "L2(8)", "L2(81)", "L3(2)", "L3(4)" ]
5
[ "A5", "A6", "A7", "Alt(5)", "Alt(6)", "Alt(7)", "L2(11)", 
  "L2(125)", "L2(25)", "L2(49)", "L3(4)", "M11", "M22", "S4(7)", 
  "Sz(32)", "Sz(8)", "U4(2)", "U4(3)" ]
7
[ "A7", "A8", "A9", "Alt(7)", "Alt(8)", "Alt(9)", "G2(3)", "HS", 
  "J1", "J2", "L2(13)", "L2(49)", "L2(8)", "L2(97)", "L3(2)", 
  "L3(4)", "M22", "O8+(2)", "S6(2)", "Sz(8)", "U3(3)", "U3(5)", 
  "U4(3)", "U6(2)" ]
11
[ "A11", "A12", "A13", "Alt(11)", "Alt(12)", "Alt(13)", "Co2", "HS", 
  "J1", "L2(11)", "L2(121)", "L2(23)", "L5(3)", "M11", "M12", "M22", 
  "M23", "M24", "McL", "ON", "Suz", "U5(2)", "U6(2)" ]
13
[ "2E6(2)", "2F4(2)'", "3D4(2)", "A13", "A14", "A15", "Alt(13)", 
  "Alt(14)", "Alt(15)", "F4(2)", "Fi22", "G2(3)", "G2(4)", "L2(13)", 
  "L2(25)", "L2(27)", "L3(3)", "L4(3)", "O7(3)", "O8+(3)", "S4(5)", 
  "S6(3)", "Suz", "Sz(8)", "U3(4)" ]
########
########> Diff in /data/gap-jenkins/workspace/GAP-minor-release-test/GAPCOPTS/\
64build/GAPGMP/gmp/GAPTARGET/packages/label/kovacs/gap4r8/pkg/ctbllib/tst/docx\
pl.tst, line 294:
# Input is:
ind:= Induced( m, t, [ infl ] );
# Expected output:
[ ClassFunction( CharacterTable( "Fi23" ), 
    [ 207766624665600, 0, 0, 0, 603832320, 127567872, 6635520, 
      663552, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
      0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
      0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
      0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
      0, 0, 0, 0, 0, 0, 0, 0, 0 ] ) ]
# But found:
[ ClassFunction( CharacterTable( "Fi23" ),
  [ 207766624665600, 0, 0, 0, 603832320, 127567872, 6635520, 663552, 
      0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
      0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
      0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
      0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
      0, 0, 0, 0, 0, 0 ] ) ]
########
########> Diff in /data/gap-jenkins/workspace/GAP-minor-release-test/GAPCOPTS/\
64build/GAPGMP/gmp/GAPTARGET/packages/label/kovacs/gap4r8/pkg/ctbllib/tst/docx\
pl.tst, line 334:
# Input is:
AllCharacterTableNames( IsSimple, true, IsAbelian, false,
                           Size, [ 1 .. 100 ] );
# Expected output:
[ "A5", "A6M2" ]
# But found:
[ "A5", "A6M2", "Alt(5)" ]
########
########> Diff in /data/gap-jenkins/workspace/GAP-minor-release-test/GAPCOPTS/\
64build/GAPGMP/gmp/GAPTARGET/packages/label/kovacs/gap4r8/pkg/ctbllib/tst/docx\
pl.tst, line 339:
# Input is:
AllCharacterTableNames( Size, IsPrimeInt );
# Expected output:
[ "C3" ]
# But found:
[ "2.Alt(2)", "Alt(3)", "C3", "Sym(2)" ]
########
########> Diff in /data/gap-jenkins/workspace/GAP-minor-release-test/GAPCOPTS/\
64build/GAPGMP/gmp/GAPTARGET/packages/label/kovacs/gap4r8/pkg/ctbllib/tst/docx\
pl.tst, line 366:
# Input is:
NamesOfEquivalentLibraryCharacterTables( tbl );
# Expected output:
[ "A5", "A6M2" ]
# But found:
[ "A5", "A6M2", "Alt(5)" ]
########
########> Diff in /data/gap-jenkins/workspace/GAP-minor-release-test/GAPCOPTS/\
64build/GAPGMP/gmp/GAPTARGET/packages/label/kovacs/gap4r8/pkg/ctbllib/tst/docx\
pl.tst, line 479:
# Input is:
UnipotentCharacter( tbl, [ [ 0, 1 ], [ 2 ] ] );
# Expected output:
Character( CharacterTable( "U4(2).2" ), 
[ 15, 7, 3, -3, 0, 3, -1, 1, 0, 1, -2, 1, 0, 0, -1, 5, 1, 3, -1, 2, 
  -1, 1, -1, 0, 0 ] )
# But found:
Character( CharacterTable( "U4(2).2" ),
 [ 15, 7, 3, -3, 0, 3, -1, 1, 0, 1, -2, 1, 0, 0, -1, 5, 1, 3, -1, 2, 
  -1, 1, -1, 0, 0 ] )
########
########> Diff in /data/gap-jenkins/workspace/GAP-minor-release-test/GAPCOPTS/\
64build/GAPGMP/gmp/GAPTARGET/packages/label/kovacs/gap4r8/pkg/ctbllib/tst/docx\
pl.tst, line 552:
# Input is:
BrowseCTblLibInfo();
# Expected output:
# But found:
#I  cannot switch to visual mode because of TERM = "dumb"
########
########> Diff in /data/gap-jenkins/workspace/GAP-minor-release-test/GAPCOPTS/\
64build/GAPGMP/gmp/GAPTARGET/packages/label/kovacs/gap4r8/pkg/ctbllib/tst/docx\
pl.tst, line 566:
# Input is:
BrowseCommonIrrationalities();;
# Expected output:
# But found:
#I  cannot switch to visual mode because of TERM = "dumb"
Error, no method found! For debugging hints type ?Recovery from NoMeth\
odFound
Error, no 1st choice method found for `DuplicateFreeList' on 1 argumen\
ts
########
########> Diff in /data/gap-jenkins/workspace/GAP-minor-release-test/GAPCOPTS/\
64build/GAPGMP/gmp/GAPTARGET/packages/label/kovacs/gap4r8/pkg/ctbllib/tst/docx\
pl.tst, line 578:
# Input is:
BrowseCTblLibDifferences();
# Expected output:
# But found:
#I  cannot switch to visual mode because of TERM = "dumb"
########
########> Diff in /data/gap-jenkins/workspace/GAP-minor-release-test/GAPCOPTS/\
64build/GAPGMP/gmp/GAPTARGET/packages/label/kovacs/gap4r8/pkg/ctbllib/tst/docx\
pl.tst, line 583:
# Input is:
BrowseCTblLibDifferences();
# Expected output:
# But found:
#I  cannot switch to visual mode because of TERM = "dumb"
########
########> Diff in /data/gap-jenkins/workspace/GAP-minor-release-test/GAPCOPTS/\
64build/GAPGMP/gmp/GAPTARGET/packages/label/kovacs/gap4r8/pkg/ctbllib/tst/docx\
pl.tst, line 589:
# Input is:
BrowseCTblLibDifferences();
# Expected output:
# But found:
#I  cannot switch to visual mode because of TERM = "dumb"
########
Input file: docxpl.tst
GAP4stones: 99
#I  RunPackageTests("ctbllib", "1.2.2", "tst/testauto.g", auto): runtime 
101103
============================OUTPUT END================================
@olexandr-konovalov olexandr-konovalov added the topic: packages issues or PRs related to package handling, or specific to a package (for packages w/o issue tracker) label Aug 23, 2017
@olexandr-konovalov
Copy link
Member Author

Also notified @ThomasBreuer by email.

@olexandr-konovalov
Copy link
Member Author

As @ThomasBreuer confirmed by email, these differences are harmless, so you may ignore them.

@olexandr-konovalov
Copy link
Member Author

My suggestion to @ThomasBreuer was to move tests which fail to another test files in ctbllib/tst directory, and put

TestFile := "tst/docxpl.tst",

in PackageInfo.g?. After all, there are 13 diffs in the test file with 1039 lines and 227 GAP
prompts, while the rest of test actually runs.

@olexandr-konovalov
Copy link
Member Author

I have found a setting (calling the test with TERM=xterm) which removes all #I cannot switch to visual mode because of TERM = "dumb" messages.

Now in the stable-4.8 branch tests status is as follows:

With no packages loaded (GAP started with -r option) :  9 DIFFS
With all packages loaded with LoadAllPackages()      :  9 DIFFS
With default packages loaded at GAP startup          :  9 DIFFS

and logs are at https://travis-ci.org/gap-system/gap-docker-pkg-tests-stable-staging/jobs/331189402

In the stable-4.9 branch tests status is as follows:

With no packages loaded (GAP started with -r option) :  10 DIFFS
With all packages loaded with LoadAllPackages()      :  10 DIFFS
With default packages loaded at GAP startup          :  10 DIFFS

with logs available at https://travis-ci.org/gap-system/gap-docker-pkg-tests-stable-4.9-staging/jobs/327912057

@olexandr-konovalov
Copy link
Member Author

I have noticed that CTbliLib tests have some more diffs when it is tested with the current GAP master branch - @ThomasBreuer please see the log at https://travis-ci.org/gap-system/gap-docker-pkg-tests-master-staging/jobs/415554743 (scroll till the end of the log file to see three log files and their explanation). For example, this one

########> Diff in /home/gap/inst/gap-master/pkg/ctbllib/tst/docxpl.tst:85
# Input is:
s2:= CharacterTable( mx[2] );
# Expected output:
CharacterTable( "M12M2" )
# But found:
Error, no method found! For debugging hints type ?Recovery from NoMeth\
odFound
Error, no 1st choice method found for `Length' on 1 arguments
########

does not occur in the tests under the stable-4.9 branch (e.g. https://travis-ci.org/gap-system/gap-docker-pkg-tests-stable-4.9-staging/jobs/415728861).

@ThomasBreuer ThomasBreuer mentioned this issue Aug 15, 2018
Closed
@olexandr-konovalov olexandr-konovalov mentioned this issue Oct 6, 2018
Closed
50 tasks
@olexandr-konovalov olexandr-konovalov mentioned this issue Nov 9, 2018
Closed
31 tasks
@olexandr-konovalov
Copy link
Member Author

@ThomasBreuer I have posted a comment about AtlasRep at #1617 - the same applies to CTblLib, of course. You can inspect the status of standard tests for CTblLib in GAP 4.10.0 release at https://travis-ci.org/gap-system/gap-docker-pkg-tests/jobs/453290388.

@olexandr-konovalov olexandr-konovalov mentioned this issue Dec 6, 2018
Closed
@fingolfin fingolfin added the kind: enhancement Label for issues suggesting enhancements; and for pull requests implementing enhancements label Mar 26, 2019
@olexandr-konovalov
Copy link
Member Author

@ThomasBreuer have you seen this issue? I have posted a new comment about AtlasRep in #1617, and want to remind about a similar problem here. The tests for CTblLib fail due to diffs, see e.g. most recent logs here:

for master and stable-4.10 branches respectively. They may be harmless, but that does not provide a relief - one can not rely on someone manually inspecting logs regularly to see if there are any new problems, and I do not feel comfortable that this is still the situation for some central packages in GAP.

@olexandr-konovalov
Copy link
Member Author

@ThomasBreuer the new release of CTblLib is picked up, and should appear in the next GAP release. The default test, specified in its PackageIno.g now passes - thanks. So, CTblLib tests has been now moved to:

and it there will be any changes in GAP or in packages that will break CTblLib tests, we should be able to discover that easily. That's a good improvement for such a crucial package as CTblLib.

I am looking forward for the ability to automatically detect test failures for AtlasRep too, as discussed in #1617. For now, AtlasRep are in "staging" test, where you can always check their status yourself - I hope you are keeping an eye on these:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind: enhancement Label for issues suggesting enhancements; and for pull requests implementing enhancements topic: packages issues or PRs related to package handling, or specific to a package (for packages w/o issue tracker)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@fingolfin @olexandr-konovalov