Fix unit tests from live stream #1

[codereport]

[codereport]

Looks like there are four main categories:

• E.
• Large matrices
• Random # ~ y
• Segfaults

[codereport]

This is the updated list:

Also, recently the following were commented out:

• gmmf
• g430
• g420

[Sebanisu]

gstack fails on my computer in 200ms. Causes a SegFault. This is running the test in debug mode.

383: Test command: /mnt/d/dev/jsource/cmake-build-debug/jsrc/jconsole "gstack.ijs"
383: Test timeout computed to be: 1500
383: Hello YouTube Viewers, all 22 of you!
383: see: tsu_notes, tsu_usage, tsu_pacman, and tsu_jd
383: 
383:    RUN  ddall  NB. report scripts that fail
383:    RECHO ddall NB. echo script names as run and final count of failures
383: 
383: j902/j64/linux/beta/GPL3/unknown/2021-01-26T20:43:34/gcc-10-2
Exception: SegFault

Program received signal SIGSEGV, Segmentation fault.
0x00007ffffec9ec10 in jtgaf (jt=0x0, blockx=0) at /mnt/d/dev/jsource/jsrc/m.c:905
905	RESTRICTF A jtgaf(J jt,I blockx){A z;I mfreeb;I n=(I)2<<blockx;  // n=size of allocated block

[codereport]

@Sebanisu Are you on osx or linux? Does ctest work?

[Sebanisu]

I'm on windows 10 with wsl on Ubuntu 20.4. I'm running the test from the Clion IDE. All other tests that are enabled are passing. I can run the tests from console if you want. I just don't know the command line for ctest. Also I have gcc 10.2 and cmake 19.

[codereport]

I'm on windows 10 with wsl on Ubuntu 20.4. I'm running the test from the Clion IDE. All other tests that are enabled are passing. I can run the tests from console if you want. I just don't know the command line for ctest. Also I have gcc 10.2 and cmake 19.

ninja -C build test is the command (with -j$(nproc) if you want)

[juntuu]

Do flaky tests also fall under this issue?

The g001 seems to be timing issue (see #39 (comment)), which get worse with more load on the system (at least on my machine).

[codereport]

Do flaky tests also fall under this issue?

The g001 seems to be timing issue (see #39 (comment)), which get worse with more load on the system (at least on my machine).

Yea, g001 is probably the most flaky for me, g430/1a also pretty flaky. I ended up commenting out all of the gmmf* tests, maybe we should just do the same for these ones.

[juntuu]

I identified a problem with the g420 test:

The failing testcases were testing fold, starting from line 730

NB. Fold  F. F.. F.: F: F:. F:: ------------------------------------------------------------------
NB. monad
10 -: ] F.. + i. 5

The failure comes from foldr.ijs not been found, and so all the folds fail. Trying the first line in jconsole gave a "file not found error"

Hello YouTube Viewers, all 22 of you!
   10 -: ] F.. + i. 5
not found: /.../jsource/jlibrary/addons/dev/fold/foldr.ijs
|nonce error
|   10-:]    F..+i.5

So, it seems to looks for fold in jlibrary/addons/dev/fold/foldr.ijs
when the code is in jlibrary/dev/fold/foldr.ijs.

The loading is done in adverbs/ar.c:656

  switch(step){  // try the startup, from the bottom up
  case 1: eval("load'~addons/dev/fold/foldr.ijs'");  // fall through
  case 0: if((foldconj=nameref(nfs(8,"Foldr_j_"),jt->locsyms))&&AT(foldconj)&CONJ)goto found;  // there is always a ref, but it may be to [:
  }

I got the tests to pass by moving the jlibrary/dev folder to jlibrary/addons/dev, but I'm not sure if this is the right solution, or if the root cause is some other misconfiguration.

Maybe someone who knows more about the j system, and how it locates and loads files, could chime in.

[Sebanisu]

ninja -C build test is the command (with -j$(nproc) if you want)

Oh it's in the CONTRIBUTING.md. 🙃 I never used ninja till now. It SegFault'ed via the console with ninja as well.

383/397 Test #383: gstack ...........................***Exception: SegFault  0.16 sec

[juntuu]

gstack fails on my computer in 200ms. Causes a SegFault. This is running the test in debug mode.

I'm on windows 10 with wsl on Ubuntu 20.4.

At the top of the test file, there is a comment about stack size issues

NB. stack issues (function call limit) ----------------------------------

0 0 $  0 : 0
The recursion limit is constrained by the stack size available to 
the J executable file. Crashes due to stack errors can be overcome 
by increasing the stack size. Under Windows, the stack size can be 
queried and set as follows:
...

AFAIK windows gives programs less stack by default, compared to linux/mac, so it could be stack overflow. I'm not sure how running in wsl affects the stack size though.

[juntuu]

Found a promising looking commit jsoftware/jsource@6adb504 for the g001 "issue".

Then looking for "THRESHOLD" in tsu.ijs I found this

THRESHOLD=: 0 NB. allow timing tests to trigger failure 
THRESHOLD=: 1 NB. force timing tests to pass

and this

tsu_notes=: 0 : 0
many scripts have timing tests
typically comparing timing/result of j vs j model
these tests can be essential for new/changed code
but vary greatly across environments
and can result in false failures

THRESHOLD set to 1 by default - ignores timing test failures
threshold set to 0.2 for loose tests (0.75 for tighter tests)

THRESHOLD should be applied as false failures are discovered

gfft/glapack not in ddall - run separately with: RUN1'gfft'
g18x fails on subsequent runs - no idea why
g401 occasionally fails (random data?) but then runs clean
)