Doc test tool failure: xt/aspell.t

[tbrowder]

Problem or new feature

While testing the current doc master branch on a Debian 11 host I get test failures from xt/aspell.t:

make xtest >& xtest.log

The full output of the log:

`if [ "" != "" ]; then prove -j  -e raku t xt; else prove -e raku t xt; fi
t/00-meta.t ................ ok
t/02-pod-valid.t ........... ok
t/03-tests-valid.t ......... ok
t/04-pod6.t ................ ok
t/05-tabs.t ................ ok
xt/01-raku-version.t ....... ok
A worker in a parallel iteration (hyper or race) initiated here:
  in block <unit> at xt/aspell.t line 68

Died at:
    A react block:
      in block  at xt/aspell.t line 102

    Died because of the exception:
        Malformed UTF-8 near bytes 74 e2 20

Failed to write 299 bytes to filehandle: Broken pipe
  in block <unit> at doc/Type/CurrentThreadScheduler.pod6 line 15

Failed to write 4541 bytes to filehandle: Broken pipe
  in block <unit> at doc/Type/Rational.pod6 line 172

Failed to write 6449 bytes to filehandle: Broken pipe
  in block <unit> at doc/Language/setbagmix.pod6 line 235

xt/aspell.t ................ 
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 846/846 subtests 
xt/braces.t ................ ok
xt/camelia-invocations.t ... ok
xt/check-signatures.t ...... skipped: To run check-signatures, please specify the path to the Rakudo git checkout with the RAKUDO_SRC environment variable
xt/double-dots.t ........... ok
xt/duplicates.t ............ ok
xt/examples-compilation.t .. ok
xt/final-newline.t ......... ok
xt/headings.t .............. ok
xt/lib-test-files.t ........ ok
xt/links-not-links.t ....... ok
xt/perl-nbsp.t ............. ok
xt/return-type.t ........... ok
xt/routine-definitions.t ... ok
xt/search-categories.t ..... ok
xt/space-after-comma.t ..... ok
xt/trailing-whitespace.t ... ok
xt/type-graph.t ............ ok
xt/vim-mode.t .............. ok
xt/word-variants.t ......... ok
xt/words.t ................. ok

Test Summary Report
-------------------
xt/aspell.t              (Wstat: 65280 Tests: 0 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 846 tests but ran 0.
Files=27, Tests=14383, 772 wallclock secs ( 2.26 usr  0.15 sys + 1392.27 cusr 86.56 csys = 1481.24 CPU)
Result: FAIL
make: *** [Makefile:68: xtest] Error 1

Note Aspell is locally installed from source to its current version: 0.68.0.

Suggestions

This occurs on all my Debian hosts running versions 10 or 11 (Debian is the the only OS I use except Windows which does not have Raku installed). I plan to try the same test on an Ubuntu host.

UPDATE

I get the same results with Ubuntu 22.

[coke]

For the version that is failing; please be specific about the OS version & the aspell version, and which language packs you have installed for aspell

[tbrowder]

Failing on:

1. Debian 11, Aspell 0.68.0, "en"
2. Ubuntu 22, Aspell 0.68.0, "en"

[coke]

Do you need to be running all of xtest? do you get the same issue running e.g. 'xt/aspell.t doc/Type/CurrentThreadScheduler.pod6'? do you need to be running against more than one file? If you run aspell.t multiple times does it fail on the same file each time?

what version of rakudo are you using?

[coke]

Failed to write 299 bytes to filehandle: Broken pipe sounds like the aspell process was killed and you can't write to its stdin.

[coke]

please experiment with the env var RAKUDO_MAX_THREADS (might need at least 2)

[tbrowder]

Do you need to be running all of xtest?

I get the same problem running classtut.po6 only on the master branch.

do you get the same issue running e.g. 'xt/aspell.t

Yes

so you need to be running against more than one file?

No

If you run aspell.t multiple times does it fail on the same file each time?

Yes

what version of rakudo are you using?

2022.12

experiment with RAKUDO_MAX_THREADS

I tried 2 and 8, no help

[coke]

also what is raku -e 'say $*SCHEDULER.max_threads' (64 here on an 8 core box)

[tbrowder]

I get the same: 64 (also 8 cores)

[tbrowder]

I still haven't heard of anyone else successfully running on Linux. Most devs running doc tests seem to be Mac ppl.

[codesections]

I've repeatedly run the xt/aspell.t test on Linux [edit: on multiple distros] (including just now, to make sure it still works). I ran into some trouble initially, but added instructions in #3597 and haven't had any issues since then.

[tbrowder]

I've repeatedly run the xt/aspell.t test on Linux [edit: on multiple distros] (including just now, to make sure it still works). I ran into some trouble initially, but added instructions in #3597 and haven't had any issues since then.

Can you be very specific about ONE set of variables that worked:

1. the OS and verisons that worked (I pefer Debian if that worked)
2. the specific Aspell version and dictionaries used

Thanks.

[codesections]

I don't have a Debian box with Rakudo installed at the moment, but I can run xt/aspell.t on the following two distros (note that I tested two Rakudo versions on Void):

Distro: GNU Guix 1.3
Rakudo: v2022.07
Aspell: International Ispell Version 3.1.20 (but really Aspell 0.60.8)
Dictionary: en 2020.12.07

Distro: Void Linux 5.19.17
Rakudo: v2022.07, v2022.12-1-gd52342eb0
Aspell: International Ispell Version 3.1.20 (but really Aspell 0.60.8)
Dictionary: en 2020.12.07

[tbrowder]

Thanks, David. Are you running those as VMs on your own host? Or a docker somehow that I could replicate?

What I'm trying to do is use the identical setup as you (or anyone else who is having success). It's easiest for me to use a distro available on Digital Ocean, Linode, or Vultr. Generally that is Debian, Ubuntu, or Centos, or similar.

[codesections]

Thanks, David.

Glad I could help (but it's "Daniel", actually).

Are you running those as VMs on your own host?

That's on my own hosts (Guix on a desktop, void on a laptop).

It's easiest for me to use a distro available on Digital Ocean, Linode, or Vultr.

I have a debian server that I can test on later today or tomorrow – it just doesn't have raku/the docs site installed at the moment.

[tbrowder]

(but it's "Daniel")

Daniel, I know that, sorry, please chalk it up to a "senior moment," I just recently talked to a "David."

And thanks for your help. I need to find a good, repeatable solution for testing docs so @[Coke] won't ban me! I'm looking for a Mac Mini but having a budget crunch a the moment.

I do have a new local host I was going to install Debian on. Are Guix or Void good choices for a long-time Debian user? I'll have to look at them. I have been considering using FreeBSD but that's a bit of a learning curve.

[patrickbkr]

@tbrowder: I bought a M1 Mac Mini two years ago to do the rakudo.org precomp releases. I sold it recently, because there are now online services that offer access to M1 Mac machines. I have applied and been granted a free MacStadium account for my work on Rakudo. I believe you can to.

[tbrowder]

@patrickbkr, thanks, great idea!

[tbrowder]

I'm willing to provide access to a Debian host on one of the cloud services iif someone will use it to find out why I'm having problems with aspell.t on Debian. Just let me know here or on IRC #raku.

[codesections]

I just ran the xspell test on my debian buster server (on linode, incidentally) without any problems (after updating the dictionary as described in the aspell test file – that's the bit of setup instructions I mentioned upthread).

Here's the info: (I know it's an old Rakudo version; let me know if you'd like me to update – it's just a slow server, so that'll take a bit).

Distro: Debian Buster
Rakudo: v2021.08
Aspell: International Ispell Version 3.1.20 (but really Aspell 0.60.8)
Dictionary: en 2020.12.07

Are Guix or Void good choices for a long-time Debian user?

Probably not. Both are rolling release distros (and Guix in particular still has some growing pains to work through – though it's super promising in the long term).

As an aside, I'm a bit surprised to hear you say that you Raku developers seem more likely to use Macs. I haven't done any formal survey or anything, but my sense has been that Linux is 60%+. But maybe that's wildly wrong…

[tbrowder]

Daniel, I followed those instructions. I also tried to search for bad utf-8 chars in all the pertinent input files using "grep -axv '.*'" (from reddit or stack overflow) and thought I found some bad chars in some of the pws files, but that changed later. I'm kinda pulling my hair out here because I can't seem to eliminate any possibilities to zero in on the problem (remember the definition of insanity), especially when I seem to be the only person having the problem.

Regarding Macs, at least two of the prominent devs do: Lizmat and [Coke]. I think Tony-o might, too. So I may have ovestated the case. But neither one has a problem with aspell.t.

[codesections]

Hmm, I reran with an up-to-date version of Rakudo (v2022.12) and it still ran fine on the debian buster linode server, so I really have no idea what the issue could be.

I'll spin up a totally new linode server tomorrow and check with a clean install and see if I can reproduce the issue – it really seems like we should be able to get to the bottom of this.

[coke]

your errors here are about failures writing to processes, not invalid utf8. that was on the other report, I think.

[tbrowder]

I don't know much about threading, but looking at the docs for Lock, shouldn't we be using Lock::Async instead? Is it possible in my part of the universe Lock doesn't work?

[coke]

Yup, that seems like a reasonable ask, will update tonight if no one else gets to it.

[coke]

Was a drop in replacement, done, please try

[coke]

We used Lock in 3 places in the test suite, but only aspell has the need to write to the process that was launched, so it makes sense you'd only experience issues there.

[tbrowder]

I tried with the Lock::Async change and my runs still fail.

I see that the Makefile provides options for using a dockerfile. I tried make docker-xtest but no go. How can I get a dockerfile to try that?

[tbrowder]

I've asked @JJ to tell repo users how to use the Makefile Docker targets which may help isolate my problem during aspell testing.

[coke]

I am unable to duplicate your error on the debian box you gave me access to. I followed the following directions:

sudo apt-get install build-essential git libssl-dev
git clone https://github.com/rakudo/rakudo.git
cd rakudo
perl Configure.pl --gen-moar --gen-nqp -- backends=moar
make
make install
cd
export PATH=$HOME/rakudo/install/bin:$HOME/rakudo/install/share/perl6/site/bin:$PATH
git clone https://github.com/ugexe/zef.git
cd zef
raku -I. bin/zef install .
cd
git clone https://github.com/raku/doc.git
cd doc
sudo apt-get install aspell
xt/aspell.t doc/Language/*tut*.pod6

(zef wasn't needed)

$ xt/aspell.t doc/Language/*tut*.pod6
1..6
@(#) International Ispell Version 3.1.20 (but really Aspell 0.60.8)
not ok 1 - doc/Language/classtut.pod6 (text) has 2 spelling errors
# Failed test 'doc/Language/classtut.pod6 (text) has 2 spelling errors'
# at xt/aspell.t line 132
#     Errors:
#     =======
#     runtime
#     runtime
ok 2 - doc/Language/classtut.pod6 (code) has 0 spelling errors
ok 3 - doc/Language/grammar_tutorial.pod6 (text) has 0 spelling errors
ok 4 - doc/Language/grammar_tutorial.pod6 (code) has 0 spelling errors
ok 5 - doc/Language/optut.pod6 (text) has 0 spelling errors
ok 6 - doc/Language/optut.pod6 (code) has 0 spelling errors
# You failed 1 test of 6

This is just a regular test failure due to the aspell library not being up to date (which it isn't)

[coke]

@tbrowder reports that on his box he still gets the error; I no longer have rates sudo on the box so I cannot see what his environment is. Mine is below (on a fresh login, doesn't include the PATH updates)

$ env
SHELL=/bin/bash
PWD=/home/coke
LOGNAME=coke
XDG_SESSION_TYPE=tty
MOTD_SHOWN=pam
HOME=/home/coke
LANG=en_US.UTF-8
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
LC_TERMINAL=iTerm2
SSH_CONNECTION=104.228.7.74 55286 155.138.200.44 22
XDG_SESSION_CLASS=user
TERM=xterm-256color
USER=coke
LC_TERMINAL_VERSION=3.4.18
SHLVL=1
XDG_SESSION_ID=115
XDG_RUNTIME_DIR=/run/user/1001
SSH_CLIENT=104.228.7.74 55286 22
PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
SSH_TTY=/dev/pts/0
_=/usr/bin/env

[coke]

got sudo back; seems to work fine if I run it as @tbrowder

$ sudo su - tbrowde
$ export PATH=$HOME/rakudo/install/bin:$PATH
$ echo $PATH
/home/tbrowde/rakudo/install/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
$ cd doc
$ ../rakudo/install/bin/rakudo xt/aspell.t doc/Language/*tut*
1..6
@(#) International Ispell Version 3.1.20 (but really Aspell 0.60.8)
ok 1 - doc/Language/classtut.pod6 (code) has 0 spelling errors
not ok 2 - doc/Language/classtut.pod6 (text) has 2 spelling errors
# Failed test 'doc/Language/classtut.pod6 (text) has 2 spelling errors'
# at xt/aspell.t line 132
#     Errors:
#     =======
#     runtime
#     runtime
ok 3 - doc/Language/grammar_tutorial.pod6 (text) has 0 spelling errors
ok 4 - doc/Language/grammar_tutorial.pod6 (code) has 0 spelling errors
ok 5 - doc/Language/optut.pod6 (code) has 0 spelling errors
ok 6 - doc/Language/optut.pod6 (text) has 0 spelling errors
# You failed 1 test of 6

[coke]

Anyone have any ideas about what else @tbrowder is doing that we can compare?

[coke]

@tbrowder - can you post the env you see when you login directly so there's no potential crosstalk with my env?

[tbrowder]

On Sun, Jan 1, 2023 at 16:31 Will Coleda ***@***.***> wrote: @tbrowder <https://github.com/tbrowder> - can you post the env you see when you login directly so there's no potential crosstalk with my env?

Yes, I will do that today.

…

-Tom

[coke]

@tbrowder - are you still having this issue? I'm not sure what else I can do since I cannot duplicate it using your own setup.

[tbrowder]

Sorry I've been out of touch lately. I will try one more time on a new, local host.

[coke]

Without further info from @tbrowder, closing this ticket - I had it working on his OS setup.

Regards.