Missing fresh distributions on Recent Uploads page

[eserte]

Currently the most recent distribution listed is "Museum-TePapa-0.02", uploaded 13 hours ago. However, https://pause.cpan.org/pause/authenquery?ACTION=tail_logfile&pause99_tail_logfile_1=10000&pause99_tail_logfile_sub=Tail+characters lists at least four distribution uploaded since then.

[Leont]

It's not just that, dists with a newly uploaded release are missing from the MetaCPAN website entirely, which is probably the bigger problem.

[sanko]

And smokers have missed every dist uploaded in the same period of time.

Seems to have started with Crypt::Passphrase::Scrypt which is the first dist uploaded to PAUSE last Saturday. The last dist uploaded on Friday was Data::Roundtrip which currently has 200+ reports.

[eserte]

Interesting. In http://metabase.cpantesters.org/tail/log.txt I see a mixture of lines which look normal, e.g.

[2023-03-15T07:04:59Z] [David Cantrell (DCANTRELL)] [pass] [PETDANCE/Perl-Critic-1.150.tar.gz] [x86_64-linux-thread-multi] [perl-v5.34.1] [b32d9956-c2ff-11ed-9baa-7a9c16f4574e] [2023-03-15T07:04:59Z]

and lines where the distname part is empty, e.g.

[2023-03-15T07:04:59Z] [Slaven Rezić] [unknown] [] [amd64-freebsd] [perl-v5.18.4] [b34068b0-c2ff-11ed-9c58-07e26d8775ea] [2023-03-15T07:04:59Z]

@dagolden: do you have an idea why the distname part is missing? Is there some validation around, and reports with distributions not listed somewhere (MetaCPAN?) are partially discarded?

[jkeenan]

Interesting. In http://metabase.cpantesters.org/tail/log.txt I see a mixture of lines which look normal, e.g.

[2023-03-15T07:04:59Z] [David Cantrell (DCANTRELL)] [pass] [PETDANCE/Perl-Critic-1.150.tar.gz] [x86_64-linux-thread-multi] [perl-v5.34.1] [b32d9956-c2ff-11ed-9baa-7a9c16f4574e] [2023-03-15T07:04:59Z]

and lines where the distname part is empty, e.g.

[2023-03-15T07:04:59Z] [Slaven Rezić] [unknown] [] [amd64-freebsd] [perl-v5.18.4] [b34068b0-c2ff-11ed-9c58-07e26d8775ea] [2023-03-15T07:04:59Z]

[snip]

I can confirm this. At this moment, http://metabase.cpantesters.org/tail/log.txt has 286 entries with [] where I would expect a tarball. I suspect many of these are from the upload of ExtUtils-MakeMaker-7.68.tar.gz yesterday. I did a CPANtesters run on that distribution. By all indications the upload was successful, but all that the metabase log file was showing was:

[2023-03-15T13:11:27Z] [James E Keenan] [pass] [] [amd64-freebsd-thread-multi] [perl-v5.37.10] [e522c03e-c332-11ed-ae8e-65eea9ff8ba7] [2023-03-15T13:11:27Z] 

(I tried sending email to cpan-testers-dicuss@perl.org but that, too, was down!)

[andk]

Signs of good news have appeared: reports of new uploads have started to flow through cpantesters DB again. Sample report: http://www.cpantesters.org/cpan/report/54856580-c4d7-11ed-8795-282a6e8775ea

I have seen reports coming in for these (dist,version) tuples:
(Amazon-API,2.0.4)
(File-HTTP,1.11)
(Future,0.50)
(Future-AsyncAwait,0.65)
(Future-XS,0.09)
(Geo-Location-Point,0.08)
(Log-Mini,0.3.0)
(OPCUA-Open62541,2.01)
(Parallel-Pipes,0.200)
(Sah-Schemas-WordList,0.004)
(mb,0.53)

I don't know more about the nature of the outage and in particular the fate of the uploads from LEONT/Crypt-Passphrase-Scrypt-0.003.tar.gz to KUERBIS/Term-Choose-1.758.tar.gz.

[Leont]

Yeah it seems things are working again, uploads also appear timely now.

[jkeenan]

Yeah it seems things are working again, uploads also appear timely now.

I'm dubious about that. For example, neither of the following URLs show any reports for ExtUtils-MakeMaker-7.68.tar.gz:

http://fast-matrix.cpantesters.org/?dist=ExtUtils-MakeMaker

http://matrix.cpantesters.org/?dist=ExtUtils-MakeMaker

[Leont]

I'm dubious about that. For example, neither of the following URLs show any reports for ExtUtils-MakeMaker-7.68.tar.gz:

Yeah, it seems it isn't retroactively fixed for dists released during the gap, but it works for all distributions released since md-0.53 last Friday.

[jkeenan]

I'm dubious about that. For example, neither of the following URLs show any reports for ExtUtils-MakeMaker-7.68.tar.gz:

Yeah, it seems it isn't retroactively fixed for dists released during the gap, but it works for all distributions released since md-0.53 last Friday.

I could understand that reports sent during the (as yet unexplained) gap would still be lost. What I cannot understand is that the results pages seem to be refusing to report on distributions released during the gap.

Just now I re-tested ExtUtils-MakeMaker-7.68.tar.gz (making sure to comment out the previous entry in ~/.cpanreporter/reports-sent.db so as to guarantee that a new report would be sent. I got the expected output in my terminal:

CPAN::Reporter: Test result is 'pass', All tests successful.
CPAN::Reporter: preparing a CPAN Testers report for ExtUtils-MakeMaker-7.68
CPAN::Reporter: sending test report with 'pass' via Metabase

However, this was logged in http://metabase.cpantesters.org/tail/log.txt as:

[2023-03-20T15:41:28Z] [James E Keenan] [pass] [] [amd64-freebsd-thread-multi] [perl-v5.37.10] [adbc0496-c735-11ed-b1b9-9b59aaff8ba7] [2023-03-20T15:41:28Z]

The name of the tarball is still missing. As I write there are 132 entries in the metabase tail with [].

[jkeenan]

I'm dubious about that. For example, neither of the following URLs show any reports for ExtUtils-MakeMaker-7.68.tar.gz:

Yeah, it seems it isn't retroactively fixed for dists released during the gap, but it works for all distributions released since md-0.53 last Friday.

This is still the case today (Mar 30 2023). For example, at http://matrix.cpantesters.org/?dist=ExtUtils-MakeMaker (and at the corresponding fast-matrix URL), we have reports for ExtUtils-MakeMaker-7.70 -- but still none for 7.68.

It appears that those CPAN distros released during this outage have entered a permanent memory-hole. See also https://www.nntp.perl.org/group/perl.cpan.testers.discuss/2023/03/msg4578.html.

Pinging some members of the metacpan organization: @autarch @oalders @toddr @atoomic @wolfsage @haarg. If this is not the correct place to report this problem, can you point us in a better direction?

Thank you very much.

[oalders]

What we probably need to do at this point is run an audit of missing packages and re-index them so that they get backfilled. Unfortunately, that tool doesn't really exist. I was thinking that I would probably take care of this at PTS in a few weeks as I'm really pressed for spare time right now. Having said that, if someone wants to provide me with URLs for dists they want to see re-indexed immediately, I can run some of these manually. Something like https://cpan.metacpan.org/authors/id/B/BI/BINGOS/ExtUtils-MakeMaker-7.69_01.tar.gz for each missing release that you want to see indexed would be good enough.

[andk]

@oalders : this is the most probably complete list of what the spooky permanent memory-hole comprises:

TTKCIAR/File-Valet-1.11.tar.gz PERLANCAR/Complete-Sah-0.010.tar.gz VDANJEAN/StorageDisplay-1.2.1.tar.gz FELIPE/Promise-XS-0.19.tar.gz INGY/Lingy-0.1.0.tar.gz BDFOY/Business-ISBN-Data-20230316.001.tar.gz MAMAWE/WebService-IdoitAPI-v0.2.0.tar.gz BDFOY/Business-ISMN-1.203.tar.gz HIROSE/AWS-CLIWrapper-1.25.tar.gz KIMOTO/SPVM-Sys-0.44.tar.gz JHTHORSEN/Mojo-TFTPd-0.05.tar.gz TULAMILI/App-ggdrv-0.031.tar.gz GUGOD/PerlX-ScopeFunction-0.03.tar.gz PERLANCAR/App-BPOMUtils-RPO-Checker-0.010.tar.gz KIMOTO/SPVM-File-Spec-0.07.tar.gz KIMOTO/SPVM-File-Find-0.02.tar.gz KIMOTO/SPVM-IO-0.19.tar.gz KIMOTO/SPVM-File-Basename-0.04.tar.gz UTASHIRO/App-ansifold-1.18.tar.gz VOJ/pica-data-2.07.tar.gz UTASHIRO/Term-ANSIColor-Concise-2.0201.tar.gz HOUSTON/Net-EPP-Registry-Nominet-0.07_02.tar.gz GUIDO/Template-Plugin-Gettext-0.8.tar.gz GUIDO/Locale-XGettext-0.8.tar.gz SREZIC/App-orgdaemon-1.12.tar.gz LEONT/Crypt-Passphrase-Linux-0.003.tar.gz PERLANCAR/Sah-Schemas-Data-Sah-0.003.tar.gz DCANTRELL/Data-Compare-1.28.tar.gz LEONT/Crypt-Passphrase-0.010.tar.gz BINGOS/ExtUtils-MakeMaker-7.68.tar.gz VVELOX/Time-Piece-Guess-0.0.3.tar.gz VVELOX/Virani-0.0.1.tar.gz KUERBIS/App-DBBrowser-2.320.tar.gz RHANDOM/Net-Server-2.014.tar.gz KILINRAX/HTML-Strip-2.11.tar.gz JBARRETT/Alien-RtMidi-0.05.tar.gz JBARRETT/MIDI-RtMidi-FFI-0.04.tar.gz PLICEASE/FFI-Platypus-2.07.tar.gz UTASHIRO/App-Greple-xlate-0.17.tar.gz JHTHORSEN/Mojo-Tar-0.01.tar.gz EDF/Geo-Coder-OpenCage-0.36.tar.gz IVANWILLS/App-VTide-v0.1.21.tar.gz MARKOV/Math-Formula-0.16.tar.gz JIMAVERA/ODF-lpOD_Helper-2.000.tar.gz ETJ/PDL-2.081_02.tar.gz EXODIST/Test-Simple-1.302194.tar.gz DJERIUS/Astro-FITS-CFITSIO-Utils-0.14.tar.gz PERLANCAR/Sah-Schemas-Perl-0.048.tar.gz NHORNE/Class-Simple-Readonly-Cached-0.09.tar.gz QORG/App-Sakisafe-v2.4.3.tar.gz WHINDS/Bitcoin-RPC-Client-0.12.tar.gz DERF/Travel-Status-DE-IRIS-1.82.tar.gz HAARG/Pod-Spell-1.26.tar.gz RWP/oo_sub/oo_sub-v1.1.0.tar.gz NLNETLABS/Net-DNS-1.37.tar.gz TIMLEGGE/XML-Sig-0.61-TRIAL.tar.gz TIMLEGGE/XML-Enc-0.09-TRIAL.tar.gz TIMLEGGE/Net-SAML2-0.65-TRIAL.tar.gz LANCEW/Webservice-Judobase-0.09.tar.gz BIGPRESH/Bot-BasicBot-Pluggable-1.30.tar.gz EXODIST/Test2-Harness-UI-0.000136.tar.gz KARJALA/RxPerl-v6.27.1.tar.gz AWNCORP/Venus-2.32.tar.gz INA/Jacode/Jacode-2.13.4.30.tar.gz NFERRAZ/OpenAI-API-0.27.tar.gz ZEFRAM/Perl4-CoreLibs-0.005.tar.gz ETHER/OpenAPI-Modern-0.041.tar.gz PERLANCAR/Clipboard-Any-0.010.tar.gz LEADER/Crypt-LE-0.39.tar.gz GENE/Music-CreatingRhythms-0.0602.tar.gz ETJ/Alien-Gnuplot-1.041.tar.gz ETHER/JSON-Schema-Modern-0.565.tar.gz MRDVT/Power-Outlet-0.48.tar.gz POLETTIX/Template-Perlish-1.64.tar.gz LEONT/Crypt-Passphrase-Yescrypt-0.001.tar.gz LEONT/Crypt-Yescrypt-0.001.tar.gz PLAIN/Redis-Cluster-Fast-0.087.tar.gz BINARY/MooseX-Role-Validatable-0.12.tar.gz BINARY/Finance-Calendar-0.04.tar.gz HANJE/Tk-PopList-0.03.tar.gz DRCLAW/Sub-Middler-v0.2.0.tar.gz SREZIC/YAML-PP-Schema-Env-0.02.tar.gz KUERBIS/Term-TablePrint-0.156.tar.gz PERLANCAR/Bencher-Scenario-ShellQuote-Any-0.005.tar.gz RSHEAT/Museum-TePapa-0.02.tar.gz AKALINUX/AnyEvent-SparkBot-1.015.tar.gz VVELOX/CAPE-Utils-0.0.1.tar.gz RHYSADAMS/Netkit-0.02.tar.gz RHYSADAMS/App-NetkitGui-0.01.tar.gz JOROL/MARC-Schema-0.14.tar.gz POLETTIX/App-Easer-2.007001-TRIAL.tar.gz PERLANCAR/App-TextWrapUtils-0.006.tar.gz PERLANCAR/Acme-CPANModules-TextTable-0.014.tar.gz WYANT/Astro-satpass-0.129.tar.gz SANKO/LibUI-0.02.tar.gz SANKO/Affix-0.10.tar.gz LEONT/Crypt-Passphrase-Scrypt-0.003.tar.gz

[oalders]

@andk I think these all should be indexed now/

[andk]

Thank you, @oalders. This indexing run had no recognizable effect to cpantesters. The test reports I sent over night for the above listed sample did not make it into the database. Best bet for authors affected is to make a new release, as e.g. Test-Simple 1.302195 shows. No reports for Test-Simple 1.302194 made it into the database while plenty of reports for 1.302195 did. Possibly only @preaction can clarify what should help on the end of cpantesters ?