Modules which cannot be accessed are not included in projects.json

[CIAvash]

If a module cannot be reached because the host server is down, the module won't be included in projects1.json and projects.json. And the module manager(zef) would not be able to find the module, although the module exists, it just could not be accessed.

Apparently projects1.json is generated in server/updatelist.pl and projects.json is generated in updatelist.pl.

It seems zef has 4 mirrors for getting the list of modules, first two of which are projects1.json and projects.json.

Related IRC logs

[moritz]

What's the point of offering a module for installation when it cannot be installed?

I consider this a feature.

[CIAvash]

Because it misleads the user. If they try to install a module and zef tells them it cannot find it, then they think it might have been removed or something. They might not try to install it again. But if zef shows an error that tells the user it cannot access it(for example "500 internal server error"), then they know there is something wrong with the server.

Yesterday I tried to install my own module and I was really confused, I thought maybe somebody accidentally removed it from the list. Then After some investigation I found out that Gitlab was down and that was what caused it.

[rafaelschipiura]

You mean that the error messages should be more specific?

[CIAvash]

Yes, but It's not just the error message. If projects1.json(the generated list) is incomplete, the only thing zef can say is that it cannot find the module.

[JJ]

Not clear what could be the solution here. Any suggestion?

[CIAvash]

One solution was offered by Zoffix: "probably could add a conditional in here that would read use dist's info from previous run if we fail to fetch the new one".

But there are 2 scripts generating 2 different lists. And then there are two other mirrors zef uses, I don't know if they have this problem or not.

cc @ugexe