01-index entries missing package.json

[hvr]

The following entries have a .cabal but no package.json file in 01-index:

aivika-transformers/5.3.1/aivika-transformers-5.3.1.tar.gz
amazonka-codedeploy/1.4.0/amazonka-codedeploy-1.4.0.tar.gz
amazonka-config/1.3.5/amazonka-config-1.3.5.tar.gz
amazonka-importexport/1.3.7/amazonka-importexport-1.3.7.tar.gz
amazonka-importexport/1.4.0/amazonka-importexport-1.4.0.tar.gz
amazonka-s3/1.3.6/amazonka-s3-1.3.6.tar.gz
amazonka-sdb/1.4.0/amazonka-sdb-1.4.0.tar.gz
eventloop/0.5.0.0/eventloop-0.5.0.0.tar.gz
gi-gdkpixbuf/0.2.31.12/gi-gdkpixbuf-0.2.31.12.tar.gz
gi-gio/0.2.46.13/gi-gio-0.2.46.13.tar.gz
gi-girepository/0.1.44.12/gi-girepository-0.1.44.12.tar.gz
glue-ekg/0.4.4/glue-ekg-0.4.4.tar.gz
gnuidn/0.2.2/gnuidn-0.2.2.tar.gz
gogol-admin-reports/0.2.0/gogol-admin-reports-0.2.0.tar.gz
gogol-autoscaler/0.0.1/gogol-autoscaler-0.0.1.tar.gz
hops/0.4.0/hops-0.4.0.tar.gz
hprotoc/2.1.11/hprotoc-2.1.11.tar.gz
hzaif/0.1.0.0/hzaif-0.1.0.0.tar.gz
llvm-hs/llvm-hs-5.1.1/llvm-hs-5.1.1.tar.gz
repa-io/3.4.0.2/repa-io-3.4.0.2.tar.gz
wai-middleware-prometheus/0.2.0/wai-middleware-prometheus-0.2.0.tar.gz

---

EDIT: I believe to have figured out why this occurs (and to confirm that theory I'm quite confdident in, I've put in place additional logging as per 4af31a5), and after consulting with Duncan the suggested plan forward is to refactor hackage's hook interface to allow to write package.json + .cabal file in a single transaction.

---

EDIT2: This issue is resolved at last

[edsko]

Note that glue-ekg has two versions after 0.4.4 (by April 15).

[edsko]

Dates of these releases don't seem be related to April 15:

• amazonka-codedeploy/1.4.0 released Mar 21 07:11:54 UTC 2016, prev Dec 19 06:28:25 UTC 2015, next May 9 19:13:50 UTC 2016
• amazonka-config/1.3.5 released Oct 28 05:09:42 UTC 2015, prev Oct 25 19:03:48 UTC 2015, next Nov 21 10:44:40 UTC 2015

...

[gbaz]

@edsko it would be good to review the hackage-security code (I guess?) to see what could cause this. I can take a swing myself, but any hints or suggestions as to potential trouble spots from someone more familiar would be quite welcome.

[gbaz]

what problems downstream (if any) does this cause? Also, can we "patch up" the tarball to add them in later?

[peti]

What problems downstream (if any) does this cause?

The tool that generates the Haskell package set for Nix was written with the assumption that all tarball hashes can be retrieved from 01-index.tar, so when I actually ran that tool for the first time it fell apart because that assumption turns out to be wrong.

[hvr]

@peti can't you just treat all entries w/o a package.json entry as if they didn't exist (or as if they were corrupt or non-retrievable -- that's actually something the tool will have to cope with anyway)?

[peti]

@hvr, yes, I could do that.

That policy will raise some questions, though, because Hackage itself happily advertises the existence of releases like http://hackage.haskell.org/package/gnuidn-0.2.2 and allows users to download them. So it feels odd for my tool to say "this release does not exist". It would be nice if those corrupt and non-retrievable releases were recognizable as such on hackage.haskell.org, too.

[hvr]

@peti well, to be fair, hackage's internal database is corrupted insofar as it's missing the checksums; that's going to be fixed -- this isn't a normal state that hackage's UI needs to be able to reflect.