Somewhat stabilize repository metadata format

[lberrymage]

The Problem

Accrescent needs repository metadata to find and download apps in the store. The current repodata format (as described by the data classes found here and the JSON files here and here) comes with strong cryptographic guarantees of app validity. However, it has a few significant limitations, namely:

• It expects developers to maintain a signify key for signing subrepodata and use it when updating their apps. This both raises the barrier to entry for app developers unfamiliar with signify (presumably most of them) and complicates the app upload process.
• It pins developer usernames, meaning developers would need to request the root repodata signer (myself) to change their username every time they changed their GitHub username since we use GitHub as our authentication provider. This is unnecessary and unsustainable.

New Format Goals

A new format is in order for the stable release. It should (ideally) accomplish the following:

• Protect new and existing users from malicious downloads in the event the repository is compromised
  • Validate first-time app installations, improving on Android's TOFU model for app signatures
  • Be resistant to downgrades
• Be usable for developers
  • Allow developers to upload updates to existing apps without waiting for external review/signatures (provided the update meets our eligibility requirements for unreviewed uploads, documentation for which is in-progress at App Quality Control devconsole#1
  • Only require developers to use their existing tools (i.e. not require them to use signify)
• Provide a way to fetch split APKs generated by bundletool

Non-goals

This repodata format overhaul will not define an absolutely final format, so a few things are out-of-scope for this overhaul. To name a few:

• App descriptions
• App categorization or tagging system
• App developer information
• Different release channels
• Install-time feature modules

New design (so far)

An informal proposal of the new format is being put together at https://hackmd.io/@lberrymage/accrescent-repodata. It will eventually be moved elsewhere.

Related: accrescent/accrescent#9

[lberrymage]

This is now implemented in accrescent/accrescent@c317c28. All that's left to do is the following:

• Update doc to reflect changes in how unsigned repodata is handled
• PDF-ify and host doc on Accrescent website
• Get doc reviewed by someone knowledgeable in cryptography

[lberrymage]

I'm closing this issue with the argument that the repodata format has been "somewhat stabilized." It would still be good to have Accrescent's repodata verification and other inner workings reviewed before the public beta. However, the project is rapidly changing at the moment and it wouldn't make sense to have current processes/code undergo extensive review when they may change significantly in the near future as new needs are discovered and addressed.