Home

Matt Iversen edited this page Mar 7, 2014 · 22 revisions
Clone this wiki locally

A Breakdown of the Packaging Problems

Or in other words, a (likely in-exhaustive) top down view of big problems to solve in Python Packaging.

Disclaimer: This is currently an open/unofficial effort towards grokking the space of packaging problems.

Metadata

Versions:

  • 1.0 - key/value
  • 1.1 - key/value
  • 1.2 - key/value - initial support distribution depedency
  • 2.0 - json

Use:

Problems:

  • Change wheel to indicate it's saving METADATA 1.2?
  • Update setuptools to output METADATA 1.2 information, in pure .dist-info form
  • Work towards allowing more declarative specification of metadata in packages, than that currently encourages by setuptools
  • Figure out best practices for platform and ABI compatibility, requires a lot of communication, but currently going ok

Dependency Resolution

Problems:

Solutions:

  • Introduce a proper dependency resolution solver to pip
  • Ingestion of package metadata for dependency information could be separated from download package contents, to make installs faster and more customizable.
  • Allow for more descriptive version and dependency information in metadata (python versions, implementations, platforms, abis, etc).
  • A dependency solver needs a reliable database of of what is already installed and all dependency metadata relating to that. Currently metadata 1.2 is minimum viable for this, but 2.0 would be preferred for platform/abi/etc dependencies

Prior art

Packages, Installation and Formats

Wheel:

For future Wheel-Versions

  • Add a controlled, sanctioned way of making zip-importable wheels?
  • " " " " " " " executable wheels? (python mywheel.whl or python -m mywheel.whl?)
  • Fix and version metadata inclusion
  • wheel filenames are currently used to tell a lot of compatibility information - is this appropriate? When you start to practically factor in platforms and ABIs, people can get huge filenames. Where / when should the line be drawn? Could/Should this be moved away from? i.e., when all this information comes from metadata associated with the content and available externally to the package, it no longer needs to be in the filename.

Platforming and ABIs

Setuptools

  • Update output information to metadata 1.2, to work towards viable dependency information in local package database

Logistics and Documentation

In progress

Organisation of Mailing Lists & Communication Channels

These could be restructured into two central channels per communication medium - one for users to discuss problems and questions (e.g "pypa-users" or "pypa"), and one for developers to discuss work on projects (e.g "pypa-dev"). If so, such reorganisation should be announced to people!

Furthering Communication

This needs to be improved, a lot. See

I'm sure there are quite a few other examples I've missed that all could be addressed systematically.