Reorganize package layout of repo

[stevvooe]

The current repo layout is organized around being "the registry". This means that the main handlers and webapp functionality are in the root package. This organization does not seem compatible with the current plans. I propose we do the following:

• Move registry package out of repo root #26 Carve out a distribution package in the repository root. Definitions for distribution related services and objects should make up this package as we build out this project.
• Move registry package out of repo root #26 Move the web application components to a registry package.
• Breakup common package #24 Break the common package down into constituent packages. These kinds of packages are never a good idea and end up being a ghetto. For example, this package includes regular expressions, tarsum parsing and a stringset, which are unrelated. No reason to group these just because they don't belong elsewhere.
• Move manifest to discrete package  #21 Create a package for dealing with manifests and their signatures. Currently, this is in storage. This needs to be a discrete package, allowing one to create, sign and verify manifests. I am of the opinion that this should be owned by distribution and should be part of the interface to docker core.

I'm sure this is incomplete, so suggestions are welcome.

[stevvooe]

cc @jlhawn @dmcgowan @dmp42 @BrianBland @AndreyKostov

[wking]

I'm not sure how these repository organization descisions are made, but I'd prefer one repostory per application/package. So docker/registry for the registry webapp, docker/manifest for the manifest manipulation and signing, docker/tarsum with tarsum parsing, calculation, and verification, …. Many of these packages are only interesting to a subset of registry developers, and having them managed in their own repositories makes it easier to focus on your particular part without the noise of the other orthogonal packages. Package maintainers can independently version and push their libraries to the Go package registry, and applications using the packages (like docker/registry) can pull them down the same way they pull in non-Docker Go packages. I'd keep this repository as a central clearinghouse for high-level docs and cross-cutting bugs, but push package- and application-specific bugs towards their specific repositories.

[stevvooe]

@wking When I'm talking about packages, I am specifically referring to Go packages, which map directly to directories in a repository or the GOPATH. In general, Go packages should organize a single idea, such as tarsum or manifests. Repositories, in turn, organize a collection of ideas (packages), such as the components that make up the registry or the distribution component of docker.

Having a repository per package is overkill. It makes complex, cross-cutting changes, where interfaces may be new or non-existent, nearly impossible. It also makes reorganization much harder to do. We are looking to bring the concept of distribution together, so a single repo makes sense. As interesting divisions emerge, we may split out individual packages, but we have a lot of work to do before we make that happen.

If you have any feedback on the specific plans outlined above, please feel free to comment.

[wking]

On Fri, Jan 02, 2015 at 05:28:30PM -0800, Stephen Day wrote:

If you have any feedback on the specific plans outlined above,
please feel free to comment.

The package breakdown outlined above makes sense to me. I was just
explaining why I think we currently have a “The current repo layout is
organized around being "the registry"”, and how I think that's a good
thing for the registry code. We both agree that it's good to have the
code separated out into clearly-directed packages, and whether or not
those packages live in the same repository is a secondary issue. I
just brought it up since we seemed to be shifting repositories around,
and if you did agree that each package deserved its own repository,
now would be a good time to make that move.

[stevvooe]

Closed by #26.