Permalink
Browse files

Add moby core component architecture doc.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
  • Loading branch information...
cpuguy83 committed Aug 16, 2017
1 parent de7e4c7 commit 4e7df4ae4d948789bbd48682ebf1f21b6288fb14
@@ -0,0 +1,58 @@
***Draft***

## Moby Core Component Architecture

Moby Core is a container platform built for modularity.
Moby Core uses a component architecture allows moby components to interact

This comment has been minimized.

@dnephin

dnephin Aug 16, 2017

which allows?

This comment has been minimized.

@cpuguy83

cpuguy83 Aug 16, 2017

Owner

:) I fixed this locally but didn't push

with each other with well defined API's in a way that enables components
to be swapped out for different implementations at packaging time.

Moby Core defines several of its own [components](#moby-core-services) along with
a default stack which implements those services. Packagers can take Moby Core as

This comment has been minimized.

@dnephin

dnephin Aug 16, 2017

"defines several interfaces along with default components which implement those interfaces"?

There seem to be a few terms used interchangeably here. We should reduce the number of terms.

defined, swap out components for different ones, and even supply completely new
components not specifically defined in the core architecture to extend
Moby Core's functionality.

### Components

A component is an isolated set of functionality that is likely useful in its
own right.

A component:
- Must communicate with other components through node-local sockets only, for

This comment has been minimized.

@cpuguy83

cpuguy83 Aug 16, 2017

Owner

From call:

  • Must provide a GRPC endpoint on node-local sockets with (@dnephin)
  • MAY expose a separate API for external consumption (@dnephin)
  • Proto files SHOULD be separate from the implementation (but not the generated files) (@dnephin)
example a unix socket or a named pipe.
- Components MAY communicate outside of the normal socket for special cases,
such as streaming I/O, but this SHOULD be deterministic based on the component's
request/response and well documented.
i. Example: When attempting to attach to a container, this might look like
`ContainerAttach(fifoIn, fifoOut, fifoErr)`, where each fifo represents a
file which would need to be opened by the receiver.
- SHOULD be easily deployed as a container (the primary packaging format for
Moby Core, see [Packaging](#Packaging)).
- MAY expose endpoints for consumption outside of the component architecture.
- SHOULD use GRPC for cross-component communication.

### Moby Core Services

Moby Core services are services provided by the Moby Core platform, including
their default implementations.

All Moby Core services MUST use GRPC for cross-component communication.

- moby-http-v1 - Provides the v1.* HTTP API
- moby-container-store - Provides basic CRUD operations for container
specs/metadata.
- moby-container-runtime - start/stop/kill/etc operations on containers

This comment has been minimized.

@AkihiroSuda

AkihiroSuda Aug 16, 2017

How does it relate to containerd?

This comment has been minimized.

@cpuguy83

cpuguy83 Aug 16, 2017

Owner

At the moment, this wraps containerd. It takes a "moby container" (e.g. docker run) and turns it into a containerd one.

This comment has been minimized.

@cpuguy83

cpuguy83 Aug 16, 2017

Owner

I've also considered implementing this in the container store where the returned container object can just be passed a containerd client and handle within that object. The issue I had with this is putting a lot of functionality into the store service.

This comment has been minimized.

@cpuguy83

cpuguy83 Aug 16, 2017

Owner

I was also considering making the ContainerSpec more like interface{} in the store which allows the consumer of the store component to define it and the store itself doesn't care.

This comment has been minimized.

@mlaventure

mlaventure Aug 17, 2017

I agree that the store shouldn't care. It will also make it seamless to move to a new version of the spec or a different one if using a bunch of components not based on the OCI specs for instance.

This comment has been minimized.

@cpuguy83

cpuguy83 Aug 17, 2017

Owner

@stevvooe suggested adding an Any field to the containerd container object that would let us store a spec in there rather than duplicating store logic.

This comment has been minimized.

@mlaventure

mlaventure Aug 17, 2017

Yes, 👍 on this.

- moby-rootfs-service - Provides access to container filesystems.

This comment has been minimized.

@Random-Liu

Random-Liu Aug 18, 2017

Does image management part of Moby Core? Does moby-rootfs-service do image management?
Wil it use containerd image service?

This comment has been minimized.

@cpuguy83

cpuguy83 Aug 18, 2017

Owner

It would definitely use the containerd image service.
The main purpose of this service as I see it (but I haven't spent much time on it) is to support operations like docker cp (and there are some other cases in docker that needs to interact with the container fs).

This comment has been minimized.

@Random-Liu

TODO: This list is far from exhaustive.

See the [service definitions](services) for these services.

### Discovery

### Packaging

One goal of the component architecture is to easily package Moby Core via the
[Moby Tool](github.com/moby/tool) as "services". This enables packaging Moby
Core in multiple formats, and easily adding/removing components to the system.
@@ -0,0 +1,47 @@
syntax = "proto3";

package moby.core.services.containers.rootfs.v1;

import "github.com/docker/docker/components/types/container/container.proto";
import "github.com/docker/docker/components/types/file.proto";

service RootfsService {
rpc ExtractRootfsArchive (ExtractRootfsArchiveRequest) returns (ExtractRootfsArchiveResponse) {}
rpc FetchRootfs (FetchRootfsRequest) returns (FetchRootfsResponse) {}
rpc StatRootfs (StatRootfsRequest) returns (StatRootfsResponse) {}
rpc DiffRootfs (DiffRootfsRequest) returns (DiffRootfsResponse) {}
}

message ExtractRootfsArchiveRequest {
moby.core.v1.types.Container container = 1;
string target = 2;
// the archive ???
}

message ExtractRootfsArchiveResponse {}

message FetchRootfsRequest {
moby.core.v1.types.Container container = 1;
string target = 2;
}

message FetchRootfsResponse {
// the archive ???
}

message StatRootfsRequest {
moby.core.v1.types.Container container = 1;
string target = 2;
}

message StatRootfsResponse {
moby.core.types.v1.FileInfo info = 1;
}

message DiffRootfsRequest {
moby.core.v1.types.Container container = 1;
}

message DiffRootfsResponse {

}
@@ -0,0 +1,27 @@
syntax = "proto3";

package moby.core.services.containers.runtime.v1;

import "github.com/docker/docker/components/types/container/container.proto";
import "google/protobuf/timestamp.proto";

service ContainerRuntimeService {
rpc StartContainer(StartContainerRequest) returns (StartContainerResponse) {}
rpc KillContainer(KillContainerRequest) returns (KillContainerResponse) {}
}

message StartContainerRequest {
moby.core.v1.types.Container container = 1;
}

message StartContainerResponse {
int32 pid = 1;
}

message KillContainerRequest {
moby.core.v1.types.Container container = 1;
google.protobuf.Duration timeout = 2;
uint32 signal = 3;
}

message KillContainerResponse {}
@@ -0,0 +1,86 @@
syntax = "proto3";

package moby.core.services.containers.store.v1;

import "gogoproto/gogo.proto";
import "google/protobuf/timestamp.proto";
import "github.com/docker/docker/components/types/container/container.proto";

service ContainerStore {
rpc CreateContainer(CreateContainerRequest) returns (CreateContainerResponse);
rpc UpdateContainer(UpdateContainerRequest) returns (UpdateContainerResponse);
rpc DeleteContainer(DeleteContainerRequest) returns (DeleteContainerResponse);
rpc GetContainer(GetContainerRequest) returns (GetContainerResponse);
rpc ListContainers(ListContainersRequest) returns (ListContainersResponse);
rpc StreamEvents(StreamEventsRequest) returns (stream ContainerStoreEvent);

This comment has been minimized.

@stevvooe

stevvooe Aug 21, 2017

I would recommend making events a global service, that allows both subscribe and publish.

This comment has been minimized.

@cpuguy83

cpuguy83 Aug 22, 2017

Owner

Yep sounds good, these definitely seemed a little out of place.

}

message CreateContainerRequest {
moby.core.v1.types.ContainerSpec spec = 1;
moby.core.v1.types.Annotations annotations = 2;
}

message CreateContainerResponse {
moby.core.v1.types.Container container = 1 [(gogoproto.nullable) = false];
}

message UpdateContainerRequest {
string container_id = 1;
moby.core.v1.types.Version version = 2;
moby.core.v1.types.ContainerSpec spec = 3;
}

message UpdateContainerResponse {
moby.core.v1.types.Container container = 1 [(gogoproto.nullable) = false];
}

message DeleteContainerRequest {
string container_id = 1;
}

message DeleteContainerResponse {}

message GetContainerRequest {
string container_id = 1;
}

message GetContainerResponse {
moby.core.v1.types.Container container = 1 [(gogoproto.nullable) = false];
}

message Filters {
repeated string container_ids = 3;
map<string,string> labels = 4;
repeated string names = 5;
repeated string volumes = 6;
repeated string network = 7;
}

message ListContainersRequest {
Filters filters = 1;
}

message ListContainersResponse {
repeated moby.core.v1.types.Container containers = 1;
}

message StreamEventsRequest {
StoreAction action = 1;
Filters container_filters = 2;
}

message StoreAction {
enum ActionType {
UNKNOWN = 0;
CREATE = 1;
UPDATE = 2;
DESTROY = 3;
}

ActionType action = 1;
}

message ContainerStoreEvent{
StoreAction action = 1;
moby.core.v1.types.Container container = 2 [(gogoproto.nullable) = false];
}
@@ -0,0 +1,76 @@
syntax = "proto3";

package moby.core.v1.types;

import "gogoproto/gogo.proto";
import "google/protobuf/timestamp.proto";
import "google/protobuf/duration.proto";

message ContainerSpec {
ContainerConfig config = 1;
ContainerHostConfig host_conifg = 2;
Annotations annotations = 3;
}

message HealthConfig {
repeated string test = 1;
google.protobuf.Duration interval = 2;
google.protobuf.Duration timeout = 3;
google.protobuf.Duration start_period = 4;
int32 retries = 5;
}

message ContainerConfig {

This comment has been minimized.

@mlaventure

mlaventure Aug 17, 2017

I really like the typeurl package of containerd 1.0 for such types (i.e. types that are highly specific to the platform/component using it). It renders things much cleaner and easier to update.

This comment has been minimized.

@cpuguy83

cpuguy83 Aug 17, 2017

Owner

How would you suggest going about using this?
What I was thinking is:
Change ContainerSpec to use a single Any to replace config/host_config, and wrap config/host_config in a different type that's namespaced to signify "this is the docker v1 container definition".

This comment has been minimized.

@mlaventure

mlaventure Aug 17, 2017

Was thinking the same :)

Then as long as we stay consistent with the type url in your Any it's easy for a component to decide if its support the type or not.

string hostname = 1;
string domainname = 2;
string user = 3;
string group = 4;
bool tty = 5;
bool open_stdin = 6;
bool stdin_once = 7;
repeated string env = 8;
repeated string cmd = 9;
HealthConfig healthcheck = 10;
bool ars_escaped = 11;
string image = 12;
repeated string volumes = 13;
string working_dir = 14;
repeated string entrypoint = 15;
string stop_signal = 16;
google.protobuf.Duration stop_timeout = 17;
repeated int32 exposed_ports = 18;

// Maybe these should be removed
// One problem is the image format is very much tied to the legacy equivelent struct
bool attach_stdout = 19; // not even sure what these attach_<foo> are used for, not used in dockerd
bool attach_stderr = 20;
bool attach_stdin = 21; // ok this one is actually used in the attach code
repeated string shell = 22; // This should really be set on the entrypoint...
repeated string onbuild = 23; // This is a builder thing, doesn't belong here
bool network_disabled = 24; // Same as host config network type=none?
}

message ContainerHostConfig {
// HostConfig stuff
}

message Version {
uint64 index = 1;
}

message Meta {
Version version = 1 [(gogoproto.nullable) = false];
google.protobuf.Timestamp created_at = 2;
google.protobuf.Timestamp updated_at = 3;
}

message Annotations {

This comment has been minimized.

@stevvooe

stevvooe Aug 21, 2017

We came up with this in swarmkit, slightly taking inspiration from the common object model in k8s. Embedding these as main fields with an any carrier might be a better approach, however, I am not sure what this is providing at this point, other than consistency.

This comment has been minimized.

@cpuguy83

cpuguy83 Aug 22, 2017

Owner

Mostly I wanted to overhaul the storage model for containers and felt like this was a good start.

string Name = 1;

This comment has been minimized.

@mlaventure

mlaventure Aug 17, 2017

wouldn't labels be enough?

map<string,string> labels = 2;
}

message Container {
string id = 1;
ContainerSpec spec = 2;
Meta meta = 3;
}
@@ -0,0 +1,13 @@
syntax = "proto3";

package moby.core.types.v1;

import "google/protobuf/any.proto";

message FileInfo {
string name = 1;
int64 size = 2;
uint32 mode = 3;
bool is_dir = 4;

This comment has been minimized.

@stevvooe

stevvooe Aug 21, 2017

Can mode be used here?

This comment has been minimized.

@cpuguy83
google.protobuf.Any sys = 5;
}

2 comments on commit 4e7df4a

@stevvooe

This comment has been minimized.

Copy link

stevvooe replied Aug 21, 2017

@cpuguy83 What provisions do we have to let these "evolve" a little bit? I think this is a GREAT start, but there are bunch of little suggestions that we need to make, but shouldn't block progress. Both swarmkit and containerd benefit from the having some amount of time to breath on the APIs.

@cpuguy83

This comment has been minimized.

Copy link
Owner

cpuguy83 replied Aug 22, 2017

Indeed we need to make sure these are not considered stable until they've had enough time.
I can add a status section to make that clear.

Please sign in to comment.