Application Dependencies

[kow3ns]

The original KEP proposed the following notion of application Dependencies

// ApplicationSpec defines the specification for an Application.
type ApplicationSpec struct {
	// Existing fields omitted for brevity
        // Dependencies is a list of Applications on which this Application depends.
        Dependencies [] string
}

// ApplicationStatus defines controllers the observed state of Application
type ApplicationStatus struct {
	// ObservedGeneration is used by the Application Controller to report the last Generation of an Application
	// that it has observed.
	ObservedGeneration int64 `json:"observedGeneration,omitempty"`

       // Installed is a list of the currently installed components and dependencies for an Application as 
       // observed by the controller.
       Installed []string `json:"installed,omitempty"`
}

// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// Application
// +k8s:openapi-gen=true
// +resource:path=applications
// The Application object acts as an aggregator for components that comprise an Application. Its
// Spec.ComponentGroupKinds indicate the GroupKinds of the components the comprise the Application. Its Spec. Selector
// is used to list and watch those components. All components of an Application should be labeled such the Application's
// Spec. Selector matches.
type Application struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`
        // The specification object for the Application.
	Spec   ApplicationSpec   `json:"spec,omitempty"`
	// The status object for the Application.
	Status ApplicationStatus `json:"status,omitempty"`
}

The notion of Dependencies is necessarily namespace bound by this definition. The application controller would simply track the components of the installed application through their life-cycle and update the Status.Installed list when a dependent component is created.

1. Is being bound to the same namespace too restrictive?
2. Is this notion of dependencies (admittedly a weak one) useful. That is, should dependency manage be in the purview of a package manager and not the installed application itself. For instance, in debian based linux distro's apt and dpkg manage the dependencies of an application and ensure that shared libraries, as an example, that an application depends on are installed along with the application. The application simply expects its dependencies to be available at runtime.

[tamalsaha]

We have working with a Dependency definition that look like

type Dependency struct {
	metav1.TypeMeta `json:",inline" yaml:",inline"`
	// Standard object's metadata.
	// More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#metadata
	metav1.ObjectMeta `json:"metadata,omitempty" yaml:"metadata,omitempty"`

	Package string `json:"package"`
	Version string `json:"version,omitempty"`
	Branch  string `json:"branch,omitempty"`
	Folder  string `json:"folder,omitempty"`
	Repo    string `json:"repo,omitempty"`
	Fork    string `json:"fork,omitempty"`
}

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// DependencyList is a list of Dependency objects.
type DependencyList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`

	Items []Dependency `json:"items"`
}

https://github.com/kubepack/pack-server/blob/master/apis/manifest/v1alpha1/types.go#L22

Our view is that there are 2 YAML needed for a package: kube-manifest.yaml (required) and dependency-list.yaml (optional) . Each entry in DependencyList points to a package that has its own kube-manifest.yaml. By kube-manifest.yaml, I am referring to the kinflate manifest.

[mattfarina]

@tamalsaha There may be two separate things going on here.

First, we have the details needed to describe applications as they run within Kubernetes. This is an operational issue.

Second, we have details needed for packages (as in a package manager). They are not necessarily operational details.

Can you describe the use case you're targeting with this?

[mattfarina]

@kow3ns I'm unable to open the original PR for the KEP so I can't get to any details there (GitHub keeps giving an error). What's the use case for the dependency tracking outlined in the proposal?

[mattfarina]

@kow3ns Also, how would services (e.g., those from the service catalog) relate in terms or dependencies?

[hzxuzhonghu]

Is being bound to the same namespace too restrictive?

I think yes, how can we describe dependents in other namespaces/or root scoped?

Is this notion of dependencies (admittedly a weak one) useful?

Yes, in cases we depend on shared load balancer or dbs or even other middlewares.

[liyinan926]

Just want to share some of my thoughts on application dependencies here. I think it's super useful to have the notion of application dependencies and some ways of checking and enforcing the dependencies.

As a concrete example, ZooKeeper is used as a dependent system by several popular open source projects, including Kafka (at least for now, not sure about the future though as they do have a plan to get rid of the dependency on ZK), HDFS (for high availability mode), HBase, etc. Let's assume that a user runs all the mentioned applications and needs to have an Application instance for each of the applications created. Creation of the Application instances may be done manually using some manifest or by the operators for the applications at runtime. Regardless how the application instances get created, there are dependencies among the applications, i.e., the Kafka, HDFS, and HBase instances all depend on some ZooKeeper instance, and none of those will work if there's not a ZooKeeper instance available in the cluster.

Things get more interesting as dependencies involve application versions. For example, a particular Kafka version may only work with certain versions of ZooKeeper, which could be a range of minor versions or some major version. In this case, a Kafka Application is not considered successfully assembled if there's not a ZooKeeper instance with one of the compatible versions. Dependencies may or may not involve sharing. For example, all the Kafka, HDFS, and HBase instances in the cluster may share a single ZooKeeper instance, or have dedicated ZooKeeper instances for example for security or performance reasons. It would be super useful for users to be able to specify such dependency conditions.

[jzelinskie]

cc @ecordell

We shipped dependencies in the Operator Lifecycle Manager in December. Our implementation is decoupled as much as possible into a separate optional component that we call catalog, but even doing so has implicated a lot of behavior on our ClusterServiceVersion-v1, which is the resource closest resembling the Application definition. I'm not sure if it's possible to store this information as a top-level functionality without being also opinionated about package management as well.

[mattfarina]

@jzelinskie Did your dependency tracking system allow for cross namespace dependencies?

[jzelinskie]

@mattfarina OLM targets the single namespace use-case. We'd like to work with the Multi-tenancy Working Group in order to have a general solution in Kubernetes.

[ant31]

I have an application composed with:

1. backend
2. frontend
3. database

I create 3 Application Instances to be able to group their resources and manage each component individually.
I also want to have an overview of the health of a whole applications stack, so I create an application Instance that matches the other 3 applications instances.

The hierarchy more or less looks like this:

application/myapp
  |-> application/myapp-db
        |-> deployment/myapp-db
        |-> svc/myapp-db
        |-> pvc/myapp-db
  |-> application/myapp-frontend
       |-> deployment/myapp-frontend
       .....
  |-> application/myapp-backend
      .....

kubectl get applications
NAME
myapp
myapp-backend
myapp-frontend
myapp-db

How the statuses are aggregated to 'myapp' ?
How can I build a UI/UX that shows only the top-level application stack and then its sub-microservices?

# Show me only applications-stacks
kubectl get applications --top-level
NAME
myapp

# Show me all the dependencies of an application 
kubectl get applications --dependencies-of=myapp
NAME
myapp-backend
myapp-frontend
myapp-db

cc @cmoulliard

[cmoulliard]

This use case @ant31 is definitely what our customers are developing and where, what we call an application, is defined by a collection of components / services.

Such a project corresponds for the java developers to a maven project hierarchy as defined her after:

myapp // contains a pom.xml file with modules definition
  frontend // maven frontend module to be deployed using deployment, service, configmap, 
  backend // another module containing code of the backend spring boot 
  service // project containing parameters to setup a service using the ServiceCatalog (db, ...)
  ....

This application will be deployed using a myapp Application CRD and should contain a list of Components where each component describes with specific fields and labels what is the description/definition of the component (spring boot runtime, apache tomcat web application, ) or Service (postgresql database, mysql database, artemis messaging broker, ...)

Remark : This is the reason why I proposed within this ticket to use as type a Component to describe what is the module part of the application

If we use this approach, then we should be able (at the condition to update the cli-runtime project used by kubectl) to offer such commands

kubectl get application myapp
-->
NAME    VERSION    STATUS
myapp   1.0        waiting

and

kubectl get components --application myapp
NAME          RUNTIME       VERSION   STATUS
backend       spring-boot   1.15      running
frontend      nodejs        10        running
db            postgresqldb  9.0       deployed

[cmoulliard]

notion of application dependencies

I think that your idea is different from the definition suggested within this kep -> // Dependencies is a list of Applications on which this Application depends.. I like nevertheless your suggestion as it allows an application/component to express something which is needed, should be installed on k8s, is required to operate correctly. I'm referring to a list of features into my project's definition -> https://github.com/snowdrop/component-operator/blob/master/pkg/apis/component/v1alpha1/component_types.go#L55-L58

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[gyliu513]

What is the plan for this? Any update?

[gyliu513]

/remove-lifecycle rotten

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.