Skip to content

Commit

Permalink
Change Golang to Go (#356)
Browse files Browse the repository at this point in the history
  • Loading branch information
mmitoraj committed Sep 17, 2019
1 parent 5c13f08 commit c397eb9
Show file tree
Hide file tree
Showing 6 changed files with 51 additions and 46 deletions.
9 changes: 5 additions & 4 deletions collaboration/sig-core/decisions/dr-001-Technologies.md
Expand Up @@ -5,19 +5,20 @@ Created on 2017-11-20 by Łukasz Górnicki (@derberg).
## Context

The Kyma developers need to select specific technologies. Their purpose is to:

* Enable the most lightweight solution and cost-effective solution for the cluster installation.
* Facilitate the cross-teams collaboration development.
* Enable easy context-switching between teams.

## Decision

The decision is to use the **Golang** language for all new implementations in Kyma. Golang allows to create very efficient applications with low memory usage and a vast set of system libraries. Many projects which Kyma depends on are written in Golang, including Kubernetes.
The decision is to use the **Go** language for all new implementations in Kyma. Go allows to create very efficient applications with low memory usage and a vast set of system libraries. Many projects which Kyma depends on are written in Go, including Kubernetes.

Use the following front-end technologies within Kyma:

- Open UI5
- Angular (version 4 and later)
- React
* Open UI5
* Angular (version 4 and later)
* React

The recommended technologies fulfill the Kyma principles.

Expand Down
Expand Up @@ -8,13 +8,15 @@ Although Kubernetes manages service accounts, the solution leaves the management
For this reason, Kyma needs an authentication provider for end-users.

Kubernetes has the following requirements for authentication providers:

- The authentication provider must support the [OpenID connect discovery](https://openid.net/specs/openid-connect-discovery-1_0.html).
- The authentication provider must run in TLS with non-obsolete ciphers.
- The authentication provider must support CA-signed certificates, even for commercial and self-signed CAs.

The solution must also fulfill the general platform requirements:

- It must be a cloud-native solution.
- It must be lightweight, preferably written in GoLang.
- It must be lightweight, preferably written in Go.
- It must be open-source, with the ability to contribute to the project development.

Additionally, the solution must allow to delegate authentication to SAML2-compliant Identity Providers.
Expand All @@ -29,7 +31,7 @@ Accepted on 2018-03-01.

## Consequences

Dex is a lightweight application written in GoLang.
Dex is a lightweight application written in Go.

Using connectors, Dex supports both the OIDC- and SAML2-compliant Identity Providers. Dex does not have its own user database, and therefore requires at least one connector to work in the production
environment. The solution supports static users but, as it is not a completely secure method of user management, it should be used for test purposes only.
Expand Down
28 changes: 11 additions & 17 deletions collaboration/sig-core/proposals/central-connector-service.md
Expand Up @@ -13,8 +13,6 @@ Currently, the connection between an external solution and Kyma is always point-

As the customers work with multiple Kyma clusters, they will benefit from extending the provisioning of Kyma client certificates. A central Connector Service would manage the provisioning of certificates for multiple Kyma clusters and connected clients. Such approach allows the users to control their entire Kyma ecosystem from a single, central point.



## Goal

1. The Connector service can be deployed outside of Kyma cluster
Expand All @@ -25,19 +23,17 @@ As the customers work with multiple Kyma clusters, they will benefit from extend
1. The Connector service handles certificate revocation.
1. The Connector service returns information about the available cluster endpoints.


## Suggested solution

The Connector Service (CS) is deployed as a central component.

- The CS is deployed as a independent component inside Kyma cluster or Kubernetes.
- The CS exposes a secured connection for requesting client certificates signed with root CA.
- The CS exposes a secured connection for requesting server certificates signed with root CA and deployed to Kyma cluster.
- The client certificate enables a trusted connection with the central Kyma cluster where the App Registry is stored.
- The client certificate enables a trusted connection with the Kyma cluster where the server certificate is delivered.
- The server certificate enables a trusted connection with central Kyma cluster.
- For standalone Kyma clusters, the Connector Service is deployed locally and works in the same manner.

- The CS is deployed as a independent component inside Kyma cluster or Kubernetes.
- The CS exposes a secured connection for requesting client certificates signed with root CA.
- The CS exposes a secured connection for requesting server certificates signed with root CA and deployed to Kyma cluster.
- The client certificate enables a trusted connection with the central Kyma cluster where the App Registry is stored.
- The client certificate enables a trusted connection with the Kyma cluster where the server certificate is delivered.
- The server certificate enables a trusted connection with central Kyma cluster.
- For standalone Kyma clusters, the Connector Service is deployed locally and works in the same manner.

### Component Diagram

Expand Down Expand Up @@ -77,7 +73,6 @@ The assumption is that certificates are short-living - 24 hours.

The endpoint for renewal is protected with a certificate. If a certificate expires then the whole pairing process need to be started again.


### Certificate revocation

The client certificates and Kyma cluster client certificates must be revoked as soon as they are compromised. Due to the fact that certificates are short-living that is enough to block the certificate renewal.
Expand All @@ -97,10 +92,9 @@ The information about real endpoints will be passed to the Connector service in
1. The application certificates and Kyma cluster certificates will be renewed frequently.
1. The Root CA certificate, used for signing client certificates, will be rotated in a special way.


## Proof of Concept

### Prerequisites:
### Prerequisites

- Private key (`rootCA.key`) and certificate (`rootCA.crt`) generated as a root CA
- Kyma cluster provisioned with `rootCA.key` and `rootCA.crt` as a CA
Expand All @@ -123,7 +117,7 @@ The information about real endpoints will be passed to the Connector service in
openssl x509 -req -sha256 -in client.csr -out client.crt -CAkey rootCA.key -CA rootCA.crt -days 1800 -CAcreateserial -CAserial serial
```

4. Create the certificate chain containing `rootCA.crt` and `client.crt`.
4. Create the certificate chain containing `rootCA.crt` and `client.crt`.

```
cat rootCA.crt client.crt > client-chain.crt
Expand Down Expand Up @@ -223,7 +217,7 @@ The following assumptions were taken:
- Application ID is not needed in paths.
- Resource names can be changed to ensure consistent naming.
- Payloads returned in certificate generation flow cannot change.
- Application ID may be obtained from the certificate used for accessing the endpoints, however, it needs to be confirmed if it is technically feaseable with Nginx Ingress and Golang.
- Application ID may be obtained from the certificate used for accessing the endpoints, however, it needs to be confirmed if it is technically feaseable with Nginx Ingress and Go.

The connector service exposes the following groups of endpoits:

Expand Down Expand Up @@ -288,4 +282,4 @@ The following API has been defined:

### Returning API spec from the service

The user should be able to get specification from the service. There should be `v1/api.yaml` endpoint for this purpose.
The user should be able to get specification from the service. There should be `v1/api.yaml` endpoint for this purpose.
32 changes: 18 additions & 14 deletions collaboration/sig-core/proposals/common-libraries.md
Expand Up @@ -10,33 +10,35 @@ Proposed on 2018-10-17.

The Kyma project was consolidated and now all core projects written in Go are placed in one repository. Unfortunately, all of them are still written in a different way.

This situation causes the following problems:
This situation causes the following problems:

- Often the common package is copied across different projects which means that we break the DRY rule.
- When we do not follow the DRY rule, we must fix the detected bugs in all places. It makes it more complex to fix bugs and also makes it easy to forget about some projects.
- If a developer is not aware that a given functionality is already implemented and can be reused, the approach for the same business logic (such as logging pattern, metrics, graceful shutdown, testing, or configuration) is different.
- Because projects often have different approaches to the same problem, it's difficult for internal and external contributors to get acquainted with the project.
- When a given functionality is reimplemented several times by different developers, the testing approach is often different. It means that sometimes the functionality is well tested but when it's not, it might not work as expected. The good example is a **wait** functionality. We reimplemented a well-known pattern [here](https://github.com/kyma-project/kyma/blob/09b945ec0a897822b40666e518f2f258ae66e7e8/tests/acceptance/servicecatalog/wait/wait.go#L8-L24) but with the small bug which caused that the function never reached a given timeout. There are a lot of reimplementations of this pattern in new PRs, for example [here](https://github.com/kyma-project/kyma/blob/ea5979bf2d82740e84d119df857f9e34541e070e/tests/backup-restore-e2e/utils/wait_util.go#L21-L42) (also not tested).
- Because projects often have different approaches to the same problem, it's difficult for internal and external contributors to get acquainted with the project.
- When a given functionality is reimplemented several times by different developers, the testing approach is often different. It means that sometimes the functionality is well tested but when it's not, it might not work as expected. The good example is a **wait** functionality. We reimplemented a well-known pattern [here](https://github.com/kyma-project/kyma/blob/09b945ec0a897822b40666e518f2f258ae66e7e8/tests/acceptance/servicecatalog/wait/wait.go#L8-L24) but with the small bug which caused that the function never reached a given timeout. There are a lot of reimplementations of this pattern in new PRs, for example [here](https://github.com/kyma-project/kyma/blob/ea5979bf2d82740e84d119df857f9e34541e070e/tests/backup-restore-e2e/utils/wait_util.go#L21-L42) (also not tested).

## Solution

Extract common code from the Kyma components to the common package and use it in all required places. This allows us to use unified and well-tested libraries in our projects, which will be easier to understand and make future development faster.

### Scope

There is a question whether to use the general Go SDK or tight-coupled SDK for the Kyma project. This is the first phase, so the idea is to have an internal library used only by the Kyma developers.
There is a question whether to use the general Go SDK or tight-coupled SDK for the Kyma project. This is the first phase, so the idea is to have an internal library used only by the Kyma developers.
Those libraries should not be treated as public and we should not focus on having the generic libraries there. Instead, we should place there all things which are helpful for Kyma developers even if they are tightly coupled with the Kyma domain.

In the future, we should execute the next phase and review our libraries. We should also check if we are ready to extract common libraries outside and announce them as the official SDK for all Kyma internal and external developers.
In the future, we should execute the next phase and review our libraries. We should also check if we are ready to extract common libraries outside and announce them as the official SDK for all Kyma internal and external developers.

### Package name
### Package name

In Go ecosystem, there is a pretty popular `pkg` layout pattern. Many well-known projects, such as [Kubernetes](https://github.com/kubernetes/kubernetes/tree/master/pkg), [Docker](https://github.com/moby/moby/tree/master/pkg), [Grafana](https://github.com/grafana/grafana/tree/master/pkg), [Influxdb](https://github.com/influxdata/influxdb/tree/master/pkg), [Etcd](https://github.com/etcd-io/etcd/tree/master/pkg), use this pattern.
However, the `pkg` directory is used mainly for public libraries. They can be used internally by your application but also by external projects.
This is an informal contract between you and the external users of your code.
In Go ecosystem, there is a pretty popular `pkg` layout pattern. Many well-known projects, such as [Kubernetes](https://github.com/kubernetes/kubernetes/tree/master/pkg), [Docker](https://github.com/moby/moby/tree/master/pkg), [Grafana](https://github.com/grafana/grafana/tree/master/pkg), [Influxdb](https://github.com/influxdata/influxdb/tree/master/pkg), [Etcd](https://github.com/etcd-io/etcd/tree/master/pkg), use this pattern.
However, the `pkg` directory is used mainly for public libraries. They can be used internally by your application but also by external projects.
This is an informal contract between you and the external users of your code.

In Go, there is a possibility to hide code using internal directories. If you put your code in an `internal` directory, no external project will be able to import that code.
In Go, there is a possibility to hide code using internal directories. If you put your code in an `internal` directory, no external project will be able to import that code.
Initially, the idea was to put our private libraries in `kyma/internal/pkg/{private_libs}` but we consolidated all the project to one repository, so there is no option to vendor such libraries in our project. In this case, when we execute the `dep ensure` command, we receive such error:
```

```text
use of internal package kyma/internal/pkg/http not allowed
```

Expand Down Expand Up @@ -100,6 +102,7 @@ The new proposition is to name the package for libraries as `common` and place
### Examples

These are the example libraries that we can put in the package library:

- wait - a package where we place utils used for waiting for some condition. Used almost in all our acceptance and integration tests.
- http - a package where we put the default handlers required for healthz; required in all applications executed in Kuberenetes
- signals - a package where we put signal related implementations, such as graceful shutdown; required in all applications
Expand All @@ -110,19 +113,20 @@ These are the example libraries that we can put in the package library:

### Restrictions

Libraries must be well-tested. Tests must be executed with the `-count 100` flag to ensure stability and with the `-race` flag to detect all race conditions.
Libraries must be well-tested. Tests must be executed with the `-count 100` flag to ensure stability and with the `-race` flag to detect all race conditions.
Each library must contain good documentation. It is also highly recommended that each library has the [Example Test](https://blog.golang.org/examples) similar to this [one](https://github.com/mszostok/kyma/blob/libs/common/http/client_example_test.go#L13).

### Next step

The next step is to create a proposal about the best practices. For example, we should get rid of the [`Sleep` functions](https://github.com/kyma-project/kyma/blob/09b945ec0a897822b40666e518f2f258ae66e7e8/tests/acceptance/servicecatalog/wait/wait.go#L8-L24) from our test cases and require a better approach.
The next step is to create a proposal about the best practices. For example, we should get rid of the [`Sleep` functions](https://github.com/kyma-project/kyma/blob/09b945ec0a897822b40666e518f2f258ae66e7e8/tests/acceptance/servicecatalog/wait/wait.go#L8-L24) from our test cases and require a better approach.
But first, we need a place where we can refer to the proper code which can be used out of the box. In such case, the common libraries can be used.

Another example is to unify the logging approach in our projects. This will be also resolved by the other proposal in which the source code will be placed in the common libraries package, so all projects can simply use it.

## Consequences

Once approved, these are the consequences:

- new package for common libraries must be created and maintained
- new expert CODEOWNERS must be defined for reviewing libraries added for common package
- new package must be announced and other projects should switch to use the official libraries instead of custom implementation
Expand Down
2 changes: 1 addition & 1 deletion collaboration/sig-core/proposals/common-validator.md
Expand Up @@ -220,7 +220,7 @@ Thanks to `template.go.mk`, we will have one place in a repository with target d
### Sandbox validation

Developers often work in a different environment than the one used by the CI system. For example, they use different versions of Golang. It is important to provide a way of verifying the code against the CI environment. This can be enabled by creating targets with the `-sandbox` suffix that will execute targets in the same `buildpack` Docker image as the one the CI system uses.
Developers often work in a different environment than the one used by the CI system. For example, they use different versions of Go. It is important to provide a way of verifying the code against the CI environment. This can be enabled by creating targets with the `-sandbox` suffix that will execute targets in the same `buildpack` Docker image as the one the CI system uses.

To achieve that, we need to update `template.go.mk` and the makefile for a given component.

Expand Down
20 changes: 12 additions & 8 deletions guidelines/technical-guidelines/04-docker-images.md
Expand Up @@ -9,20 +9,21 @@ This document provides guidelines for the Docker image provided in the context o
Place images in the Kyma Docker registry located at `eu.gcr.io/kyma-project`. For development and proof of concepts, use the following location: `eu.gcr.io/kyma-project/snapshot`.

All images use the following attributes:

- an image name which is the same as the related project. Do not use prefixes. If the image requires sub-modularization, append it as in "istio-mixer"
- a tag with a semantic version number, like `0.3.2`

Assume an initializer image for the Helm Broker extension. This is the example of the location and the name of the image:

```
```bash
eu.gcr.io/kyma-project/helm-broker-initializer:0.1.0
```

## Base images

Base all images on an image that is as small as possible in size and dependency. A base image must have a specified version. Do not use the `latest` tag.

An application based on Golang should originate from a `scratch` image. If a `scratch` image does not have the specific tooling available, you can use an `alpine` base image having the package catalog updated.
An application based on Go should originate from a `scratch` image. If a `scratch` image does not have the specific tooling available, you can use an `alpine` base image having the package catalog updated.
A JavaScript-based application should originate from an `nginx-alpine` base image with an updated package catalog.

## Label images
Expand All @@ -31,23 +32,25 @@ All images use the `source` label with a link to the GitHub repository containin

Define labels as in the following example:

```
```bash
source = git@github.com:kyma-project/examples.git
```

## Examples

Golang from scratch:
```
Go from scratch:

```Dockerfile
FROM scratch
LABEL source=git@github.com:kyma-project/examples.git

ADD main /
CMD ["/main"]
```

Golang from alpine:
```
Go from alpine:

```Dockerfile
FROM alpine:3.7
RUN apk --no-cache upgrade && apk --no-cache add curl

Expand All @@ -58,7 +61,8 @@ CMD ["/main"]
```

JavaScript from nginx:
```

```Dockerfile
FROM nginx:1.13-alpine
RUN apk --no-cache upgrade

Expand Down

0 comments on commit c397eb9

Please sign in to comment.