Skip to content

Sync #243

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
FabianKramm merged 2 commits into from
Sep 24, 2018
Merged

Sync #243

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
670fd1c
Improve sync unit test
FabianKramm Sep 24, 2018
b697412
Improve config.yaml documentation
FabianKramm Sep 24, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
150 changes: 110 additions & 40 deletions docs/docs/configuration/config.yaml.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,35 +4,57 @@ title: /.devspace/config.yaml

This is an example of a [.devspace/config.yaml](#)
```yaml
# Devspace version, currently is always v1
version: v1
devSpace:
release:
# Name of helm release that is used for deploying
# the devspace chart (contents of /chart)
name: my-project
# Release namespace
namespace: my-namespace
# Automatically forwarded ports on `devspace up` (same functionality as running manually kubectl port-forward)
portForwarding:
# Currently only pod is supported
- resourceType: pod
# Map of key value matchLabel selectors
labelSelector:
release: my-app
# Array of port mappings
portMappings:
# The local machine port
- localPort: 3000
# The selected pod port
remotePort: 3000
- localPort: 8080
remotePort: 80
sync:
# Currently only resource type pod is supported
- resourceType: pod
labelSelector:
release: my-app
# Sync the complete local project path
localSubPath: ./
# Into the remote container path /app
containerPath: /app
# Exclude node_modules from up and download
excludePaths:
- node_modules/
# A list of images that should be build during devspace up
images:
default:
# Image name
name: devspace-user/devspace
# Image tag (auto-generated)
tag: 9u5ye0G
# Registry where the build image will be pushed to
registry: default
build:
engine:
docker:
# Use docker for image building
enabled: true
# Use the minikube docker daemon, if the current kubectl context is minikube
preferMinikube: true
database:
name: devspace-user/devspace
Expand All @@ -41,101 +63,149 @@ images:
build:
engine:
kaniko:
# Use kaniko within the target cluster to build the image
# instead of local or minikube docker
enabled: true
# The registries the images should be pushed to
registries:
default:
url: hub.docker.com
# Internal registry that will be automatically deployed to the target
# cluster if desired
internal:
# Auto-generated user and password
user:
username: user-XXXXX
password: XXXXXXXXXX
# Services that are used within the target cluster
services:
# The deployed internal registry within the cluster
internalRegistry:
release:
# The helm release name of the internal registry
name: devspace-registry
namespace: my-namespace
# Tiller server that should be used within the cluster
tiller:
# A list of namespaces where tiller is allowed to deploy namespaces to
appNamespaces:
- my-namespace
release:
# Namespace where the tiller server is located
namespace: my-namespace
# Target cluster configuration
cluster:
# Use the local kubectl client config
useKubeConfig: true
```
A [.devspace/config.yaml](#) contains any public/shared configuration for running a DevSpace for the respective project. It is highly recommended to put this file under version control (e.g. git add).

**Note: You can easily re-configure your DevSpace by running `devspace init -r`.**

## devspace
Defines your DevSpace including everything related to portForwarding, sync, and the release config.
Defines the DevSpace including everything related to portForwarding, sync, and the helm release config.

### devspace.release
Defines how the DevSpace is deployed to your cluster. See [Type: Release](#type-release) for details.

### devspace.portForwarding
To access applications running inside a DevSpace, the DevSpace CLI allows to configure port forwardings. A port forwarding consists of the following:
- `resourceType` (currently only `pod` is supported)
- `labelSelector` (usually the release/app name)
- a list of `portMappings` (each specifying a `localPort` on localhost and a `remotePort` within the DevSpace)
- `resourceType` _string_ kubernetes resource type that is selected (currently only `pod` is supported)
- `labelSelector` _map[string]string_ usually the release/app name
- `portMappings` _PortMapping array_

### devspace.portForwarding[*].portMappings[*]
PortMapping:
- `localPort` _string_ on localhost
- `remotePort` _string_ remote pod port

In the example above, you could open `localhost:8080` inside your browser to see the output of the application listening on port 80 within your DevSpace.

### devspace.sync
To comfortably sync code to a DevSpace, the DevSpace CLI allows to configure real-time code synchronizations. A sync config consists of the following:
- `resourceType` (currently only `pod` is supported)
- `labelSelector` (usually the release/app name)
- `localSubPath` (relative to your local project root)
- `containerPath` (absolute path within your DevSpace)
- `excludePaths` (for excluding files/folders from sync in .gitignore syntax)
- `DownloadExcludePaths` (for excluding files/folders from download in .gitignore syntax)
- `UploadExcludePaths` (for excluding files/folders from upload in .gitignore syntax)
- `resourceType` _string_ kubernetes resource type that is selected (currently only `pod` is supported)
- `labelSelector` _map[string]string_ usually the release/app name
- `localSubPath` _string_ relative path to the folder that should be synced (default: path to your local project root)
- `containerPath` _string_ absolute path within the container
- `excludePaths` _string array_ paths to exclude files/folders from sync in .gitignore syntax
- `downloadExcludePaths` _string array_ paths to exclude files/folders from download in .gitignore syntax
- `uploadExcludePaths` _string array_ paths to exclude files/folders from upload in .gitignore syntax

In the example above, the entire code within the project would be synchronized with the folder `/app` inside the DevSpace.
In the example above, the entire code within the project would be synchronized with the folder `/app` inside the DevSpace, with the exception of the `node_modules/` folder.

## images
This section of the config defines a map of images that can be used in the helm chart that is deployed during `devspace up`. An image is defined by:
- `name` of the image that is being pushed to the registry
- `tag` stating the latest tag pushed to the registry
- `registry` referencing one of the keys defined in the `registries` map
- `build` defining the build procedure for this image

## images[*].build
An image build is mainly defined by the build engine. There are 2 build engines currently supported:
- `docker` uses the local Docker daemon or a Docker daemon running inside a Minikube cluster (if `preferMinikube` == true)
- `kaniko` builds images in userspace within a build pod running inside the Kubernetes cluster
This section of the config defines a map of images that can be used in the helm chart that is deployed during `devspace up`.

### images[*]
An image is defined by:
- `name` _string_ of the image that is being pushed to the registry
- `tag` _string_ stating the latest tag pushed to the registry (auto-generated)
- `registry` _string_ referencing one of the keys defined in the `registries` map
- `build` _BuildConfig_ defines the build procedure for this image

### images[*].build
BuildConfig:
- `engine` _Engine_ The engine that should be used for building the image

### images[*].build.engine
Engine:
An image build is mainly defined by the build engine. There are 2 build engines currently supported (choose only one):
- `docker` _DockerConfig_ uses the local Docker daemon or a Docker daemon running inside a Minikube cluster (if `preferMinikube` == true)
- `kaniko` _KanikoConfig_ builds images in userspace within a build pod running inside the Kubernetes cluster

### images[*].build.engine.docker
DockerConfig:
- `enabled` _bool_ if true the local docker daemon is used for image building
- `preferMinikube` _bool_ if true and the current kubectl context is minikube, the minikube docker daemon is used for image building

### images[*].build.engine.kaniko
KanikoConfig:
- `enabled` _bool_ if true a kaniko build pod is used for image building
- `namespace` _string_ specifies the namespace where the build pod should be started

## registries
This section of the config defines a map of image registries. You can use any external registry or link to the [services.internalRegistry](#services-internal-registry)
- `url` of the registry (format: myregistry.com:port)
- `user` credentials (`username`, `password`) for pushing to / pulling from the registry
- `insecure` flag to allow pushing to registries without HTTPS

### registries[*]
ImageRegistry:
- `url` _string_ of the registry (format: myregistry.com:port)
- `insecure` _bool_ flag to allow pushing to registries without HTTPS
- `user` _RegistryUser_ credentials for pushing to / pulling from the registry

### registries[*].user
RegistryUser:
- `username` _string_ that should be used for pushing and pulling from the registry
- `password` _string_ that should be used for pushing and pulling from the registry

## services
Defines additional services for your DevSpace.
Defines cluster services that the DevSpace uses.

### services.internalRegistry
The `internalRegistry` is used to tell the DevSpace CLI to deploy a private registry inside the Kubernetes cluster:
- `release` for deploying the registry (see [Type: Release](#type-release))
- `release` _Release_ for deploying the registry (see [Type: Release](#type-release))

### services.tiller
The `tiller` service is defined by:
- `release` definition for tiller (see [Type: Release](#type-release))
- `appNamespaces` defining a list of namespace that tiller may deploy applications to
- `release` _Release_ definition for tiller (see [Type: Release](#type-release))
- `appNamespaces` _string array_ defines a list of namespace that tiller may deploy applications to

## cluster
The `cluster` field specifies:
- `useKubeConfig` (yes to use the credentials defined in $HOME/.kube/config)
- `useKubeConfig` _bool_ if true use the credentials defined in $HOME/.kube/config

If `useKubeConfig` is `false`, the following fields need to be specified:
- `apiServer` _string_ (Kubernetes API-Server URL)
- `caCert` _string_ (CaCert for the Kubernetes API-Server in PEM format)
- `user` _ClusterUser_

If `useKubeConfig: false` is used, the following fields need to be specified:
- `apiServer` (Kubernetes API-Server URL)
- `caCert` (CaCert for the Kubernetes API-Server in PEM format)
- `user` specifying the following:
- `username`
- `clientCert` (PEM format)
- `clientKey` (PEM format)
### cluster.user
ClusterUser:
- `username` _string_
- `clientCert` _string_ (PEM format)
- `clientKey` _string_ (PEM format)

## Type: Release
A `release` is specified through:
- `name` of the release
- `namespace` to deploy the release to
- `values` that are set during the deployment (contents of the values.yaml in helm)
- `name` _string_ of the release
- `namespace` _string_ to deploy the release to
- `values` _map[string] any_ that are set during the deployment (contents of the values.yaml in helm)
27 changes: 5 additions & 22 deletions pkg/devspace/sync/sync_config_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,8 +63,8 @@ func createTestSyncClient(testLocalPath, testRemotePath string) *SyncConfig {
}

func TestInitialSync(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("Skipping test on windows")
if runtime.GOOS != "linux" {
t.Skip("Skipping test on non linux platform")
}

remote, local, outside := initTestDirs(t)
Expand Down Expand Up @@ -116,13 +116,12 @@ func TestInitialSync(t *testing.T) {
return
}

//This seems to hang in travis
checkFilesAndFolders(t, filesToCheck, foldersToCheck, local, remote, 10*time.Second)
}

func TestNormalSync(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("Skipping test on windows")
if runtime.GOOS != "linux" {
t.Skip("Skipping test on non linux systems")
}

remote, local, outside := initTestDirs(t)
Expand Down Expand Up @@ -187,16 +186,12 @@ func TestNormalSync(t *testing.T) {
t.Error(err)
return
}
checkFilesAndFolders(t, filesToCheck, foldersToCheck, local, remote, 10*time.Second)

checkFilesAndFolders(t, filesToCheck, foldersToCheck, local, remote, 20*time.Second)
}

func setExcludePaths(syncClient *SyncConfig, testCases testCaseList) {

syncClient.ExcludePaths = []string{}

syncClient.DownloadExcludePaths = []string{}

syncClient.UploadExcludePaths = []string{}

for _, testCase := range testCases {
Expand Down Expand Up @@ -298,7 +293,6 @@ func makeBasicTestCases() (testCaseList, testCaseList) {
}

func makeRemoveAndRenameTestCases(filesToCheck testCaseList, foldersToCheck testCaseList) (testCaseList, testCaseList) {

for n, array := range [2]testCaseList{filesToCheck, foldersToCheck} {
for _, f := range array {

Expand Down Expand Up @@ -406,13 +400,10 @@ func makeRemoveAndRenameTestCases(filesToCheck testCaseList, foldersToCheck test
foldersToCheck = append(foldersToCheck, renameFolderFromOutside...)

return filesToCheck, foldersToCheck

}

func makeRemoteTestCases(testCases testCaseList) testCaseList {

for _, f := range testCases {

if strings.Contains(f.path, "Upload") {
f.path = strings.Replace(f.path, "Upload", "Download", -1)
} else if strings.Contains(f.path, "Download") {
Expand All @@ -432,9 +423,7 @@ func makeRemoteTestCases(testCases testCaseList) testCaseList {
}

func makeDeepTestCases(testCases testCaseList) testCaseList {

for _, f := range testCases {

if f.path == "testFolder" {
continue
}
Expand All @@ -452,7 +441,6 @@ func makeDeepTestCases(testCases testCaseList) testCaseList {
}

func createTestFilesAndFolders(local string, remote string, outside string, filesToCheck testCaseList, foldersToCheck testCaseList) error {

for _, f := range foldersToCheck {
parentDir, err := getParentDir(local, remote, outside, f.editLocation)
if err != nil {
Expand Down Expand Up @@ -482,7 +470,6 @@ func createTestFilesAndFolders(local string, remote string, outside string, file
}

func removeSomeTestFilesAndFolders(local string, remote string, filesToCheck testCaseList, foldersToCheck testCaseList, removeSuffix string) (testCaseList, testCaseList, error) {

var completeSuffix string
removeIfSuffixMatch := func(path string, f os.FileInfo, err error) error {
if strings.HasSuffix(path, completeSuffix) {
Expand Down Expand Up @@ -520,11 +507,8 @@ func removeSomeTestFilesAndFolders(local string, remote string, filesToCheck tes
}

func renameSomeTestFilesAndFolders(local string, remote string, outside string, filesToCheck testCaseList, foldersToCheck testCaseList) (testCaseList, testCaseList, error) {

for n, array := range [2]testCaseList{filesToCheck, foldersToCheck} {

for n, f := range array {

if !strings.Contains(f.path, "_Rename") {
continue
}
Expand Down Expand Up @@ -572,7 +556,6 @@ func renameSomeTestFilesAndFolders(local string, remote string, outside string,
}

array[n] = f

}

if n == 0 {
Expand Down
Loading