Permalink
Browse files

small refactors, change admin app name (#1069)

* small refactors, change admin app name

* Update gateway.json

* Update codefresh.yml

remove tests parallelism

* Update codefresh.yml

fix?

* Update codefresh.yml

another attempt

* Update codefresh.yml

* Update subject_extraction_rules.rego

* Update test.sh

fix smoketests entrypoint

* Update test.sh

* Update test.sh

attempt to fix smoke tests

* Update test.sh

* Update test.sh

* Delete CorsTests.cs

* Delete SwaggerTests.cs

* fix

* fix

* fix

* fix

* set file syncing for skaffold

* add some documentation

* fix

* more documentation

* add back country

* Update editor.yaml

* Update skaffold.yaml

* Update server.ts
  • Loading branch information...
Yshayy committed Jan 22, 2019
1 parent 3f6d33f commit 0902b4b23e646af4c12576e85239912a735ea113
Showing with 327 additions and 461 deletions.
  1. +104 −106 CI/codefresh.yml
  2. +14 −75 CONTRIBUTING.md
  3. +43 −14 README.md
  4. +1 −1 core/Engine/Tweek.Engine.Tests/Drivers/RulesRepositoryTests.cs
  5. +1 −7 deployments/dev/docker-compose.e2e.yml
  6. +0 −3 deployments/dev/docker-compose.local-editor.yml
  7. +15 −3 deployments/dev/docker-compose.override.yml
  8. +26 −9 deployments/dev/gateway/config/gateway.json
  9. +3 −1 deployments/kubernetes/README.md
  10. +1 −3 deployments/kubernetes/api.yaml
  11. +1 −2 deployments/kubernetes/authoring.yaml
  12. +0 −9 deployments/kubernetes/editor.yaml
  13. +3 −4 deployments/kubernetes/gateway.yaml
  14. +1 −1 deployments/kubernetes/git.yaml
  15. +1 −1 deployments/kubernetes/infra/gateway-config.yaml
  16. +1 −3 deployments/kubernetes/infra/minio.yaml
  17. +5 −5 deployments/kubernetes/infra/oidc-server-mock.yaml
  18. +6 −1 e2e/integration/spec/authoring-api/test-data/subject_extraction_rules.rego
  19. +0 −37 services/api/Tweek.ApiService.SmokeTests/CorsTests.cs
  20. +0 −27 services/api/Tweek.ApiService.SmokeTests/SwaggerTests.cs
  21. +5 −6 services/api/Tweek.ApiService.SmokeTests/test.sh
  22. +1 −0 services/authoring/src/server.ts
  23. +0 −1 services/editor/Dockerfile
  24. +8 −0 services/editor/debug.Dockerfile
  25. +1 −1 services/editor/package.json
  26. +3 −0 services/gateway/main.go
  27. +13 −7 services/gateway/security/authentication.go
  28. +1 −1 services/gateway/security/authorization_test.go
  29. +0 −61 services/gateway/security/testdata/authorization.rego
  30. +3 −0 services/git-service/BareRepository/source/manifests/@tweek/schema/user.json
  31. +25 −45 services/git-service/BareRepository/source/security/policy.json
  32. +6 −1 services/git-service/BareRepository/source/security/subject_extraction_rules.rego
  33. +20 −20 services/git-service/BareRepository/tests-source/security/policy.json
  34. +6 −1 services/git-service/BareRepository/tests-source/security/subject_extraction_rules.rego
  35. +2 −2 services/publishing/Tweek.Publishing.Service/Startup.cs
  36. +2 −2 services/publishing/Tweek.Publishing.Service/Sync/Converters/ExternalAppsConverter.cs
  37. +5 −1 {deployments/kubernetes → }/skaffold.yaml

Large diffs are not rendered by default.

Oops, something went wrong.
@@ -9,8 +9,10 @@
- git-service (stand-alone git rules repository for bootstrap, dev & testing)
- minio (object storage) - rules storage
- redis/couchbase/mongo - context database
- nats - pubsub
- deployments
- dev (docker compose files for devlopment)
- kubernetes - use together with Skaffold
- core
- Tweek calculation lib (.Net)
- addons
@@ -40,87 +42,24 @@
git clone https://github.com/Soluto/tweek.git
cd tweek
```
2. `docker-compose -f ./deployments/dev/docker-compose.yml build`
3. `docker-compose -f ./deployments/dev/docker-compose.yml up -d`
2. Yarn start
3. Go to http://localhost:8080/login and use basic auth with (user: admin-app, password: 8v/iUG0vTH4BtVgkSn3Tng==)

All tweek microservices should be run on ports 4001-4004:
4001 - Git server (ssh)
4003 - Api (http)
4004 - Editor (http)
4005 - Authoring (http)
4009 - Publishing (ssh)
4010 - Publishing (http)
4099 - Gateway (http)
Access Tweek gateway using localhost:8080.
Tweek gateway route all traffic to other resources based on: https://github.com/Soluto/tweek/blob/master/services/gateway/settings/settings.json
The root path redirect to Tweek Editor UI

## Debugging Tweek api
If you use k8s (comes bundled with Docker for mac/pc, enable using UI), you can use Skaffold (https://github.com/GoogleContainerTools/skaffold).
The main benefit of using Skaffold is that it provide watch, build for all Tweek services (editor has also support for hot code reloading).
Since Skaffold/k8s run all services and dependencies together, it can take few minutes to stabilize. (k8s will attempt to restart failed services)

### Install tools

VS CODE

1. Download VS Code: <https://code.visualstudio.com/>
2. Install C# extension: <https://code.visualstudio.com/docs/languages/csharp>

VS 2017

1. Install VS 2017

### RUN

1. If you haven't built the full environment before, pull management service:
```bash
docker-compose -f ./deployments/dev/docker-compose.yml pull tweek-management tweek-git
```
2. If you haven't ran the full environment before, run management service:
```bash
docker-compose -f ./deployments/dev/docker-compose.yml up -d tweek-management
```
3. Debug tweek in VS2017 or VSCODE tweek-api task

### TESTS

#### UNIT

1. Run in Visual studio 2017

or
mac: find . -wholename '*.Tests.csproj' -print0 | xargs -0 -n 1 dotnet test (only with shell)
#### BLACKBOX/SMOKE
1. Run service
2. Run smoke tests in VS or run:
```bash
dotnet test services/api/Tweek.ApiService.SmokeTests/Tweek.ApiService.SmokeTests.csproj -c Release --no-build
```
After installing Skaffold, use ```skaffold dev --port-forward=false```

## Debugging Tweek editor

1. go to services\editor
2. run npm i/yarn
### environment
- if you didn't change anything in the environment, you can just pull it from the server using this command:
```bash
npm run docker-compose pull [SERVICES]
```
- if you made any changes in the environment, build it using this command:
```bash
npm run docker-compose build [SERVICES]
```
- the possible services are: `tweek-git` `tweek-management` `tweek-api
#### Examples
- get latest management and api version from server:
```bash
npm run pull-env tweek-management tweek-api
```
- build local management version:
```bash
npm run build-env tweek-management
```
2. run yarn
3. run yarn start:full-env

### Debug

@@ -129,7 +68,7 @@ mac: find . -wholename '*.Tests.csproj' -print0 | xargs -0 -n 1 dotnet test (onl

### Unit Tests

- run `npm test`
- run `yarn test`

## E2E

@@ -29,15 +29,30 @@ Tweek aims to be a complete open-source alternative to other industry feature/co
### Getting Started
The easiest way to start evaluating Tweek is to run it locally on docker, make sure you have the latest [docker (for windows/mac/etc..)](https://www.docker.com/get-docker) version installed (17-06+).
#### Running Tweek
- clone the repo (``` git clone https://github.com/Soluto/tweek.git ```)
- run (``` yarn start ```) - this might take a few minutes for the first time

- Clone the repo (``` git clone https://github.com/Soluto/tweek.git ```)
- [optional] Pull images, run ```yarn docker-compose pull --parallel``` (optional for getting started fast with Tweek as it's skip build)

### Using Docker Compose
- Run (``` yarn start ```) - this might take a few minutes for the first time

### Using Kubernetes
- Install Skaffold (https://github.com/GoogleContainerTools/skaffold)
- Run ```skaffold dev --port-forward=false```
- Wait for environment to be stable (will take about 10m first time due to building all images, afterward it can take about 2m for environment to stabilize)

### Troubleshooting
- Run (``` yarn start --build```) to rebuild all images and start Tweek.

#### Edit your first key
After setting up our environment, we're going to create our first key.
Keys in tweek are the most basic building blocks and they represent a container for dynamic value that affect feature behaviors.
Our first key, will be a key that is responsible for the color of a "sign up" button.

- Open http://editor.dev.tweek.localtest.me:81 in browser.
- Open http://localhost:8080/login in browser.
- Login
- User Basic auth (user: admin-app, password: 8v/iUG0vTH4BtVgkSn3Tng==)
- Can also use OIDC mock server login button for testing OIDC (user: User, password: pwd)
- Go to keys page.
- Click on "Add Key"
- Type my_app/sign_button/color
@@ -53,31 +68,45 @@ More on [keys and paths](https://docs.tweek.fm/concepts/keys/keys-ands-paths)

#### Querying Tweek
Use curl/postman/chrome to fire GET Request:
- http://api.dev.tweek.localtest.me:81/api/v1/keys/my_app/sign_button/color -> expected to be "red"
- http://api.dev.tweek.localtest.me:81/api/v1/keys/my_app/sign_button/color?user.Country=canada -> expected to be "blue"
- http://api.dev.tweek.localtest.me:81/api/v1/keys/my_app/sign_button/_?user.Country=canada -> expected to be {"color":"blue"}
- http://localhost:8080/api/v2/values/my_app/sign_button/color -> expected to be "red"
- http://localhost:8080/api/v2/values/my_app/sign_button/color?user.Country=canada -> expected to be "blue"
- http://localhost:8080/api/v2/values/my_app/sign_button/_?user.Country=canada -> expected to be {"color":"blue"}
Using the rest api, an application can query Tweek for getting the right set of values for each specific user.
More on Tweek [Rest api](https://docs.tweek.fm/api/rest-api).
#### Adding context data
Tweek provide's REST api for saving context data.
Using the API, use curl/postman to fire POST Request:
- http://api.dev.tweek.localtest.me:81/api/v1/context/user/john {"Country":"Canada"}
Tweek provide UI and rest api for editing context.
- Go to context
- Set Identity Type to User
- Set User id to John
- Click enter
- Set value "Candada" for property Country
After that, we can query Tweek API with:
- http://api.dev.tweek.localtest.me:81/api/v1/keys/my_app/sign_button/color?user=john -> expected to be "blue"
- http://localhost:8080/api/v1/keys/my_app/sign_button/color?user=john -> expected to be "blue"
You can also use the api for updating Tweek context:
- curl -X POST http://localhost:8080/api/v2/context/user/john \
-H 'content-type: application/json' \
-H 'x-client-id: admin-app' \
-H 'x-client-secret: 8v/iUG0vTH4BtVgkSn3Tng==' \
-d '{
"country": "Canada"
}'
More on [Context.](https://docs.tweek.fm/concepts/context/intro-to-context)
#### Gradual Feature Release
Create new key in the editor "my_app/sign_button/is_enabled" with value type "boolean" and default value False.
Add new rule, remove all conditions, set the the rule value to gradual release with 50%.
Try querying configuration with different users and You'll have different results.
- http://api.dev.tweek.localtest.me:81/api/v1/keys/my_app/sign_button/is_enabled?user=barny
- http://api.dev.tweek.localtest.me:81/api/v1/keys/my_app/sign_button/is_enabled?user=robin
- http://api.dev.tweek.localtest.me:81/api/v1/keys/my_app/sign_button/is_enabled?user=ted
- http://api.dev.tweek.localtest.me:81/api/v1/keys/my_app/sign_button/is_enabled?user=lily
- http://localhost:8080/api/v1/keys/my_app/sign_button/is_enabled?user=barny
- http://localhost:8080/api/v1/keys/my_app/sign_button/is_enabled?user=robin
- http://localhost:8080/api/v1/keys/my_app/sign_button/is_enabled?user=ted
- http://localhost:8080/api/v1/keys/my_app/sign_button/is_enabled?user=lily
- etc...
More on how multi-variant keys work in Tweek. (link)
@@ -125,7 +125,7 @@ public async Task ExceptionThrown_ShouldRecoverFromException()

versions.OnError(new Exception());

await Task.Delay(20);
await Task.Delay(50);

// Assert
Assert.Equal("10002", repository.CurrentLabel);
@@ -33,10 +33,6 @@ services:
- Context__Validation__ErrorPolicy=throw

gateway:
depends_on:
- minio
- nats
- oidc-server-mock
environment:
- CONFIGOR_ENV=test

@@ -46,10 +42,8 @@ services:
ports:
- "4011:80"
environment:
- ASPNETCORE_ENVIRONMENT=Development
- CLIENT_ID=tweek-openid-mock-client
- REDIRECT_URIS=http://gateway/auth-result/oidc
- TEST_USER={"SubjectId":"00000000-0000-0000-0000-000000000000","Username":"User1","Password":"pwd"}


selenium:
image: selenium/standalone-chrome-debug:3.8.1
@@ -45,10 +45,7 @@ services:
ports:
- "4011:80"
environment:
- ASPNETCORE_ENVIRONMENT=Development
- CLIENT_ID=tweek-openid-mock-client
- REDIRECT_URIS=http://localhost:3000/auth-result/oidc
- TEST_USER={"SubjectId":"00000000-0000-0000-0000-000000000000","Username":"User1","Password":"pwd"}

secrets:
tweek_ssh_private_key:
@@ -109,13 +109,25 @@ services:
ports:
- "4008:8222"
- "4222:4222"

oidc-server-mock:
container_name: oidc-server-mock
image: soluto/oidc-server-mock
ports:
- "4011:80"
environment:
- ASPNETCORE_ENVIRONMENT=Development
- CLIENT_ID=tweek-openid-mock-client
- REDIRECT_URIS=http://localhost:8080/auth-result/oidc
- TEST_USER={"SubjectId":"user","Username":"user","Password":"pwd"}

gateway:
ports:
- "4099:80"
- "8080:80"
depends_on:
- minio
- nats
- nats
- oidc-server-mock
environment:
- TWEEKGATEWAY_SECURITY_CORS_ENABLED=true
- TWEEKGATEWAY_SECURITY_CORS_MAXAGE=60
@@ -129,7 +141,7 @@ services:
- TWEEKGATEWAY_SECURITY_POLICYSTORAGE_MINIOSECURE=false
- TWEEKGATEWAY_SECURITY_POLICYSTORAGE_NATSENDPOINT=nats://nats:4222
- TWEEKGATEWAY_SECURITY_TWEEKSECRETKEY_PATH=/run/secrets/tweek_ssh_private_key
- TWEEKGATEWAY_SECURITY_AUTH_BASICAUTH_REDIRECTURLS=['http://localhost:4099']
- TWEEKGATEWAY_SECURITY_AUTH_BASICAUTH_REDIRECTURLS=['http://localhost:8080']
- TWEEKGATEWAY_CONFIGFILEPATH=/config/gateway.json
volumes:
- ./gateway/config:/config
@@ -7,17 +7,34 @@
},
"security": {
"auth": {
"providers": {}
"providers": {
"mock": {
"name": "Mock OpenId Connect server",
"issuer": "http://localhost:4011",
"authority": "http://localhost:4011",
"jwks_uri": "http://oidc-server-mock/.well-known/openid-configuration/jwks",
"client_id": "tweek-openid-mock-client",
"login_info": {
"login_type": "oidc",
"additional_info": {},
"scope": "openid profile email",
"response_type": "id_token"
}
}
},
"basic_auth": {
"redirect_urls": [
"http://localhost:4099"
]
}
}
},
"v1Hosts": {
"api": [
"api",
"api.localtest.me"
],
"authoring": [
"authoring",
"authoring.localtest.me"
"api": [
"api"
],
"authoring": [
"authoring"
]
}
}
}
@@ -17,9 +17,11 @@ First of all, run the skaffold.
```
skaffold dev -f ./deployments/kubernetes/skaffold.yaml
```
Then, create port forwarding

If using Minikube create port forwarding:
```bash
kubectl port-forward deployment/gateway 8080:80
kubectl port-forward deployment/oidc-server-mock 8081:80
```

Finally, open in browser [http://localhost:8080](http://localhost:8080).
@@ -57,10 +57,8 @@ apiVersion: v1
metadata:
name: api
spec:
type: NodePort
selector:
app: api
ports:
- port: 80
targetPort: 80
nodePort: 30003
targetPort: 80
@@ -43,5 +43,4 @@ spec:
app: authoring
ports:
- port: 3000
targetPort: 3000
nodePort: 30005
targetPort: 3000
@@ -15,23 +15,14 @@ spec:
imagePullPolicy: IfNotPresent
ports:
- containerPort: 3000
volumeMounts:
- name: tweek-secrets
mountPath: /run/secrets/tweek-secrets
volumes:
- name: tweek-secrets
secret:
secretName: tweek-secrets
---
kind: Service
apiVersion: v1
metadata:
name: editor
spec:
type: NodePort
selector:
app: editor
ports:
- port: 3000
targetPort: 3000
nodePort: 30004
Oops, something went wrong.

0 comments on commit 0902b4b

Please sign in to comment.