Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add YAMLs to deploy development branch using Kubernetes #1349

Merged
merged 6 commits into from Jul 29, 2017

Conversation

Projects
None yet
7 participants
@singhpratyush
Copy link
Member

commented Jul 22, 2017

Short description

Fixes #1348.

I have:

  • There is a corresponding issue for this pull request.
  • Mentioned the Issue number in the pull request commit message Fixes #<number> commit message
  • There is only strictly only one commit per issue.

For the reviewers

I have:

  • Reviewed this pull request by an authorized contributor.
  • The reviewer is assigned to the pull request.
@singhpratyush

This comment has been minimized.

Copy link
Member Author

commented Jul 22, 2017

@Achint08 @djmgit @hemantjadon @kavithaenair @SKrPl @vibhcool: Please see if you can follow the deployment steps and review the files added.

@codecov-io

This comment has been minimized.

Copy link

commented Jul 22, 2017

Codecov Report

Merging #1349 into development will increase coverage by 0.02%.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff                @@
##             development   #1349      +/-   ##
================================================
+ Coverage           9.19%   9.21%   +0.02%     
- Complexity           405     408       +3     
================================================
  Files                202     202              
  Lines              17846   17846              
  Branches            3300    3300              
================================================
+ Hits                1641    1645       +4     
+ Misses             15898   15896       -2     
+ Partials             307     305       -2
Impacted Files Coverage Δ Complexity Δ
src/org/loklak/data/ElasticsearchClient.java 10.36% <0%> (+0.81%) 12% <0%> (+3%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bd20d25...eecedf0. Read the comment docs.

@singhpratyush singhpratyush force-pushed the singhpratyush:1348 branch 3 times, most recently from a2ba07e to 6930c2f Jul 22, 2017


### 3.2. Updating configurations

While updating the configurations, it should be ensured that the `api-service.yml` configuration is not recreated. This would retain pervious IP and save us from the trouble of updating DNS.

This comment has been minimized.

Copy link
@Achint08

Achint08 Jul 22, 2017

Contributor

It should be previous

This comment has been minimized.

Copy link
@singhpratyush

singhpratyush Jul 22, 2017

Author Member

@Achint08: Fixed. Thanks for pointing out.

@singhpratyush singhpratyush force-pushed the singhpratyush:1348 branch 8 times, most recently from 3e6db72 to dcc8cd2 Jul 22, 2017

echo "Make sure that persistent disk is already created."
kubectl create -R -f ${DIR}/yamls/development/
echo "Waiting for server to start up. ~40s."
sleep 40

This comment has been minimized.

Copy link
@Achint08

Achint08 Jul 22, 2017

Contributor

Why 40?

This comment has been minimized.

Copy link
@singhpratyush

singhpratyush Jul 22, 2017

Author Member

We need to start the server and also wait for LoadBalancer to assign a public IP.

@singhpratyush singhpratyush changed the title Add YAMLs to deploy development branch using Kubernetes [WIP] Add YAMLs to deploy development branch using Kubernetes Jul 22, 2017

@singhpratyush singhpratyush force-pushed the singhpratyush:1348 branch from dcc8cd2 to 570d08b Jul 23, 2017

@singhpratyush singhpratyush force-pushed the singhpratyush:1348 branch 3 times, most recently from 4512d80 to a041e91 Jul 27, 2017

@singhpratyush singhpratyush changed the title [WIP] Add YAMLs to deploy development branch using Kubernetes Add YAMLs to deploy development branch using Kubernetes Jul 27, 2017

@singhpratyush singhpratyush force-pushed the singhpratyush:1348 branch 3 times, most recently from fbfa6b6 to 138c6a5 Jul 27, 2017

sed -i.bak 's/^\(elasticsearch_transport.enabled\).*/\1=true/' conf/config.properties && \
sed -i.bak 's/^\(elasticsearch_transport.addresses\).*/\1=elasticsearch.elasticsearch:9300/' conf/config.properties && \
echo "while true; do sleep 10;done" >> bin/start.sh && \
cat conf/config.properties && \

This comment has been minimized.

Copy link
@singhpratyush

singhpratyush Jul 28, 2017

Author Member

[TODO] Remove this.

This comment has been minimized.

Copy link
@vibhcool

vibhcool Jul 28, 2017

Member

I didn't understand. what does this review mean, are you reviewing your own code. 😆

This comment has been minimized.

Copy link
@singhpratyush

singhpratyush Jul 28, 2017

Author Member

Just some reminders so that I do not miss this.

This comment has been minimized.

Copy link
@vibhcool

vibhcool Jul 28, 2017

Member

could you add these comments in code instead. These aren't added in PR, these reminders are supposed to be added in code

This comment has been minimized.

Copy link
@singhpratyush

singhpratyush Jul 28, 2017

Author Member

Made the changes already.

echo "Creating loklak deployment"
kubectl create -R -f ${DIR}/yamls/development/api-server/
echo "Trying to fetch public IP"
kubectl get services --namespace=web

This comment has been minimized.

Copy link
@singhpratyush

singhpratyush Jul 28, 2017

Author Member

[TODO] Wait for some time.

@singhpratyush singhpratyush force-pushed the singhpratyush:1348 branch from 138c6a5 to ec04527 Jul 28, 2017

@kavithaenair
Copy link
Member

left a comment

@singhpratyush good job 👍 . Tested it and it works fine for me.

@vibhcool

This comment has been minimized.

Copy link
Member

commented Jul 28, 2017

@singhpratyush , I am not clear what this change is doing.
Went through the related issue.
It deploys the project to kubernetes. In this task what are the subtasks?
some explaination will be good 😬

@singhpratyush

This comment has been minimized.

Copy link
Member Author

commented Jul 29, 2017

So this Kubernetes deployment would have two major components -

  1. The loklak server
  2. Elasticsearch cluster

The configurations point to creating these two services with appropriate properties. The shell scripts in kubernetes/bin are used to create, delete or update the deployments.

Also, we needed a separate Dockerfile as some configuration changes were required so that loklak could use the external ES cluster. During the build process, these changes are built and pushed to Docker hub too (along with the previous images).

Please comment inline if you still need more explanation.

@singhpratyush singhpratyush force-pushed the singhpratyush:1348 branch from ec04527 to cb42352 Jul 29, 2017

@singhpratyush singhpratyush force-pushed the singhpratyush:1348 branch from cb42352 to 36a7a30 Jul 29, 2017

@@ -0,0 +1,54 @@
apiVersion: v1
kind: ReplicationController

This comment has been minimized.

Copy link
@niranjan94

niranjan94 Jul 29, 2017

Member

Make it a deployment instead of replication controller. (Deployment is the newer and better version of a replication container)
https://kubernetes.io/docs/concepts/workloads/controllers/deployment/

This comment has been minimized.

Copy link
@singhpratyush

singhpratyush Jul 29, 2017

Author Member

I took the reference from Kubernetes example for ES and this comes from there. Will make the changes.

@@ -0,0 +1,27 @@
# Main deployment for API server
apiVersion: apps/v1beta1
kind: Deployment

This comment has been minimized.

Copy link
@niranjan94

niranjan94 Jul 29, 2017

Member

No disk needed for this ?

This comment has been minimized.

Copy link
@singhpratyush

singhpratyush Jul 29, 2017

Author Member

Since the deployment wouldn't be generating dumps, we do not need a disk here.

This comment has been minimized.

Copy link
@niranjan94

niranjan94 Jul 29, 2017

Member

So where will the dump generating instance be ? The one that consolidates everything ?

This comment has been minimized.

Copy link
@singhpratyush

singhpratyush Jul 29, 2017

Author Member

Yes. The master branch deployment.

This comment has been minimized.

Copy link
@niranjan94

niranjan94 Jul 29, 2017

Member

I meant ... where is the kubernetes config for that sorry didn't see PR title

@@ -0,0 +1,54 @@
apiVersion: v1

This comment has been minimized.

Copy link
@niranjan94

niranjan94 Jul 29, 2017

Member

And call the file es-deployment.yml

@@ -0,0 +1,17 @@
apiVersion: v1

This comment has been minimized.

Copy link
@niranjan94

niranjan94 Jul 29, 2017

Member

Call the file es-service.yml

@singhpratyush singhpratyush force-pushed the singhpratyush:1348 branch 2 times, most recently from e7e5539 to 49d1ea9 Jul 29, 2017

containers:
- name: server
image: loklak/loklak_server:latest-kubernetes-development
livenessProbe:

This comment has been minimized.

Copy link
@niranjan94

niranjan94 Jul 29, 2017

Member

No redinessProbe ?

This comment has been minimized.

Copy link
@singhpratyush

singhpratyush Jul 29, 2017

Author Member

Added.

value: "true"
- name: HTTP_ENABLE
value: "true"
ports:

This comment has been minimized.

Copy link
@niranjan94

niranjan94 Jul 29, 2017

Member

No liveliness probe and readiness probe ?

This comment has been minimized.

Copy link
@singhpratyush

singhpratyush Jul 29, 2017

Author Member

This image will not be updated. Do we still need it here?

This comment has been minimized.

Copy link
@niranjan94

niranjan94 Jul 29, 2017

Member

Yep. We will need it here. It is not just used when image is updated. livenessProbe probe frequently checks the pods to see if it hasn't crashed for some reason. And if it has crashed, kubernetes will attempt a recovery by restarting.

This comment has been minimized.

Copy link
@singhpratyush

singhpratyush Jul 29, 2017

Author Member

Done.

@singhpratyush singhpratyush force-pushed the singhpratyush:1348 branch 2 times, most recently from b1b8e76 to 0846456 Jul 29, 2017

- name: es
livenessProbe:
httpGet:
path: /

This comment has been minimized.

This comment has been minimized.

Copy link
@singhpratyush

singhpratyush Jul 29, 2017

Author Member

@niranjan94: Made the change for both probes.

@singhpratyush singhpratyush force-pushed the singhpratyush:1348 branch from 0846456 to eecedf0 Jul 29, 2017

@mariobehling mariobehling merged commit 63e3b06 into loklak:development Jul 29, 2017

4 checks passed

codacy/pr Good work! A positive pull request.
Details
codecov/patch Coverage not affected when comparing bd20d25...eecedf0
Details
codecov/project 9.21% (+0.02%) compared to bd20d25
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@singhpratyush singhpratyush deleted the singhpratyush:1348 branch Jul 29, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.