Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cloudify #632

Closed
wants to merge 76 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
03a2199
Fix responder (temporary fix) (#606)
waghanza Nov 8, 2018
fad9c9f
wip : create cloud target using cli
waghanza Oct 28, 2018
c913db5
upload source file to target (in a specific directory)
waghanza Oct 30, 2018
356b4ff
add a remote exexutor
waghanza Oct 30, 2018
0ea58a6
fix conflict
waghanza Oct 30, 2018
867deed
do not user systemd -> too heavy to maintain
waghanza Oct 30, 2018
28e58b0
create command to delete droplet
waghanza Oct 30, 2018
30bbfde
add tool to extract results (as json)
waghanza Oct 31, 2018
0ed78ef
add simple exporter
waghanza Oct 31, 2018
d15114b
use .env standard to define custom/private parameters
waghanza Nov 15, 2018
0bb1d15
use native loop, because we break on a dynamic condition
waghanza Nov 15, 2018
2e17dca
forget to commit shard manifest
waghanza Nov 15, 2018
98ef85b
jobs definition to build on digitalocean
waghanza Nov 15, 2018
7b021b5
upload files just after droplet creation
waghanza Nov 15, 2018
237d361
store ip between jobs
waghanza Nov 15, 2018
d143bc1
use ubuntu since only dist with cloud-init >= 18
waghanza Nov 16, 2018
d681612
add an option to wait for cloud-init provisioning to complete
waghanza Nov 16, 2018
694d070
re-design job configuration (since a strict order is required)
waghanza Nov 16, 2018
91a7f0a
add ability to send files not in basedir
waghanza Nov 16, 2018
2f9ea33
add configuration files to run one framework in node / php / python /…
waghanza Nov 16, 2018
3e32716
add a configuration tool
waghanza Nov 19, 2018
5dfe0e4
use .env configuration file
waghanza Nov 19, 2018
93dd97c
fix jobs
waghanza Nov 19, 2018
877d7b9
wait for bundler to install
waghanza Nov 19, 2018
15a9b4c
uniformize jobs
waghanza Nov 19, 2018
2aa68c1
add setup instaruction for a python env
waghanza Nov 20, 2018
fe38455
add starlette / responder configuration file (those frameworks are di…
waghanza Nov 20, 2018
519db7c
fix jobs for one framework per language
waghanza Nov 20, 2018
1fad208
add symfony
waghanza Nov 20, 2018
0251870
clean symfony implementation
waghanza Nov 20, 2018
628e0ad
complete node suite
waghanza Nov 21, 2018
8f769cc
lint javascript / json files
waghanza Nov 21, 2018
74f152c
use environment to define droplet size / region / dist
waghanza Nov 21, 2018
5ebaa0e
add ability to use private network
waghanza Nov 21, 2018
205595b
forget to add ssh2 dependencie
waghanza Nov 21, 2018
7ee169e
fix cast issue
waghanza Nov 21, 2018
5c34ec5
little refactor
waghanza Nov 21, 2018
ccffafc
fix looping waiting for network to be live
waghanza Nov 21, 2018
632c764
use double quotes instead of simple quotes
waghanza Nov 22, 2018
f6802a8
add django
waghanza Nov 22, 2018
9efde7b
add environment variable that will allow to wait before first ssh con…
waghanza Nov 22, 2018
6bbfbf1
missing file
waghanza Nov 22, 2018
c1e2061
display errors
waghanza Nov 22, 2018
b973c96
fix laravel config file
waghanza Nov 23, 2018
5d65dba
fix typo
waghanza Nov 23, 2018
b0473eb
avoid using an environment variable to wait for ssh connection
waghanza Nov 23, 2018
7bc7d89
missing setting file for django
waghanza Nov 23, 2018
1c8d9d9
disable django
waghanza Nov 23, 2018
2df5b22
remove trailing (cli) error management
waghanza Nov 23, 2018
9e57cb5
use node 10.x
waghanza Nov 23, 2018
f86d74a
fix port on foxify / fastify run
waghanza Nov 23, 2018
7efdd2f
Could not use node 10.x on ubuntu cosmic (not available)
waghanza Nov 23, 2018
ed715eb
run test on each jobs
waghanza Nov 23, 2018
b6a1f4c
use dot instead of unerscore in host naming
waghanza Nov 23, 2018
f082295
comment framework that are not (yet) ready
waghanza Nov 23, 2018
dc6c31d
compile inside containers
waghanza Nov 26, 2018
f5dfa65
use static compilation
waghanza Nov 26, 2018
1be7bb2
add cli tool to compile binary, before upload
waghanza Nov 26, 2018
3530933
simple upload -> for binaries
waghanza Nov 26, 2018
b204337
wip : start benchmarking compiled frameworks
waghanza Nov 26, 2018
16999e2
add quart
waghanza Nov 25, 2018
bc33436
use sftp session for file upload -> upload binary
waghanza Nov 26, 2018
47464c9
use systemd to launch / monitor app
waghanza Nov 26, 2018
c51c973
clean jobs config
waghanza Nov 26, 2018
5491d01
allow only read (dynamic languages) or execute (compiled languages)
waghanza Nov 26, 2018
0f872ef
add ability to reboot (to take advantages of all updates)
waghanza Nov 27, 2018
73a87d4
wip -> working version on droplet manager
waghanza Nov 27, 2018
da6306f
wip -> complete go framework list
waghanza Nov 27, 2018
40e1761
fix go / systemd declaration
waghanza Nov 28, 2018
b7137b4
remove unused code
waghanza Nov 28, 2018
04f9996
add rust challengers
waghanza Nov 30, 2018
195d15a
update crystal service declaration
waghanza Nov 30, 2018
5ad2966
configure rust jobs
waghanza Nov 30, 2018
0bfdb32
use IO to upload file throught SSH
waghanza Nov 30, 2018
7912ba9
increase max descrptor limit (open file)
waghanza Dec 6, 2018
11b6d56
add rust challengers
waghanza Nov 30, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
24 changes: 24 additions & 0 deletions .env.template
@@ -0,0 +1,24 @@
# Main configuration file

###############################################
### SSH ACCESS (upload, start servers ...) ####
###############################################

# Key used to connect to servers (via ssh)
# Must be libssh2 compatible
SSH_KEY=

# SSH fingerprint
# Send one some providers to specify ssh key to allow on each servers
SSH_FINGERPINT=

############################
#### PROVIDERS SPECIFIC ####
############################


######################
#### DIGITALOCEAN ####
######################

DO_TOKEN=
3 changes: 3 additions & 0 deletions .gitignore
@@ -1,3 +1,6 @@
# Priate configuration file
.env

## Developer content

.DS_Store
Expand Down
2 changes: 1 addition & 1 deletion .travis.yml
Expand Up @@ -80,7 +80,7 @@ env:
- FRAMEWORK=restana
- FRAMEWORK=hapi
- FRAMEWORK=fastify
- FRAMEWORK=foxify
# - FRAMEWORK=foxify
- FRAMEWORK=koa
- FRAMEWORK=polka
- FRAMEWORK=rayo
Expand Down
41 changes: 41 additions & 0 deletions crystal/config.yml
@@ -0,0 +1,41 @@
providers:
digitalocean:
version: 0.27
config:

# Update distibution

package_update: true
package_upgrade: true

# Write systemd config file

write_files:
- path: /lib/systemd/system/web.service
permission: 0644
content: |
[Unit]
Description=web
After=network.target

[Service]
Type=simple
WorkingDirectory=/usr/src/app
ExecStart=/usr/src/app/server

[Install]
WantedBy=multi-user.target


runcmd:

# Reload systemd config
- systemctl daemon-reload

# Disable cloud-init
- systemctl disable cloud-init

- systemctl start web

# Enable created service
- systemctl enable web
2 changes: 2 additions & 0 deletions crystal/kemal/config.yml
@@ -0,0 +1,2 @@
files:
- server
18 changes: 18 additions & 0 deletions crystal/kemal/digitalocean.dockerfile
@@ -0,0 +1,18 @@
FROM jrei/crystal-alpine

# RUN apt-get -y install libssl-dev
# RUN apt-get -y install libxml2-dev
# RUN apt-get -y install libyaml-dev
# RUN apt-get -y install libgmp-dev
# RUN apt-get -y install libreadline-dev


WORKDIR /usr/src/app

COPY src src
COPY shard.yml ./

ENV shards install
RUN shards build --release --no-debug --static

RUN cp bin/server .
2 changes: 1 addition & 1 deletion crystal/kemal/src/server.cr
Expand Up @@ -20,5 +20,5 @@ end
Kemal.config.env = "production"
Kemal.run do |config|
server = config.server.not_nil!
server.bind_tcp "0.0.0.0", 3000, reuse_port: true
server.bind_tcp "0.0.0.0", 80, reuse_port: true
end
2 changes: 2 additions & 0 deletions go/beego/config.yml
@@ -0,0 +1,2 @@
files:
- server
16 changes: 16 additions & 0 deletions go/beego/digitalocean.dockerfile
@@ -0,0 +1,16 @@
FROM golang:1.11.2

WORKDIR /go/src/app

COPY main.go ./
COPY go.mod ./

ENV GO111MODULE=on
ENV CGO_ENABLED=0
ENV GOOS=linux
ENV GOARCH=amd64

RUN go build .

RUN mkdir -p /usr/src/app
RUN mv main /usr/src/app/server
2 changes: 1 addition & 1 deletion go/beego/main.go
Expand Up @@ -18,5 +18,5 @@ func main() {
ctx.WriteString("")
})

beego.Run(":3000")
beego.Run(":80")
}
2 changes: 2 additions & 0 deletions go/chi/config.yml
@@ -0,0 +1,2 @@
files:
- server
16 changes: 16 additions & 0 deletions go/chi/digitalocean.dockerfile
@@ -0,0 +1,16 @@
FROM golang:1.11.2

WORKDIR /go/src/app

COPY main.go ./
COPY go.mod ./

ENV GO111MODULE=on
ENV CGO_ENABLED=0
ENV GOOS=linux
ENV GOARCH=amd64

RUN go build .

RUN mkdir -p /usr/src/app
RUN mv main /usr/src/app/server
2 changes: 1 addition & 1 deletion go/chi/main.go
Expand Up @@ -22,5 +22,5 @@ func main() {
w.Write([]byte(""))
})

http.ListenAndServe(":3000", r)
http.ListenAndServe(":80", r)
}
45 changes: 45 additions & 0 deletions go/config.yml
@@ -0,0 +1,45 @@
providers:
digitalocean:
version: 1.11
config:

# Update distibution

package_update: true
package_upgrade: true

# Write systemd config file

write_files:
- path: /lib/systemd/system/web.service
permission: 0644
content: |
[Unit]
Description=web
After=network.target

[Service]
Type=simple
WorkingDirectory=/usr/src/app
ExecStart=/usr/src/app/server

# customize limit for this service only
# @see https://unix.stackexchange.com/questions/345595/how-to-set-ulimits-on-service-with-systemd
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target


runcmd:

# Reload systemd config
- systemctl daemon-reload

# Disable cloud-init
- systemctl disable cloud-init

- systemctl start web

# Enable created service
- systemctl enable web
2 changes: 2 additions & 0 deletions go/echo/config.yml
@@ -0,0 +1,2 @@
files:
- server
16 changes: 16 additions & 0 deletions go/echo/digitalocean.dockerfile
@@ -0,0 +1,16 @@
FROM golang:1.11.2

WORKDIR /go/src/app

COPY main.go ./
COPY go.mod ./

ENV GO111MODULE=on
ENV CGO_ENABLED=0
ENV GOOS=linux
ENV GOARCH=amd64

RUN go build .

RUN mkdir -p /usr/src/app
RUN mv main /usr/src/app/server
2 changes: 1 addition & 1 deletion go/echo/main.go
Expand Up @@ -20,5 +20,5 @@ func main() {
return c.String(http.StatusOK, "")
})

e.Start(":3000")
e.Start(":80")
}
2 changes: 2 additions & 0 deletions go/fasthttprouter/config.yml
@@ -0,0 +1,2 @@
files:
- server
16 changes: 16 additions & 0 deletions go/fasthttprouter/digitalocean.dockerfile
@@ -0,0 +1,16 @@
FROM golang:1.11.2

WORKDIR /go/src/app

COPY main.go ./
COPY go.mod ./

ENV GO111MODULE=on
ENV CGO_ENABLED=0
ENV GOOS=linux
ENV GOARCH=amd64

RUN go build .

RUN mkdir -p /usr/src/app
RUN mv main /usr/src/app/server
2 changes: 1 addition & 1 deletion go/fasthttprouter/main.go
Expand Up @@ -19,5 +19,5 @@ func main() {
fmt.Fprintf(ctx, "")
})

fasthttp.ListenAndServe(":3000", router.Handler)
fasthttp.ListenAndServe(":80", router.Handler)
}
2 changes: 2 additions & 0 deletions go/gf/config.yml
@@ -0,0 +1,2 @@
files:
- server
16 changes: 16 additions & 0 deletions go/gf/digitalocean.dockerfile
@@ -0,0 +1,16 @@
FROM golang:1.11.2

WORKDIR /go/src/app

COPY main.go ./
COPY go.mod ./

ENV GO111MODULE=on
ENV CGO_ENABLED=0
ENV GOOS=linux
ENV GOARCH=amd64

RUN go build .

RUN mkdir -p /usr/src/app
RUN mv main /usr/src/app/server
2 changes: 1 addition & 1 deletion go/gf/main.go
Expand Up @@ -18,6 +18,6 @@ func main() {
r.Response.Write(r.Get("id"))
})
s.SetFileServerEnabled(false)
s.SetPort(3000)
s.SetPort(80)
s.Run()
}
2 changes: 2 additions & 0 deletions go/gin/config.yml
@@ -0,0 +1,2 @@
files:
- server
16 changes: 16 additions & 0 deletions go/gin/digitalocean.dockerfile
@@ -0,0 +1,16 @@
FROM golang:1.11.2

WORKDIR /go/src/app

COPY main.go ./
COPY go.mod ./

ENV GO111MODULE=on
ENV CGO_ENABLED=0
ENV GOOS=linux
ENV GOARCH=amd64

RUN go build .

RUN mkdir -p /usr/src/app
RUN mv main /usr/src/app/server
2 changes: 1 addition & 1 deletion go/gin/main.go
Expand Up @@ -24,5 +24,5 @@ func main() {
c.String(http.StatusOK, "")
})

r.Run(":3000")
r.Run(":80")
}
2 changes: 2 additions & 0 deletions go/gorilla-mux/config.yml
@@ -0,0 +1,2 @@
files:
- server
16 changes: 16 additions & 0 deletions go/gorilla-mux/digitalocean.dockerfile
@@ -0,0 +1,16 @@
FROM golang:1.11.2

WORKDIR /go/src/app

COPY main.go ./
COPY go.mod ./

ENV GO111MODULE=on
ENV CGO_ENABLED=0
ENV GOOS=linux
ENV GOARCH=amd64

RUN go build .

RUN mkdir -p /usr/src/app
RUN mv main /usr/src/app/server
2 changes: 1 addition & 1 deletion go/gorilla-mux/main.go
Expand Up @@ -27,5 +27,5 @@ func main() {
})

http.Handle("/", r)
http.ListenAndServe(":3000", r)
http.ListenAndServe(":80", r)
}
2 changes: 2 additions & 0 deletions go/iris/config.yml
@@ -0,0 +1,2 @@
files:
- server
16 changes: 16 additions & 0 deletions go/iris/digitalocean.dockerfile
@@ -0,0 +1,16 @@
FROM golang:1.11.2

WORKDIR /go/src/app

COPY main.go ./
COPY go.mod ./

ENV GO111MODULE=on
ENV CGO_ENABLED=0
ENV GOOS=linux
ENV GOARCH=amd64

RUN go build .

RUN mkdir -p /usr/src/app
RUN mv main /usr/src/app/server
2 changes: 1 addition & 1 deletion go/iris/main.go
Expand Up @@ -17,5 +17,5 @@ func main() {
ctx.WriteString("")
})

app.Run(iris.Addr(":3000"))
app.Run(iris.Addr(":80"))
}
2 changes: 2 additions & 0 deletions go/muxie/config.yml
@@ -0,0 +1,2 @@
files:
- server
16 changes: 16 additions & 0 deletions go/muxie/digitalocean.dockerfile
@@ -0,0 +1,16 @@
FROM golang:1.11.2

WORKDIR /go/src/app

COPY main.go ./
COPY go.mod ./

ENV GO111MODULE=on
ENV CGO_ENABLED=0
ENV GOOS=linux
ENV GOARCH=amd64

RUN go build .

RUN mkdir -p /usr/src/app
RUN mv main /usr/src/app/server