Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/windows #78

Closed
wants to merge 59 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
591bd2d
spike: windows integration
shoeffner Aug 9, 2016
9c7a1fc
adds bootstrapping
shoeffner Aug 12, 2016
144cafa
file preps, usage, command parsing, simple build
shoeffner Aug 19, 2016
fba1c5b
Wasabi starts and removes, but database connections do not work corre…
shoeffner Aug 30, 2016
dccf7cd
start up modifications
shoeffner Sep 1, 2016
66a6925
Merge branch 'feature/windows' of github.com:intuit/wasabi into featu…
shoeffner Sep 1, 2016
458d873
docker wasabi on windows is alive!
shoeffner Sep 1, 2016
957cbc0
Merge branch 'feature/windows' of github.com:intuit/wasabi into featu…
shoeffner Sep 2, 2016
3c10ef8
wasabi.bat supports all but status and package from wasabi.sh
shoeffner Sep 5, 2016
d063d3b
adding status for completeness
shoeffner Sep 5, 2016
5e88f3f
windows bootstrap to README
shoeffner Sep 5, 2016
b779aa6
Merge remote-tracking branch 'origin/develop' into feature/windows
shoeffner Sep 6, 2016
bc5ff4d
less verbose
shoeffner Sep 6, 2016
7a929bd
text eol to lf instead of auto
shoeffner Sep 6, 2016
eccae0f
Grunt accepts --apiHost=..., --apiPort=..., --apiHostBaseUrlValue=...…
shoeffner Sep 6, 2016
b8249cb
updated README for windows (developers): added installed dependencies…
shoeffner Sep 6, 2016
d3685d6
fixing EOL to LF
shoeffner Sep 6, 2016
ca79dc9
moving all gitattributes to one place
shoeffner Sep 7, 2016
a914ceb
simplified Docker image creation
shoeffner Sep 7, 2016
f72dc82
better tagging using current commit-hash
shoeffner Sep 7, 2016
438d39c
try: simplifying container.sh as used on windows
shoeffner Sep 7, 2016
c9566da
Merge branch 'feature/windows' of github.com:intuit/wasabi into featu…
shoeffner Sep 7, 2016
88feb4c
fixing EOL
shoeffner Sep 7, 2016
686285f
simplified docker building processes and fixed fixme: reuse start_con…
shoeffner Sep 7, 2016
06b1509
reactivating unit tests
shoeffner Sep 7, 2016
abbdd4b
Merge branch 'feature/windows' of https://github.com/intuit/wasabi in…
shoeffner Sep 7, 2016
61276f4
Removed mv from Dockerfile and adjusted Entrypoint.
shoeffner Sep 7, 2016
1257398
Merge branch 'feature/windows' of github.com:intuit/wasabi into featu…
shoeffner Sep 7, 2016
55c6d76
unified network
shoeffner Sep 7, 2016
84807ff
easier to read docker create command
shoeffner Sep 7, 2016
5b23a79
simplified dockerization
shoeffner Sep 7, 2016
5765c51
Merge branch 'feature/windows' of https://github.com/intuit/wasabi in…
shoeffner Sep 7, 2016
4e81af5
git versioning fallback and dependency
shoeffner Sep 7, 2016
5319379
taping time drift issue by resetting it before running integration tests
shoeffner Sep 7, 2016
d0d7c2f
Dockerfile jdk -> jre
shoeffner Sep 7, 2016
f23d9ba
call instead of cmd /c call; typos
shoeffner Sep 8, 2016
e1c4b63
making rolluptests configurable with -D parameters and applying them
shoeffner Sep 8, 2016
357a085
Ignoring race-conditioned unit tests again. Reopens #20, and #45.
shoeffner Sep 8, 2016
290c3a1
duct taping long methods to poll for experiments to start before assi…
shoeffner Sep 8, 2016
67b4e11
Merge remote-tracking branch 'origin/develop' into feature/windows
shoeffner Sep 8, 2016
b1a3e33
fixed usage command outputs on container.sh failure
shoeffner Sep 9, 2016
d2ccec6
adjusting for code review comments
shoeffner Sep 12, 2016
d548d6e
added mysql password back
shoeffner Sep 12, 2016
796412c
made resource:api more robust
shoeffner Sep 16, 2016
4119543
Merge branch 'feature/windows' of https://github.com/intuit/wasabi in…
shoeffner Sep 21, 2016
982ec82
variables for paths
shoeffner Sep 21, 2016
edbb50e
cassandra -> Cassandra
shoeffner Sep 21, 2016
1b83047
fixing "\FOO was not expected" if PATH contains "Program Files (x86)"…
shoeffner Sep 22, 2016
2381839
moved troubleshoot to FAQ and added virtualization information
shoeffner Sep 24, 2016
9d9d230
added git initialization if downloading without git
shoeffner Sep 24, 2016
d072363
Merge remote-tracking branch 'origin/develop' into feature/windows
shoeffner Sep 25, 2016
8cb8b2a
added alias dev_ui for ui_dev on resource:, made localhost explicit, …
shoeffner Oct 3, 2016
f02d333
point UI to correct endpoint
shoeffner Oct 10, 2016
f4d9c5f
added explanations about getting the correct feature branch on window…
shoeffner Oct 12, 2016
9b2b510
UI blocking to avoid variable passing to new shells
shoeffner Nov 3, 2016
1c9b302
Merge branch 'develop' into feature/windows
shoeffner Nov 3, 2016
f6d176f
fixing openssl issue with gems
shoeffner Nov 10, 2016
f799bda
Merge remote-tracking branch 'origin/develop' into feature/windows
shoeffner Dec 21, 2016
216671a
simplified note about downloading the zip on windows
shoeffner Dec 21, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
* text eol=lf
*.bat text eol=crlf

*.asc.enc binary

*.png binary
*.ico binary
*.gif binary
161 changes: 127 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Learn more about how Wasabi can empower your team to move from hunches to action

## Get Started

The following steps will help you install the needed tools, then build and run a complete Wasabi stack. Note, at this time, only Mac OS X is supported.
The following steps will help you install the needed tools, then build and run a complete Wasabi stack.

#### Bootstrap Your Environment

Expand All @@ -60,7 +60,7 @@ The following steps will help you install the needed tools, then build and run a

Installed tools include: [homebrew 0.9](http://brew.sh), [git 2](https://git-scm.com),
[maven 3](https://maven.apache.org), [java 1.8](http://www.oracle.com/technetwork/java/javase/overview/index.html),
[docker 1.12](https://docker.com), [node 6](https://nodejs.org/en) and [python 2.7](https://www.python.org).
[docker 1.12](https://docker.com), [node 6](https://nodejs.org/en).


##### Ubuntu
Expand All @@ -77,13 +77,65 @@ Bootstrapping on Ubuntu requires sudo privileges to install all the required dep

NOTE: A reboot is required after running the bootstrap command on Ubuntu.

For all other processes (build, start etc.) the commands are same for Ubuntu and Mac OS.
For all other processes (build, start etc.) the commands are the same for Ubuntu and Mac OS.

Installed tools include: [git 2](https://git-scm.com),
[maven 3](https://maven.apache.org), [OpenJdk 8](http://openjdk.java.net/projects/jdk8/),
[docker 1.12](https://docker.com), [node 6](https://nodejs.org/en) and [python 2.7](https://www.python.org)

Similar tooling will work for Windows. Contribute a patch :)

##### Windows (7+)

To install Wasabi's dependencies on Windows we use [Chocolatey][win_choco] which
needs administrator rights in your [cmd.exe][win_cmd_admin].

If you have git, just run:
```dos
% git clone https://github.com/intuit/wasabi.git
% cd wasabi
% bin\wasabi.bat bootstrap
```

If you don't have git the easiest way is to [download][url_develop_zip] the
latest code and unzip it. You will then only have to run
```dos
% bin\wasabi.bat bootstrap
```

For all other processes (build, start etc.) the commands are almost the same as
for the other operating systems: just make sure to replace `bin/wasabi.sh`
with `bin\wasabi.bat`. Note that the UI needs to be started with a separate server, so
`bin\wasabi.bat resource:ui` is a must on Windows.

If you run into problems, please consult the [FAQ](#FAQ+Windows) and create
an issue if it doesn't help you.

Installed tools include: [Chocolatey][win_choco], [git][choco_git],
[Docker][choco_docker], [Maven 3][choco_maven], [jdk 1.8][choco_jdk],
[docker-machine][choco_docker-machine], [VirtualBox][choco_virtualbox],
[node.js][choco_nodejs] (+ [bower][npm_bower], [grunt-cli][npm_grunt-cli],
[yo][npm_yo]), and [ruby][choco_ruby] (+ [compass][gem_compass],
[fpm][gem_fpm]).

[win_choco]: https://chocolatey.org/
[win_cmd_admin]: https://technet.microsoft.com/en-us/library/cc947813(v=ws.10).aspx)
[url_develop_zip]: https://github.com/intuit/wasabi/archive/develop.zip
[win_hyperv]: https://msdn.microsoft.com/virtualization/hyperv_on_windows/quick_start/walkthrough_compatibility
[choco_docker]: https://chocolatey.org/packages/docker
[choco_git]: https://chocolatey.org/packages/git.install
[choco_docker-machine]: https://chocolatey.org/packages/docker-machine
[choco_jdk]: https://chocolatey.org/packages/jdk8
[choco_maven]: https://chocolatey.org/packages/maven
[choco_nodejs]: https://chocolatey.org/packages/nodejs.install
[choco_virtualbox]: https://chocolatey.org/packages/virtualbox
[choco_ruby]: https://chocolatey.org/packages/ruby
[npm_bower]: https://www.npmjs.com/package/bower
[npm_grunt-cli]: https://www.npmjs.com/package/grunt-cli
[npm_yo]: https://www.npmjs.com/package/yo
[gem_compass]: https://rubygems.org/gems/compass/versions/1.0.3
[gem_fpm]: https://rubygems.org/gems/fpm/versions/1.4.0



#### Start Wasabi

Expand Down Expand Up @@ -124,33 +176,6 @@ Server: Jetty(9.3.z-SNAPSHOT)

Congratulations! You are the proud owner of a newly minted Wasabi instance. :)

#### Troubleshooting

* While starting Wasabi, if you see an error when the docker containers are starting up, you could do the following:

* Look at the current docker containers that have been successfully started.

```bash
% ./bin/wasabi.sh status

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8c12458057ef wasabi-main "entrypoint.sh wasabi" 25 minutes ago Up 25 minutes 0.0.0.0:8080->8080/tcp, 0.0.0.0:8090->8090/tcp, 0.0.0.0:8180->8180/tcp wasabi-main
979ecc885239 mysql:5.6 "docker-entrypoint.sh" 26 minutes ago Up 26 minutes 0.0.0.0:3306->3306/tcp wasabi-mysql
2d33a96abdcb cassandra:2.1 "/docker-entrypoint.s" 27 minutes ago Up 27 minutes 7000-7001/tcp, 0.0.0.0:9042->9042/tcp, 7199/tcp, 0.0.0.0:9160->9160/tcp wasabi-cassandra
```

* The above shell output shows a successful start of 3 docker containers needed by Wasabi: wasabi-main (the Wasabi server),
wasabi-mysql, and wasabi-cassandra. If any of these are not running, try starting them individually. For example, if the
MySQL container is running, but Cassandra and Wasabi containers failed to start (perhaps due to a network timeout docker
could not download the Cassandra image), do the following:

```bash
% ./bin/wasabi.sh start:cassandra

% ./bin/wasabi.sh start:wasabi
```



#### Call Wasabi

Expand Down Expand Up @@ -329,9 +354,7 @@ development: {
```

Now while that was fun, in all likelihood you will be using an IDE to work on Wasabi. In doing so, you need only
add the configuration information above to the JVM commandline prior to startup:

> Wasabi runtime configuration:
add the configuration information below to the JVM commandline prior to startup:

```bash
-DnodeHosts=localhost -Ddatabase.url.host=localhost
Expand Down Expand Up @@ -390,3 +413,73 @@ Steps to contribute:
8. Obtain 2 approval _squirrels_ before your changes can be merged

Thank you for your contribution!


## Troubleshooting & FAQ

#### General

##### I see errors when starting up docker containers.
If you see an error when the docker containers are starting up, check which docker containers that have been successfully started.

```bash
% ./bin/wasabi.sh status

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8c12458057ef wasabi-main "entrypoint.sh wasabi" 25 minutes ago Up 25 minutes 0.0.0.0:8080->8080/tcp, 0.0.0.0:8090->8090/tcp, 0.0.0.0:8180->8180/tcp wasabi-main
979ecc885239 mysql:5.6 "docker-entrypoint.sh" 26 minutes ago Up 26 minutes 0.0.0.0:3306->3306/tcp wasabi-mysql
2d33a96abdcb cassandra:2.1 "/docker-entrypoint.s" 27 minutes ago Up 27 minutes 7000-7001/tcp, 0.0.0.0:9042->9042/tcp, 7199/tcp, 0.0.0.0:9160->9160/tcp wasabi-cassandra
```
The above shell output shows a successful start of 3 docker containers needed by Wasabi: wasabi-main (the Wasabi server),
wasabi-mysql, and wasabi-cassandra. If any of these are not running, try starting them individually. For example, if the
MySQL container is running, but Cassandra and Wasabi containers failed to start (perhaps due to a network timeout docker
could not download the Cassandra image), do the following:

```bash
% ./bin/wasabi.sh start:cassandra
% ./bin/wasabi.sh start:wasabi
```

#### FAQ Windows
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice points!


##### I can't run VirtualMachine properly.
Check if your computer has virtualization enabled. For this you need to enter your BIOS (or UEFI)
and in general search for Intel VT or AMD-V and enable them.

##### How to connect an IDE's wasabi-process to the containers?
One important difference: Since Docker native only supports very specific
[Windows 10 distributions][win_hyperv], we have to user docker-machine.
This means you can not reach the service at `http://localhost:8080/` but
at `http://192.168.99.100:8080/` (by default - see `docker-machine ip wasabi`).
For development this also means to supply the Java VM arguments
`-DnodeHosts=192.168.99.100 -Ddatabase.url.host=192.168.99.100` when running
Wasabi to connect to Cassandra and MySQL inside the docker network.

##### How to connect the UI to IDE's wasabi-process?
You will also need to redirect the UI. By default `bin\wasabi.bat resource:ui`
points to `docker-machine ip wasabi`. You can use the Windows exclusive
`bin\wasabi.bat resource:dev_ui` to use `localhost` as your UI's target.

##### When I open 127.0.0.1:9000 in my browser, I don't see the UI.
For Windows, you have to first start a little server. Just run `bin\wasabi.bat resource:ui`
to use the development grunt server. The UI should start automatically.

##### Wasabi has incorrect times! My IDE-run integration tests fail! Help!
It is a known problem with docker-machine that its time server stops while
the host machine sleeps. If your system behaves weird, check if
`docker-machine ssh wasabi date` gives you the expected UTC time. If not, run
`for /f %I in ('ruby -e "puts Time.now.utc.strftime(%Q{%Y%m%d%H%M.%S})"') do @docker-machine ssh wasabi "sudo date --set %I"`.
Wasabi does this time correction whenever you start the integration tests via
`bin\wasabi.bat test`.

##### Swagger mentions `curl`, how do I get it?
To use the `curl` commands consider `choco install curl`.

##### I want to test a specific branch, but when I download the zip I end up on develop.
If you chose to use the zip file over git you can checkout the branch after you bootstrapped wasabi
(i.e. installed git). For example if the branch is `feature/new-auth-mechanism` you would run the
following command:
```dos
git checkout feature/new-auth-mechanism
```
Note that the `git checkout` is usually enough, if not, try `git reset --hard origin/feature/new-auth-mechanism` before.
45 changes: 10 additions & 35 deletions bin/container.sh
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,8 @@ stop_docker() {
}

start_container() {
# fix: do not re-create
docker network create --driver bridge ${docker_network} >/dev/null 2>&1
docker network ls | grep ${docker_network} 1>/dev/null || \
docker network create --driver bridge ${docker_network} >/dev/null 2>&1

cid=$(docker ps -aqf name=${1})

Expand All @@ -96,7 +96,7 @@ start_container() {
usage "docker run --net=${docker_network} --name ${1} ${3} -d ${2} ${4}" 1
# todo: ?better way? ... see about moving polling to the app-start
beerMe 30
elif [ "${cid}" != "running" ]; then
else
cids=$(docker inspect --format '{{.State.Status}}' ${cid})

if [ "${cids}" == "paused" ]; then
Expand Down Expand Up @@ -141,27 +141,18 @@ start_wasabi() {
start_docker

id=$(fromPom modules/main development application.name)
wcip=$(docker inspect --format "{{ .NetworkSettings.Networks.${docker_network}.IPAddress }}" ${project}-cassandra)
wmip=$(docker inspect --format "{{ .NetworkSettings.Networks.${docker_network}.IPAddress }}" ${project}-mysql)

remove_container ${project}-main

if [ "${verify}" = true ] || ! [ docker inspect ${project}-main >/dev/null 2>&1 ]; then
echo "${green}${project}: building${reset}"

# sed -i -e "s|\(http://\)localhost\(:8080\)|\1${mip}\2|g" modules/main/target/${id}/content/ui/dist/scripts/config.js 2>/dev/null;
docker build -t ${project}-main:${USER}-$(date +%s) -t ${project}-main:latest modules/main/target/${id}
docker build -t ${project}-main:$(git rev-parse --short=8 HEAD) -t ${project}-main:latest modules/main/target/${id}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@felixgao's idea :-)

fi

echo "${green}${project}: starting${reset}"

wenv="WASABI_CONFIGURATION=-DnodeHosts=${wcip} -Ddatabase.url.host=${wmip}"

# fixme: try to reuse the start_container() method instead of 'docker run...' directly; currently a problem with quotes in ${wenv} being passed into container.
docker run --net=${docker_network} --name ${project}-main -p 8080:8080 -p 8090:8090 -p 8180:8180 \
-e "${wenv}" -d ${project}-main || \
usage "docker run --net=${docker_network} --name ${project}-main -p 8080:8080 -p 8090:8090 -p 8180:8180 -e \"${wenv}\" -d ${project}-main" 1

start_container ${project}-main ${project}-main:$(git rev-parse --short=8 HEAD) "-p 8080:8080 -p 8090:8090 -p 8180:8180 -e WASABI_CONFIGURATION=\"-DnodeHosts=${project}-cassandra -Ddatabase.url.host=${project}-mysql\""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quite nice :)

echo -ne "${green}chill'ax ${reset}"

status
Expand All @@ -175,39 +166,23 @@ start_cassandra() {
}

console_cassandra() {
wcip=$(docker inspect --format "{{ .NetworkSettings.Networks.${docker_network}.IPAddress }}" ${project}-cassandra)

docker run --net=${docker_network} -it --rm ${cassandra} cqlsh ${wcip} || \
usage "unable to run command: docker run --net=${docker_network} -it --rm ${cassandra} cqlsh ${wcip}" 1
docker run --net=${docker_network} -it --rm ${cassandra} cqlsh ${project}-cassandra || \
usage "unable to run command: % docker run --net=${docker_network} -it --rm ${cassandra} cqlsh ${project}-cassandra" 1
}

start_mysql() {
pwd=mypass

start_docker
start_container ${project}-mysql ${mysql} "-p 3306:3306 -e MYSQL_ROOT_PASSWORD=${pwd}"

wmip=$(docker inspect --format "{{ .NetworkSettings.Networks.${docker_network}.IPAddress }}" ${project}-mysql)
sql=$(cat << EOF
create database if not exists ${project};
grant all privileges on ${project}.* to 'readwrite'@'localhost' identified by 'readwrite';
grant all on *.* to 'readwrite'@'%' identified by 'readwrite';
flush privileges;
EOF
)

docker run --net=${docker_network} -it --rm ${mysql} mysql -h${wmip} -P3306 -uroot -p${pwd} -e "${sql}" || \
usage "unable to run command: % docker run --net=${docker_network} -it --rm ${mysql} mysql -h${wmip} -P3306 -uroot -p${pwd} -e \"${sql}\"" 1
start_container ${project}-mysql ${mysql} "-p 3306:3306 -e MYSQL_ROOT_PASSWORD=${pwd} -e MYSQL_DATABASE=${project} -e MYSQL_USER=readwrite -e MYSQL_PASSWORD=readwrite"

[ "${verify}" = true ] && console_mysql
}

console_mysql() {
pwd=mypass
wmip=$(docker inspect --format "{{ .NetworkSettings.Networks.${docker_network}.IPAddress }}" ${project}-mysql)

docker run --net=${docker_network} -it --rm ${mysql} mysql -h${wmip} -P3306 -uroot -p${pwd} || \
usage "unable to run command: % docker run --net=${docker_network} -it --rm ${mysql} mysql -h${wmip} -P3306 -uroot -p${pwd}" 1
docker run --net=${docker_network} -it --rm ${mysql} mysql -h${project}-mysql -P3306 -uroot -p${pwd} || \
usage "unable to run command: % docker run --net=${docker_network} -it --rm ${mysql} mysql -h${project}-mysql -P3306 -uroot -p${pwd}" 1
}

status() {
Expand Down
Loading