Skip to content

Commit

Permalink
fixup! fixup! Fixes #16914: Clarify webapp contributing documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
gpoblon committed Mar 30, 2020
1 parent 7cf6f83 commit 5629291
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 67 deletions.
75 changes: 19 additions & 56 deletions contributing/dev_env_setup.sh → contributing/setup_dev_env.sh
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/bin/bash

# Do NOT run this script as root
# Disclaimer: the --full script has been written and tested on and for a Debian10 and may not work on other distributions
# Disclaimer: the --install script has been written and tested on and for a Debian10 and may not work on other distributions
# Note that this script has been designed to work as a full and standalone installer, every single package it needs is installed from within it

# If running on a VM
Expand All @@ -11,28 +11,22 @@
set -xe
username=$1
gituser=$2
action=$3
rudder_dir=/home/$username/rudder


if [ $action = "--full" ]
then
######################################################
# PACKAGE DEPENDENCIES INSTALLATION + HOME PRE-SETUP #
######################################################

cd into a directory that is easily accessible by other users
sudo /usr/sbin/usermod -aG sudo $username
sudo apt update
# apt upgrade
sudo apt install -y git openssh-server python3 python3-pip python python-pip openjdk-11-jdk net-tools ldap-utils maven apache2
sudo service ssh start

sudo chown $username -R /home/$username/

mkdir -p $rudder_dir
cd $rudder_dir

sudo /usr/sbin/usermod -aG sudo $username
sudo apt update
sudo apt install -y git openssh-server curl wget python3 python3-pip openjdk-11-jdk net-tools ldap-utils maven
sudo service ssh start
pip3 install docopt requests pexpect urllib3
###################################
# REQUIRED SOFTWARES INSTALLATION #
###################################
Expand Down Expand Up @@ -88,13 +82,16 @@ then
git clone https://github.com/Normation/rudder-techniques.git
git clone https://github.com/Normation/rudder-tests
git clone https://github.com/Normation/rudder-api-client
git clone https://github.com/Normation/ncf.git

########################
# RTF DEV ENV VM SETUP #
########################

ln -s rudder-api-client $rudder_dir/rudder-tests
sudo mkdir -p /var/rudder/
sudo chown -R $username /var/rudder
cd $rudder_dir/rudder-tests
sed -i "s/env python/env python3/" $rudder_dir/rudder-tests/rtf
ln -s ../rudder-api-client
cd $rudder_dir/rudder-api-client/lib.python
sh build.sh # must be executed from within its own directory
cd $rudder_dir/rudder-tests
Expand All @@ -104,49 +101,23 @@ then
netstat -laputn | grep 15432 # check purpose only, should print : `tcp 0 0 0.0.0.0:15432 0.0.0.0:* LISTEN`
cd $rudder_dir/

####################################
# NCF API & TECHNIQUE EDITOR SETUP #
####################################
# local rudder agent required only if the technique editor is part of the test
wget --quiet -O- "https://repository.rudder.io/apt/rudder_apt_key.pub" | sudo apt-key add -
echo "deb http://repository.rudder.io/apt/6.0/ $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/rudder.list
sudo apt update
sudo apt install -y rudder-agent

# setup ncf api
wget https://repository.rudder.io/build-dependencies/virtualenv/virtualenv-16.5.0.tar.gz
tar xzvf virtualenv-16.5.0.tar.gz
rm -rf virtualenv-16.5.0.tar.gz
mv virtualenv-16.5.0/* $rudder_dir/ncf/api/
cd $rudder_dir/ncf/api/
python virtualenv.py flask
flask/bin/pip install -r requirements.txt
# setup technique editor
# depending on your distrib, folder might be: `/etc/apache2/conf.d/`
sudo cp $rudder_dir/ncf/api/dev_env/ncf-builder.conf /etc/apache2/conf-enabled/
sudo sed -i "s/8042/8080/g" /etc/apache2/conf-enabled/ncf-builder.conf
sudo sed -i "s/\/path\/to\/ncf\/builder/\/home\/$username\/rudder\/ncf\/builder/g" /etc/apache2/conf-enabled/ncf-builder.conf
chmod 755 $rudder_dir/ncf/builder -R
sudo a2enmod rewrite
sudo a2enmod proxy
sudo a2enmod proxy_http
sudo service apache2 restart

###########################
# PRE SETUP FOR SOFTWARES #
###########################

mkdir -p /var/rudder/inventories/incoming /var/rudder/share /var/rudder/inventories/accepted-nodes-updates /var/rudder/inventories/received /var/rudder/inventories/failed /var/log/rudder/core /var/log/rudder/compliance/ /var/rudder/run/
touch /var/log/rudder/core/rudder-webapp.log /var/log/rudder/compliance/non-compliant-reports.log /var/rudder/run/api-token

sudo mkdir -p /var/rudder/inventories/incoming /var/rudder/share /var/rudder/inventories/accepted-nodes-updates /var/rudder/inventories/received /var/rudder/inventories/failed /var/log/rudder/core /var/log/rudder/compliance/ /var/rudder/run/ /var/rudder/configuration-repository
sudo touch /var/log/rudder/core/rudder-webapp.log /var/log/rudder/compliance/non-compliant-reports.log /var/rudder/run/api-token
# since a lot has been installed as root apps launched as $username could need permissions to access its home space
sudo chown $username -R /var/rudder/
sudo chown $username -R /var/rudder
sudo chown $username -R /var/log/rudder
sudo chown $username -R /home/$username/

cp -R $rudder_dir/rudder-techniques/techniques /var/rudder/configuration-repository/
# setup maven dependencies in intellij
mkdir -p /home/$username/.m2
cp $rudder_dir/rudder/contributing/settings.xml /home/$username/.m2/settings.xml
sed -i "s/\[PATH TO \.m2 DIRECTORY\]/\/home\/$username\//g" /home/$username/.m2/settings.xml
sed -i "s/\[PATH TO \.m2 DIRECTORY\]/\/home\/$username\//" /home/$username/.m2/settings.xml
#takes a while
cd $rudder_dir/rudder/webapp/sources && mvn clean install

Expand All @@ -159,14 +130,6 @@ then

# https://github.com/Normation/rudder/blob/master/contributing/webapp.md#test-ldap-connection to set it up
# /opt/ApacheDirectoryStudio/ApacheDirectoryStudio
fi

#######################
# RUN PART, NCF API * #
#######################

export PYTHONPATH=$rudder_dir/ncf/tools:$PYTHONPATH
$rudder_dir/ncf/api/run.py

# at every startup, run:
# $rudder_dir/rudder-tests/rtf platform setup debian9_dev
# at every startup, run:
# $rudder_dir/rudder-tests/rtf platform setup debian9_dev
18 changes: 7 additions & 11 deletions contributing/webapp.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,11 +54,7 @@ This is a **straightforward guide** aims to setup all tools and settings on a Li
### Needed packages
If you are starting with a clean machine, you might want to take it all, should be needed at some point
`apt update && apt install -y git openssh-server python3 python3-pip python python-pip openjdk-11-jdk net-tools ldap-utils maven`
`pip3 install docopt requests pexpect urllib3` (or `pip`)

Required only if you need to use the Technique Editor
`apt update && apt install -y apache2` (could be `httpd` depending on your OS. RHEL for example)
`pip3 install flask` (or `pip`)
`pip install docopt requests pexpect urllib3` (or `pip3`)

### VirtualBox installation
```bash
Expand Down Expand Up @@ -330,19 +326,19 @@ If your testing case requires it to be running, please follow this documentation
> Note: the setup process might not be painless. Some tweaks could be necessary depending on your distribution
#### Entire Script
All the previous steps, including the rtf setup and the ncf api / technique editor are summed up in the following script: [dev_env_setup.sh](https://github.com/Normation/rudder/blob/master/contributing.sh)
All the previous steps BUT the ncf-api/technique editor are summed up in the following script: [setup_dev_env.sh](https://github.com/Normation/rudder/blob/master/contributing/setup_dev_env.sh).\
Though, another automated setup script is available directly in the ncf repository (see previous section).

Do not run this script as sudo! Though some commands require sudo privileges, you may be prompted to type sudo password

> Important: as a requirement, if not done yet you need to fork the following repo: `https://github.com/Normation/rudder`.
The first param is your local user
The second is your gitusername (used to clone your `Normation/rudder` fork)
The third param defines if the `--full` install should be done (or simply the default `--run` part).
This script should be copied anywhere on your machine and simply executed. It makes use of your `<user>` name and takes it as its second parameter:
This script should be copied anywhere on your machine and simply executed.
```bash
./dev_env_setup.sh <user> <gituser> --full
./setup_dev_env.sh <user> <gituser>
```
Then let it run (the running script is the ncf api)

> Important note: This script does not setup Intellij ([Setup workspace development with IntelliJ and Maven](#part-2---setup-workspace-development-with-intellij-and-maven))
and Apache Directory Studio ([LDAP connection](#test-ldap-connection)). These still have to be set manually.
Expand All @@ -355,7 +351,7 @@ Then let it run (the running script is the ncf api)
> All rudder repos will be cloned (including rtf and ncf) in `/home/<user>/rudder/`
> Disclaimer: this script might not work on your machine. If it does not, it still is a good guideline when trying to setup a dev environment since it traces every single step of this readme + rtf setup readme + ncf api and technique editor readme
> Disclaimer: this script might not work on your machine. If it does not, it still is a good guideline when trying to setup a dev environment since it traces every single step of this readme + rtf setup readme

#### Documentations
Expand Down

0 comments on commit 5629291

Please sign in to comment.