Release v0.2

CDN.yml

@@ -0,0 +1,50 @@
+---
+#
+# Copyright 2017 Intel Corporation
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom
+# the Software is furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+# IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
+# DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH
+# THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+#
+
+# This Playbook would deploy the whole CDN components.
+
+
+- hosts: trafficservers
+  environment: "{{ proxy_env }}"
+  roles:
+  - role: trafficserver
+
+
+- hosts: ffmpeg_servers
+  environment: "{{ proxy_env }}"
+  roles:
+  - role: ffmpeg
+
+
+- hosts: svt_servers
+  environment: "{{ proxy_env }}"
+  roles:
+  - role: svt
+
+
+- hosts: nginx_servers
+  environment: "{{ proxy_env }}"
+  roles:
+  - role: nginx
+
+

README.md

@@ -0,0 +1,228 @@
+## Content Delivery Network (CDN):
+
+  Content Delivery Network are growing in importance are being viewed as new class of CPSs.
+  The objective of this project is to share an optimize recipe and a CDN reference solution
+  based on open source frameworks. This is based on SKL-Purley platform and is a pre cursor
+  to upcoming launch of Intel Select Solution for CDN on Cascade Lake platform.
+
+
+
+  > Require Ansible 2.4 or newer \
+  > Expects NFVi-BKC Ubuntu-16.04 or Ubuntu-18.04, CentOS-7.6 and RHEL-7.6 hosts
+  > Package will detect the respective Operating System and install the CDN components
+
+
+ 1. These playbooks deploy a implementation of various components of CDN such as `ATS` (version 7.1.5),
+    `ffmpeg` (n4.0.3), `nginx` (version 1.14.0)+ `rtmp-module` and `SVT` (v1.2.0). 
+
+ 2. To use CDN components, first edit `group_vars/all` (hostname of nginx node for creating ssl 
+    certificate and proxy environment). This can also be done by:
+
+    Please set following environment variables-
+  ```
+     $ export CDN_DIR="Enter the path of CDN directory"
+     $ export http_proxy="Enter the http_proxy"
+     $ export https_proxy="Enter the https_proxy"
+     $ export hostname="Enter the hostname or IP of server"
+
+     $ printf "proxy_env\n http_proxy=\"$http_proxy\"\n https_proxy=\"$https_proxy\"\n\nhostname=\"$hostname\"\n" | sudo tee $CDN_PATH/group_vars/all
+  ```     
+
+ 3. Also edit the `inventory` file that contains the hostnames/IPs and login credential of 
+    the machines on which you want these components to install.
+
+ 4. Ansible has host key checking enabled by default. If a host is reinstalled and has a different 
+    key in `known_hosts`, this will result in an error message until corrected. You can disable 
+    this behavior, you can do so by editing `/etc/ansible/ansible.cfg`
+
+  ```
+    [defaults]
+    host_key_checking = False
+  ```
+
+  Alternatively this can be set by the ANSIBLE_HOST_KEY_CHECKING environment variable:
+
+   ```
+    $ export ANSIBLE_HOST_KEY_CHECKING=False
+   ```
+
+ 5. Generate ssh key in the ansible machine, which we have to copy to all the remote hosts for 
+    doing deployments or configurations on them:
+
+   ``` 
+    $ ssh-keygen -t rsa -b 4096 -C "username@ip_add_of_ansible_machine"
+   ```
+
+  Now copy the ssh key generated to the remote hosts (Before copying the ssh key make sure that 
+  you are able to ssh the remote host where you want to copy the key):
+
+   ```
+    $ ssh-copy-id remote_user@remote_ip
+   ```
+
+  To set up SSH agent to avoid retyping passwords, you can do:   
+
+   ```
+    $ ssh-agent bash
+    $ ssh-add ~/.ssh/id_rsa
+   ```
+  For more detail about ansible setup refer link [Getting Started Ansible](https://docs.ansible.com/ansible/latest/user_guide/intro_getting_started.html).
+
+ 6. Now as we have setup our `inventory` file and other configurations are done, lets try pinging 
+    all the servers listed in the `inventory` file.
+
+   ```
+    $ ansible -i inventory all -m ping
+   ```
+
+  If pinging get successfull, then we are good and can go further :) 
+
+ 7. For running the playbook go inside CDN directory where you have `CDN.yml`, `inventory` etc., should 
+    be present. The ansible playbook command is given below:
+
+   ```
+    $ ansible-playbook -i inventory CDN.yml --become -K
+   ```
+
+  here -K, ask for privilege escalation password
+
+  When the playbook run complete, the CDN components will be installed successfully on the target machines. 
+
+
+ ### ansible_install.sh
+
+  The `ansible_install.sh` script provided will install the ansible and configure the target machines.
+  First make the script executable and then run in same directory like:
+
+```
+   $ chmod +x ansible_install.sh
+   $ ./ansible_install.sh
+```
+
+  1. Script will install the dependencies required for ansible, and ansible version 2.7.1. 
+  2. It will also generate the ssh key in ansible machine (where ansible is installing) for this it will 
+     prompt for IP address username of ansible machine.
+  3. Now it will prompt for how many target machines you want to configure. 
+  4. Then you have to enter the username and IP address of the target machines one by one, so that ssh key 
+     will successfully copied to these target machines.
+
+  If you are using ansible_install.sh script for installing ansible then you don't need to generate and copy ssh key manually. 
+
+
+
+ ## ROLES: CDN Components
+
+
+  ### Apache traffic Server (ATS)
+
+  Apache Traffic Server is a high-performance web proxy cache that improves network efficiency and 
+  performance by caching frequently-accessed information at the edge of the network. This role install 
+  and configure the apache traffic server from source. After `CDN.yml` run completes, you can start the 
+  trafficserver using `start_ats.yml`, before starting trafficserver you need to edit the `remap.config` 
+  in target machine at `/opt/ats/etc/trafficserver/remap.config`, edit the IP and port of your server, you can also edit 
+  `storage.config`, `records.config` as well as other config files accordingly in target machines at `/opt/ats/etc/trafficserver/*`.
+
+  This Playbook will copy the optimal config files to target machines, if you want to edit any of config file 
+  you can do in target machine at `/opt/ats/etc/trafficserver/*`
+
+ ```
+   $ ansible-playbook -i inventory start_ats.yml --become -K 
+ ```
+
+ If you have made any changes in config files and want to restart the trafficserver then run the 
+ `stop_ats.yml`, like:
+
+  ```
+   ansible-playbook -i inventory stop_ats.yml --become -K 
+  ```
+ Then start the trafficserver using `start_ats.yml` 
+
+  vars directory contains the variable that are used in `tasks/main.yml`
+  handlers directory contains the handlers that are triggers by notify in `tasks/main.yml`
+  template directory contains the config files that will be copied to target machines
+
+
+  ### NGINX
+
+  NGINX is server for web serving, media streaming. In addition to its HTTP and HTTPS server capabilities. 
+  **nginx+rtmp-module** Media streaming, http and https. This role install and configure the nginx+rtmp-module 
+  server from source. 
+
+  This Playbook will copy the optimal nginx.conf files to target machines, if you want to make changes in conf
+  file you can do that at `/usr/local/nginx/conf/nginx.conf`.
+
+
+  After CDN.yml run completes, you can start the nginx using `start_nginx.yml` inside nginx role, like:
+
+  ```
+    $ ansible-playbook -i inventory start_nginx.yml --become -K 
+  ```
+
+  If you made any changes in conf files and want to restart the nginx then run the `stop_nginx.yml`, like:
+
+  ```
+    $ ansible-playbook -i inventory stop_nginx.yml --become -K 
+  ```
+
+  Then start the nginx using `start_ats.yml` like above. 
+
+  vars directory contains the variable that are used in `tasks/main.yml`
+  handlers directory contains the handlers that are triggers by notify in `tasks/main.yml`
+  template directory contains the conf file that will be copied to target machines
+
+
+  ### FFMPeg
+
+  FFmpeg is a command line tool for video and audio transcoding for both live and static content.
+  This role install and configure the ffmpeg from source.
+  ffmpeg is a very fast video and audio converter that can also grab from a live audio/video source.
+  It can also convert between arbitrary sample rates and resize video on the fly with a high 
+  quality polyphase filter.  
+
+  This playbook will configure following codecs:
+
+  - H.264 (libx264) video encoder
+  - H.265 (libx265) video encoder
+  - AV1 (libaom) video encoder/decoder
+  - VP8/VP9 (libvpx) video encoder/decoder
+  - AAC (libfdk-aac) audio encoder
+  - MP3 (libmp3lame) audio encoder
+  - Opus (libopus) audio decoder and encoder
+
+  For running the ffmpeg, go to localtion where input files are stored and run the ffmpeg command
+  with or without codec:
+
+  ```
+    $ ffmpeg -i InputVideo.mpg ...[video options] [audio options] [output]
+  ```
+
+  For more detail, please refer manual page of ffmpeg:
+
+  ```
+    $ man ffmpeg 
+  ```
+
+  ### Scalabale Video Transcode(SVT)
+
+  The Scalable Video Technology for HEVC Encoder (SVT-HEVC Encoder) is an HEVC-compliant encoder library core 
+  that achieves excellent density-quality tradeoffs, and is highly optimized for Intel Xeon  Scalable 
+  Processor and on D processors. \
+  This role install ans build the SVT-HEVC from source. 
+
+  1. For running SVT copy the binary (`HevcEncoderApp`, `libHevcEncoder.so`) from */opt/SVT-HEVC/Bin/Release* 
+     to any location of your choice.
+  2. Change the permissions on the sample application `HevcEncoderApp` executable by running the command: 
+
+  ```
+     $ chmod +x HevcEncoderApp
+  ```
+
+  3. cd into your chosen location
+  4. Run the sample application to encode.
+  5. Sample application supports reading from pipe.Eg.
+
+  ```
+     $ ffmpeg -i [input.mp4] -nostdin -f rawvideo -pix_fmt yuv420p - | 
+       ./HevcEncoderApp -i stdin -n [number_of_frames_to_encode] -w [width] -h [height]
+  ```
+

ansible_install.sh

@@ -0,0 +1,100 @@
+#!/bin/bash
+
+#
+# Copyright 2017 Intel Corporation
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom
+# the Software is furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+# IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
+# DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH
+# THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+#
+
+
+# Check if we're root
+if [ "$EUID" -ne 0 ]; then
+        echo "Please run this as root"
+        exit 1
+fi
+
+if [ -f /etc/os-release ]; then
+	. /etc/os-release
+	OS=$NAME
+	VER=$VERSION_ID
+	echo
+	echo "**********************************"
+	echo "  Detected OS: $OS"
+	echo "  Installing packages..."
+	echo "**********************************"
+	echo 
+fi
+
+if [ "$OS" = "Ubuntu" ]; then
+	apt-get -y install software-properties-common sshpass python3-pip python-pip build-essential libssl-dev libffi-dev python-dev python-keyczar
+        pip3 install Jinja2
+	pip install httplib2
+	pip install cryptography
+	pip install pyyaml
+
+	apt-add-repository ppa:ansible/ansible
+	apt-get update
+	apt-get install ansible
+
+elif [ "$OS" = "Red Hat Enterprise Linux Server" ] || [ "$OS" = "CentOS Linux" ] || [ "$OS" = "Fedora" ]; then
+	yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
+	yum install -y ansible 
+	yum install -y python sshpass
+fi
+
+echo
+echo -e "**********************************"
+echo -e "  Packages installed"
+echo -e "**********************************"
+echo 
+
+read -p "Please enter the IP of host (Ansible) machine (IP of this machine): " -r HOST_IP
+
+read -p "Please enter the user name of host machine: " -r HOST_USER
+
+# Generating the ssh-key in host machine
+ssh-keygen -t rsa -b 4096 -C ${HOST_USER}@${HOST_IP}
+
+echo
+echo -e "*********************************************************"
+echo -e " ssh-key is successfully generated in host machine  "
+echo -e "**********************************************************"
+echo 
+
+read -p "Enter the numbers of target machines: " -r NUM_TARGETS
+
+# Copying the ssh-key to target machines
+for i in $(seq 1 $NUM_TARGETS)
+do
+
+read -p "Please enter the IP of target machine ${i}: " -r TARGET_IP
+
+read -p "Please enter the user name of target machine ${i}: " -r TARGET_USER
+
+ssh-copy-id ${TARGET_USER}@${TARGET_IP}
+
+echo
+echo -e "*********************************************"
+echo -e " ssh-key is copied to target machine ${i} "
+echo -e "**********************************************"
+echo 
+
+done
+
+