Skip to content
Expanding NextCloudPi
Shell Python PHP JavaScript CSS Dockerfile Other
Branch: gsoc2019-devel
Clone or download
Pull request Compare This branch is 334 commits ahead, 60 commits behind nextcloud:master.
FaniD Update README.md
Small updates on README.md file
Latest commit 0be0dec Aug 26, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github/ISSUE_TEMPLATE
.travis Update .travis dir Aug 25, 2019
Ansible
SecureWilly
Vagrant Updated create_vm.sh Aug 22, 2019
bin Fixes in the backward updates mechanism Aug 26, 2019
docker-swarm
docker
etc Small changes according to suggestions Aug 26, 2019
ncp-app ncp-app: support NC14 as well Jan 13, 2019
ncp-web ncp-web: add backups panel May 1, 2019
tests Update .travis.yml Jun 16, 2019
updates Fixes in the backward updates mechanism Aug 26, 2019
.dockerignore build: fully local builds Sep 22, 2018
.gitignore add Vagrantfile Oct 23, 2018
.travis.yml Update travis.yml and .travis dir Aug 25, 2019
CONTRIBUTING.md Replace links from github wiki to docs.nextcloudpi.com (nextcloud#881) Apr 30, 2019
README.md Update README.md Aug 26, 2019
Vagrantfile curl: build fix Jan 7, 2019
armbian.sh
batch.sh
build-SD-armbian.sh
build-SD-berryboot.sh build: berryboot tested ok Jul 5, 2018
build-SD-rpi.sh
build-VM.sh curl: build fix Jan 7, 2019
build-docker.sh Update build-docker.sh Aug 26, 2019
buildlib.sh Try arm32v7 instead of armhf Jun 17, 2019
changelog.md nc-scan-auto: recursive and home-only options May 11, 2019
clear_travis_cache.py
config-bananapi.conf
config-odroidxu4.conf general build improvements May 29, 2018
config-orangepizeroplus2-h5.conf added config for OrangePi Zero Plus2 H5 Jun 27, 2018
config-rock64.conf armbian: update rock64 branch Jul 3, 2018
docker-compose-armhf.yml add system tests Oct 19, 2018
docker-compose-nc-armhf.yml
docker-compose-nc.yml
docker-compose-ncpdev.yml
docker-compose.yml add system tests Oct 19, 2018
install.sh
lamp.sh
ncp.sh
post-inst.sh build: fix post script Jan 15, 2019
restart_failed_jobs.py
run_update_history.sh Small changes according to suggestions Aug 26, 2019
tag_and_push.sh build: add tag -> changelog -> push script May 10, 2018
update.sh Update according to suggestions Aug 26, 2019

README.md

The missing features of NextCloudPi - Google Summer of Code 2019

NCP_gsoc logo

NextCloudPi GSoC Project is one of the accepted projects by GFOSS for GSoC 2019.

Abstract

NextCloudPi

Nextcloud requires many advanced tasks in order to be installed. NextCloudPi lowers the barrier and makes it easier to the end user by preconfiguring most of the setup. Extra features with minimal input help maintaining the installation.

Preinstalling and preconfiguring Nextcloud and all the related components (apache, php, mysql, redis, etc) as well as providing an easy to use interface makes it more user friendly to anyone who wants to take control of their private data.

NextCloudPi consists of code that provides the following components:

  • A ready to use image for Raspberry Pi, Odroid HC1, rock64 and other boards
  • Docker image (arm/arm64/x86)
  • Curl Installer
  • Extra features

It is an official project of Nextcloud, which is maintained by the community.

You can find the source code on the official project's Github repository

Check the full documentation at docs.nextcloudpi.com

NextCloudPi GSoC Project

This project aims to extend NextCloudPi with additional features. These features will not only offer end users more options, but will help NextCloudPi to become more prominent and enter the professional world.

Branches

The several branches of the repository are explained below:

  • Original branches with the latest commits of the official nextcloudpi repository:
    • origin/original-master: stable
    • origin/original-devel: testing
  • Up-to-date branches to the official nextcloudpi repository, mixed with my GSoC commits:
    • master: stable
    • devel: testing
  • Outdated branches (compared to the official nextcloudpi repository), containing my GSoC commits exclusively
    • gsoc2019-master: stable
    • gsoc2019-devel: testing
  • The rest branches are the development branches of each feature

New Features

On my blog, under the category GSoC, there are several articles about all features created on this project.

1. CI/CD to build releases on GitHub

Adapt the NextCloudPi project to the modern software development, Continuous Integration / Continuous Delivery, in order to reduce risks for each build and clear the way to get valuable features out to users faster.

Travis CI was used to build, test and produce the docker images of NextCloudPi. The travis.yml, which triggers a build only at a tagged commit for gsoc2019-travis branch, implements 4 Build Stages with 3 parallel jobs running at each one of them. The DockerHub credentials should be added to Travis page before using the travis.yml.

Some extra tools are implemented to provide automatic restart of failed jobs and clearing cache:

  • restart_failed_jobs.py : Restarts any failed job, and does not return until all jobs pass. Before the execution, export you github token (export GITHUB_TOKEN=<github_token>). Execute the script after pushing a tagged commit.
  • clear_travis_cache : Clears Travis machine cache. Before the execution, export you github token (export GITHUB_TOKEN=<github_token>). Execute the script before pushing a tagged commit, in order to have a clean cache at the new build.

The following flowchart describes the travis.yml file and how these tools are used:

travis

2. NextCloudPi VM on VirtualBox easy setup and clone using Vagrant

Under the directory Vagrant, there is an automation script that creates a new NCP VM just by asking the user to provide a minimum input and can also create a clone of an existing NCP VM. The Vagrantfile produced by the automation script differs from the one provided on the official nextcloudpi repository as it contains VirtualBox specific commands. People who are having a hard time to setup a VM on their own will benefit from this script as it does not ask for any technical details.

Usage: ./create_vm.sh

This article provides a step-by-step tutorial with screenshots.

3. High availability option for big installations

Under the directory docker_swarm, there is a series of scripts that are used to create a distributed system of NextCloudPi, which replicates the data storage to all nodes. Docker Swarm is used as the container orchestrator of the system and GlusterFS docker containers to provide replication and data persistence of NCP data storage.

  • IP_library.sh: A library containing functions about the verification of IPs and choosing an available IP
  • create_swarm.sh: The automation script that creates the NCP Docker Swarm system
  • destroy_swarm.sh: Destroys existing NCP Swarm system
  • destroy_vagrant_vms.sh: Destroys any vagrant workers created by the new_worker_vagrant.sh
  • docker_concurrent.sh: Patch to increase the concurent downloads limit of Docker up to the number of workers of the Swarm
  • extra_node.sh: Adds extra node worker to the existing Swarm system
  • gluster_setup.sh: Setups GlusterFS on worker node
  • gluster_volume.sh: Configures GlusterFS volume on worker node
  • monitor.sh: Provide info about the Swarm System
  • new_worker_vagrant.sh: Create new worker VM through Vagrant

Usage: ./create_swarm.sh

ncp_swarm

This article provides a step-by-step tutorial with screenshots.

3. Ansible role

Under the directory Ansible, there are Ansible playbooks of the bash scripts of NCP server. Ansible playbooks will be used to replace the execution of NCP server's bash scripts in order to provide an alternative option of configuring NCP.

Usage (of each playbook): ansible-playbook playbooks_name.yml --extra-vars "var1=value1 var2=value2"

Final Report Gist

You can find the final report here.

Future Work

  • Complete Ansible's task: Sections CONFIG and SYSTEM are implemented. The rest sections should be converted also to Ansible playbook. When everything is converted, ncp-config should be updated using Ansible commands.
  • Fix travis.yml to produce native NCP images as well: There was an issue with the Locales at the build phase of the native images. Should be resolved. Under the directory .travis/travis_instances/non-docker there are some yml files of the attempts used to produce native images.
  • Create AppArmor profile with SecureWilly tool: An attempt was made to produce AppArmor profile for NCP but docker logs were not matching the template of SecureWilly. Needs investigation. Under the directory SecureWilly/Parser you can find the input_sample used as input to SecureWilly at this attempt.

Members

  • Google Summer of Code 2019 Participant: Fani Dimou (FaniD)
  • Mentor: Panteleimon Sarantos (Pant)
  • Mentor: Efstathios Iosifidis (iosifidis)
You can’t perform that action at this time.