Skip to content
Permalink
Browse files

Merge pull request #20 from Krocodial/release-3

Release 3
  • Loading branch information
Krocodial committed Oct 22, 2019
2 parents 7bdfc50 + 30c407c commit 6ecfcecc1e2bedc05d71277d20244aa9bf191e9d
@@ -30,7 +30,7 @@ RUN chown -R 1001:0 /opt/app-root

USER 1001

CMD python manage.py migrate && python manage.py createcachetable && python manage.py check && gunicorn --bind 0.0.0.0:8080 --access-logfile - --error-logfile - --reload wsgi
CMD python manage.py migrate && python manage.py createcachetable && python manage.py check && gunicorn --bind 0.0.0.0:8080 --access-logfile - --error-logfile - --timeout 300 -w 3 --threads 2 --keep-alive 10 --graceful-timeout 300 wsgi

#docker build --no-cache -t classy .
#docker run -p 0.0.0.0:8080:8080 --env-file .env classy
@@ -111,6 +111,9 @@ def deployTemplates(String name, String env, String tag, String pr, String git_r
"SOURCE_REPOSITORY_URL=${git_repo}",
"SOURCE_REPOSITORY_REF=${git_branch}")



openshift.selector("cronjobs").delete()

backend = openshift.process(
readFile(file:"${backendDC}"),
@@ -39,34 +39,28 @@ $ cd <project directory>

Edit the configuration file
```sh
$ vi dsc/settings.py
$ vi project/settings.py
```
Change USE_MYSQL_DB to 'True' if you are using a MYSQL DB, set it to 'False' if you want to use a SQLite DB.

If you change it to 'True' you will need to load the database credentials into the environment variables. The python dot-env library is currently used to load the credentials into memory from a hidden file in the project directory.

Make sure the variables with the corresponding names in the configuration file are base64 encoded and placed within the '.env' file in the project directory, these will then be auto-loaded by the app on startup.

WARNING: ensure this file is excluded from version control and limit the permissions so that only the owner of the file may read it. Failure to do so may result in a compromised DB.

Set BYPASS_AUTH to True
Set Debug to False unless you wish to expose debug information

Run setup script
```sh
$ chmod +x setup.sh
$ ./setup.sh
```

This will create a virtual environment, install all python dependencies inside of it, then run the included tests with a coverage report.


```sh
python3 -m venv envs
source envs/bin/activate
pip install -r requirements.txt
set -a
. .env
set +a
export POSTGRESQL_SERVICE_HOST=localhost
```

To provide file handling functionality, as well as other long-running process' django-background tasks is used. Tasks will be created and registered for the user automatically. However, to actually run these tasks a simple cron job must be setup.

Using Crontab -e the following is all you need to do:
```
*/1 * * * * /usr/bin/flock -n /tmp/QRH7mA40aRL2NVyVUbcH.lockfile ~/crontab/file_handler.sh
*/1 * * * * /usr/bin/flock -n /tmp/uj5l6n7iAGtM8gx9fNuo.lockfile ~/crontab/count_handler.sh
*/1 * * * * /usr/bin/flock -n /tmp/QRH7mA40aRL2NVyVUbcH.lockfile python ~/manage.py process_tasks --queue=uploader
*/1 * * * * /usr/bin/flock -n /tmp/uj5l6n7iAGtM8gx9fNuo.lockfile python ~/manage.py process_tasks --queue=counter
```

Flock will need to be installed to allow the creation and management of locks, preventing process bombs.
@@ -77,38 +71,54 @@ Change the path at the end of each to correspond to the project directory, and m
Finally run the development server
```sh
$ source envs/bin/activate
$ python manage.py runserver <host>:<port>
$ python manage.py runserver <host>:<port>
$ python manage.py process_tasks (if no cronjob is setup)
```

Congratulations! You now have a running metadata classification repository
Congratulations! You now have a running security classification repository

# Deployment

For local deployment in docker containers the steps are fairly easy.

Ensure docker is installed and configured correctly.
Then:
```sh
docker run -p 0.0.0.0:5432:5432 -e POSTGRES_PASSWORD=docker -d postgres
git clone https://github.com/krocodial/classy .
docker build -t classy --no-cache .
docker run -p 0.0.0.0:8080:8080 --env-file .env -d classy
cd conf
docker build -t nginx-proxy --no-cache .
docker run -p 0.0.0.0:1337:1337 --env-file .env -d nginx-proxy
```

Once the steps in 'Initial Setup' are complete follow this section to find out how to serve classy using a webserver.
It should be noted that no cronjob runs by default, which means aggregate stats, and file upload handling won't be on by default. To remedy this fire up another shell and do the following.
```sh
docker container ls
docker exec -it <container-name> /bin/bash
python manage.py process_tasks
```

Using apt or another package manager
```sh
$ sudo apt-get install libmysqlclient-dev python3 python3-venv python3-pip apache2 apache2-dev libapache-mod-wsgi-py3
```
Now you have a local development deployment of classy up and running.

Create a configuration based on the projects location for apache2, eg in /etc/apache2/sites-available. An example apache virtualhost configuration file in included in tests/config_example
## Backing up

Modify the dsc/settings.py file. An example file, as well as comments describing the different options available in the settings file is included in the tests/settings_example file
Default Django flows can be used to backup data.
```
python manage.py dumpdata --exclude=auth.permission --exclude=auth.group --exclude=contenttypes > <backup file>.json
./oc rsync <pod>:/path/to/backup.json /local/dir/to/backup/to
```


## Running Tests

Tests are built using Django's testing libraries. Once in the virtual environment (source envs/bin/activate), navigate to the project directory and run the tests via 'python manage.py test'. If the application is not setup correctly this will not work.

Note: The tests will automatically run if the 'setup.sh' script is run. Thus they are included only for development purposes to make sure you do not mess anything up.
Tests are built using Django's testing libraries. Once in the virtual environment (source envs/bin/activate), navigate to the project directory and run the tests via 'python manage.py test tests/'. If the application is not setup correctly this will not work.


## Security
Authentication can be handled by SiteMinder or alternatively Django's built-in authentication by changing the BYPASS_AUTH variable in the settings file.

Authorization is a customization of Django's provided authorization functionality. This allows segmentation of data as well as user authorization.
Authentication is provided via the bcgov keycloak service. The Authorization flow is used, with checks to allow immediate revocation in the event of account compromise. By default user accounts are denied any access to this tool.

Policies for use will be provided by the FLNR security team.
Authorization is very granular for classification access, allowing business areas to only access what they are in charge of. Users can also be granted over-arching access to functionality based on Django's default authorization flow.



@@ -103,7 +103,7 @@ def clean(self):
if classification == 'UN' or classification == 'PU':
protected_type = ''

def save(self, user, approver):
def save(self, user, approver=None):
classy = super(ClassificationForm, self).save(commit=False)
classy.save(user, approver)
for dep in self.cleaned_data['dependents']:
@@ -133,7 +133,7 @@ class Meta:
model = Classification
fields = ['classification', 'protected_type', 'owner', 'dependents', 'notes', 'masking', 'state']

def save(self, user, approver):
def save(self, user, approver=None):
classy = super(LogDetailSubmitForm, self).save(commit=False)
classy.save(user, approver)
return classy
@@ -169,7 +169,7 @@ def clean(self):
)


def save(self, user, approver):
def save(self, user, approver=None):
classy = super(LogDetailForm, self).save(commit=False)
classy.save(user, approver)
return classy
@@ -0,0 +1,30 @@
# Generated by Django 2.1.8 on 2019-09-27 18:42

from django.conf import settings
from django.db import migrations, models
import django.db.models.deletion


class Migration(migrations.Migration):

dependencies = [
('classy', '0008_auto_20190725_1404'),
]

operations = [
migrations.AlterField(
model_name='classificationlogs',
name='approver',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='Approver', to=settings.AUTH_USER_MODEL),
),
migrations.AlterField(
model_name='classificationlogs',
name='classy',
field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='classy.Classification'),
),
migrations.AlterField(
model_name='classificationreview',
name='classy',
field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='classy.Classification'),
),
]
@@ -0,0 +1,34 @@
# Generated by Django 2.1.8 on 2019-10-09 23:46

from django.db import migrations, models
import django.db.models.deletion


class Migration(migrations.Migration):

dependencies = [
('classy', '0009_auto_20190927_1142'),
]

operations = [
migrations.AlterField(
model_name='application',
name='acronym',
field=models.CharField(max_length=20, unique=True),
),
migrations.AlterField(
model_name='application',
name='name',
field=models.CharField(blank=True, max_length=100, unique=True),
),
migrations.AlterField(
model_name='classificationlogs',
name='classy',
field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='classy.Classification'),
),
migrations.AlterField(
model_name='classificationreview',
name='classy',
field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='classy.Classification'),
),
]
@@ -40,8 +40,8 @@
)

class Application(models.Model):
acronym = models.CharField(max_length=20, unique=True, blank=True)
name = models.CharField(max_length=100, unique=True)
acronym = models.CharField(max_length=20, unique=True)
name = models.CharField(max_length=100, unique=True, blank=True)
poc = models.CharField(max_length=100, blank=True, help_text='Point of contact for the application')
description = models.TextField(max_length=300, blank=True)

@@ -159,7 +159,7 @@ def clean(self):
@receiver(m2m_changed, sender=Classification.dependents.through)
def m2m_change(instance, **kwargs):
# disable during fixture loading
if kwargs['raw']:
if 'raw' in kwargs:
return
log = ClassificationLogs.objects.filter(classy_id=instance.pk).order_by('-id')[0]
log.dependents.set(instance.dependents.all())
@@ -202,7 +202,7 @@ class ClassificationLogs(models.Model):
owner = models.ForeignKey(Application, on_delete=models.CASCADE, blank=True, null=True)
dependents = models.ManyToManyField(Application, related_name="log_dependent", blank=True)
user = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE, related_name='Modifier')
approver = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE, related_name='Approver')
approver = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE, related_name='Approver', blank=True, null=True)
state = models.CharField(max_length=1, choices=state_choices)
masking_change = models.TextField(blank=True)
note_change = models.TextField(blank=True)

0 comments on commit 6ecfcec

Please sign in to comment.
You can’t perform that action at this time.