new release (#128)

* Create PR for #68 * docs: update docs * Create PR for #74 * fix: prevent storing password hashes for oidc auth * Create PR for #78 * docs: update setup guide (#73) * Create PR for #72 * chore: update compose reference * docs: update deployment guide * fix: cleanup requirements * docs: update deployment guide * docs: moved deployment guide to separate repository * fix: fixed superuser creation * chore: fix typo Co-authored-by: dnikolay-ebc <dnikolay-ebc@users.noreply.github.com> Co-authored-by: dnikolay-ebc <daniel.nikolay@rwth-aachen.de> * fix: update frontend frameworks (#61) * fix: optimize settings (#77) * Create PR for #76 * chore: optimize object creation * chore: app load by default Co-authored-by: sbanoeon <sbanoeon@users.noreply.github.com> Co-authored-by: Saira Bano <saira.bano@eonerc.rwth-aachen.de> * chore: remove automatic branch creation Use GitHub native feature for future branch and pull request creation * chore: add bootstrap icons as vendored * chore: first draft of tabs * WEB_URL missing in settings doc (#83) * Create PR for #82 * docs(fix): web_url to web_host Co-authored-by: sbanoeon <sbanoeon@users.noreply.github.com> Co-authored-by: Saira Bano <saira.bano@eonerc.rwth-aachen.de> * add users to project (#63) * Create PR for #62 * fix: static root for collection * chore: added users to project option * chore: enable owner change for server admin * fix: manual created superuser dose not have superuser access * fix: project admin should not see all projects * fix: owner field optional for non-server admins * fix: project admin limited update options Co-authored-by: sbanoeon <sbanoeon@users.noreply.github.com> Co-authored-by: Saira Bano <saira.bano@eonerc.rwth-aachen.de> Co-authored-by: JunsongDu <junsong.du@eonerc.rwth-aachen.de> Co-authored-by: Saira Bano <79838286+sbanoeon@users.noreply.github.com> * chore: service groups list view * feat: service group curl option * feat: redirection of service group tab * chore: change default value of boolean params * chore: disable editing resource and apikey * chore: fiware service exception from filip (#86) * chore: increase trigger interval (#88) * fix: both project and server admin in owner dropdown * chore: segregate loki and console logging * fix: add error handling * fix: delete related devices * chore: added oidc basic logging * chore: added entities logging * fix: fixed enabler type * 90 batch crud entities (#94) * chore: batch create entity from json * fix: used filip update function instead of post * chore: delete batch entities * chore: force delete entities * chore: include json icon * chore: include example in form * chore: exception handling in list view * fix: typo * feat: implement filter for service groups * chore: add logger for devices app * chore: add more info to logger * chore: disable other existing loggers * chore: delete session data after usage * 95 update contribution guidelines (#96) * docs: GitHub workflow for development * docs(fix): docker image * fix: addition according to review * fix: fixed consistency in subscription deletion (#102) * chore: made common utils function for session data * chore: basic logging for subscriptions app * chore: move group form to separate script and add tooltips * chore: move group form to separate script * chore: tiny adjustment * fix: checkbox bugs * chore: remove extra code * fix: explicit_attrs instead of explicitAttrs * chore: merged dev into current * feat: add logger for service group CRUD * chore: add error logger for devices app * chore: update roadmap * docs(fix): fixed typo * 107 include new user flags for user roles (#111) * chore: include maintainer flag for projects * chore: update option for maintainer on sidebar * chore: include widget * chore: include maintainers in project context * fix: fixed maintainer function * chore: handle other attribute types (#101) * chore: handle other attribute types * fix: move common entirety list text widget * chore: change attribute type to charfield in devices app * fix: error while merging * chore: update filip version * chore: disable type validator of get_attributes --------- Co-authored-by: JunsongDu <junsong.du@eonerc.rwth-aachen.de> * chore: change database environment variables BREAKING CHANGE: the env variable DATABASE_URL is now split into DATABASE_USER, DATABASE_PASSWORD, DATABASE_PORT, and DATABASE_HOST * chore: update docs * feat: show the device if duplicated * feat: batch deletion for devices and service groups * 84 user experience improvement (#100) * Add entirety logo to banner * Make website and logo optional in new project form * Add dropdown for entities in subscription form * Add entirety logo to banner * Make website and logo optional in new project form * Add dropdown for entities in subscription form * Add multi value field for entity/type pattern and id * Move subscription create/update button to bottom of form Change style to match entity create button * chore: re-arrange logo * chore: use blank image as default * chore: change tooltip to dynamic --------- Co-authored-by: Saira Bano <saira.bano@eonerc.rwth-aachen.de> Co-authored-by: JunsongDu <junsong.du@eonerc.rwth-aachen.de> * chore: single button for searching entities * fix: it should be search-entity in the html * chore: default search by id * chore: create default super user * chore: temporary change to solve the pydantic issue * chore: create image for this branch * chore: test image * chore: change image build back to main and development * 99 display metadata in entities app (#118) * chore: handle metadata in entity form * chore: tooltip for metadata * fix: fixed tooltip key title in javascript * fix: change method to load json from string and not a file * fix: error handling for metadata in Create and Update * chore: adaption of ROADMAP.md * feat: allow batch devices creation from json payload * feat: add length limitation for columns in table * chore: limit column length for entity and service group * fix: json.js conflict * chore: update devices app roadmap --------- Co-authored-by: Daniel Nikolay <88837885+dnikolay-ebc@users.noreply.github.com> Co-authored-by: dnikolay-ebc <dnikolay-ebc@users.noreply.github.com> Co-authored-by: dnikolay-ebc <daniel.nikolay@rwth-aachen.de> Co-authored-by: djs0109 <djs0109@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: sbanoeon <sbanoeon@users.noreply.github.com> Co-authored-by: Saira Bano <saira.bano@eonerc.rwth-aachen.de> Co-authored-by: Saira Bano <79838286+sbanoeon@users.noreply.github.com> Co-authored-by: richardmarston <mail@richardmarston.net> Co-authored-by: Sebastian Blechmann <sebastian.blechmann@eonerc.rwth-aachen.de>
N5GEH · Sep 5, 2023 · a4d783b · a4d783b
1 parent 5e82e64
commit a4d783b
Show file tree

Hide file tree

Showing 111 changed files with 3,176 additions and 1,022 deletions.
diff --git a/.gitattributes b/.gitattributes
@@ -1,5 +1,6 @@
 # Ignore in language detection
 app/Entirety/static/bootstrap/** linguist-vendored
+app/Entirety/static/bootstrap-icons/** linguist-vendored
 app/Entirety/static/htmx/** linguist-vendored
 
 

diff --git a/.github/issue-branch.yml b/.github/issue-branch.yml
diff --git a/.github/workflows/create-image.yml b/.github/workflows/create-image.yml
@@ -4,6 +4,7 @@ on:
   push:
     branches:
       - development
+      - main
   release:
     types:
       - created
@@ -23,11 +24,11 @@ jobs:
         uses: actions/checkout@v3
 
       - name: Log in to the Container registry
-        uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
+        uses: docker/login-action@v2
         with:
           registry: ${{ env.REGISTRY }}
           username: ${{ github.actor }}
-          password: ${{ secrets.CI_TOKEN }}
+          password: ${{ secrets.GITHUB_TOKEN }}
 
       - name: Extract metadata (tags, labels) for Docker
         id: meta

diff --git a/.github/workflows/issue-tracker.yml b/.github/workflows/issue-tracker.yml
diff --git a/.gitignore b/.gitignore
@@ -219,6 +219,7 @@ fabric.properties
 .idea
 
 app/Entirety/static/CACHE/
+app/Entirety/assets/CACHE/
 app/Entirety/**/migrations/*
 !app/Entirety/**/migrations/__init__.py
 

diff --git a/README.md b/README.md
@@ -1,17 +1,26 @@
 # n5geh.tools.entirety
 
-[![semantic-release](https://github.com/N5GEH/n5geh.tools.entirety/actions/workflows/semantic-release.yml/badge.svg)](https://github.com/N5GEH/n5geh.tools.entirety/actions/workflows/semantic-release.yml)
-[![issue-tracker](https://github.com/N5GEH/n5geh.tools.entirety/actions/workflows/issue-tracker.yml/badge.svg)](https://github.com/N5GEH/n5geh.tools.entirety/actions/workflows/issue-tracker.yml)
-
 [![Conventional Commits](https://img.shields.io/badge/Conventional%20Commits-1.0.0-yellow.svg)](https://conventionalcommits.org)
+[![semantic-release](https://github.com/N5GEH/n5geh.tools.entirety/actions/workflows/semantic-release.yml/badge.svg)](https://github.com/N5GEH/n5geh.tools.entirety/actions/workflows/semantic-release.yml)
 
 ## Built With
 
-- Django 4.0.5
-- Bootstrap 5.2.0-beta1
-- htmx 1.7.0
+- Django 4.1
+- Bootstrap 5.2
+- htmx 1.8.2
+
+## Roadmap
+Have a look at our [roadmap](./docs/ROADMAP.md) to see what features we plan to work on in the short and long run. We kindly invite you to participate in [discussions](https://github.com/N5GEH/n5geh.tools.entirety/discussions) about possible features as well.
+## Deployment
+
+To deploy the application please refer to
+our [deployment guide](https://github.com/N5GEH/n5geh.tutorials.entirety_step_by_step)
+
+## Contributing
+
+See the [contributing guide](./docs/CONTRIBUTING.md) for detailed instructions on how to get started with our project.
 
-## Getting Started
+## Development
 
 ### Prerequisites
 
@@ -41,7 +50,7 @@ pre-commit
 Migrate Database
 
 ```bash
-  python manage.py makemigrations projects users examples
+  python manage.py makemigrations
   python manage.py migrate
 ```
 
@@ -51,30 +60,38 @@ Starting the Django server:
   python manage.py runserver
 ```
 
-## Contributing
-
-See the [contributing guide](./docs/CONTRIBUTING.md) for detailed instructions on how to get started with our project.
-
-## Development
-
 To run the application in your development setup you'll need to
 provide following settings in your env file.
 
 ### Required
 
 * [DJANGO_SECRET_KEY](./docs/SETTINGS.md#django_secret_key)
+* [CB_URL](./docs/SETTINGS.md#cb_url)
+* [IOTA_URL](./docs/SETTINGS.md#iota_url)
+* [QL_URL](./docs/SETTINGS.md#ql_url)
+* [DATABASE_USER](./docs/SETTINGS.md#DATABASE_USER)
+* [DATABASE_PASSWORD](./docs/SETTINGS.md#DATABASE_PASSWORD)
+* [DATABASE_HOST](./docs/SETTINGS.md#DATABASE_HOST)
+* [DATABASE_PORT](./docs/SETTINGS.md#DATABASE_PORT)
+
+### Optional
+
+* [DJANGO_DEBUG](./docs/SETTINGS.md#django_debug)
+* [COMPRESS_ENABLED](./docs/SETTINGS.md#compress_enabled)
+* [ENTITIES_LOAD](./docs/SETTINGS.md#entities_load)
+* [DEVICES_LOAD](./docs/SETTINGS.md#devices_load)
+* [NOTIFICATIONS_LOAD](./docs/SETTINGS.md#django_secret_key)
+
+### OIDC
+
+* [LOCAL_AUTH](./docs/SETTINGS.md#local_auth)
 * [OIDC_OP_AUTHORIZATION_ENDPOINT](./docs/SETTINGS.md#oidc_op_authorization_endpoint)
 * [OIDC_OP_JWKS_ENDPOINT](./docs/SETTINGS.md#oidc_op_jwks_endpoint)
 * [OIDC_OP_TOKEN_ENDPOINT](./docs/SETTINGS.md#oidc_op_token_endpoint)
 * [OIDC_OP_USER_ENDPOINT](./docs/SETTINGS.md#oidc_op_user_endpoint)
 * [OIDC_RP_CLIENT_ID](./docs/SETTINGS.md#oidc_rp_client_id)
 * [OIDC_RP_CLIENT_SECRET](./docs/SETTINGS.md#oidc_rp_client_secret)
 
-### Optional
-
-* [DJANGO_DEBUG](./docs/SETTINGS.md#django_debug)
-* [COMPRESS_ENABLED](./docs/SETTINGS.md#compress_enabled)
-
 For a full list of settings see [settings](./docs/SETTINGS.md).
 
 ## Changelog
@@ -89,7 +106,7 @@ See [changelog](./docs/CHANGELOG.md) for detailed overview of changes.
 
 [@djs0109](https://github.com/djs0109)
 
-[@dnikolay-ebc](https://github.com/dnikolay-ebc)
+[@mwr-ebc](https://github.com/mwr-ebc)
 
 ## License
 

diff --git a/app/Entirety/.env.EXAMPLE b/app/Entirety/.env.EXAMPLE
@@ -3,17 +3,21 @@ ENTITIES_LOAD=True
 DEVICES_LOAD=True
 NOTIFICATIONS_LOAD=True
 
+# Database
+DATABASE_PORT=5432
+DATABASE_USER=postgres
+DATABASE_PASSWORD=postgrespw
+DATABASE_HOST=localhost
+
 # Django
 DJANGO_SECRET_KEY=
-DJANGO_DEBUG=False
-ALLOWED_HOSTS=["localhost","127.0.0.1"]
-LANGUAGE_CODE=en-us
-TIME_ZONE=Europe/Berlin
+DJANGO_DEBUG=True #Development only!!
 
 # JS/SCSS compression
 COMPRESS_ENABLED=True
 
 # LOGGING
+LOKI_ENABLE=False
 LOKI_HOST=localhost
 LOKI_LEVEL=DEBUG
 LOKI_PORT=3100
@@ -22,14 +26,7 @@ LOKI_PROTOCOL=http
 LOKI_SRC_HOST=entirety
 LOKI_TIMEZONE=Europe/Berlin
 
-# Static files location
-STATIC_ROOT=/var/entirety/static/
-MEDIA_ROOT=/var/entirety/media/
-
 # OIDC
-LOGIN_URL=/oidc/authenticate
-LOGIN_REDIRECT_URL=/oidc/callback/
-LOGOUT_REDIRECT_URL=/
 OIDC_OP_AUTHORIZATION_ENDPOINT=
 OIDC_OP_JWKS_ENDPOINT=
 OIDC_OP_TOKEN_ENDPOINT=
@@ -45,7 +42,4 @@ OIDC_TOKEN_ROLE_FIELD=roles
 # FIWARE
 CB_URL=http://localhost:1026
 IOTA_URL="http://localhost:4041"
-MQTT_BASE_TOPIC=/Entirety
 QL_URL="http://localhost:8668"
-
-WEB_URL=localhost
diff --git a/app/Entirety/cache/info.txt b/app/Entirety/cache/info.txt
@@ -1,2 +1,2 @@
 DO NOT REMOVE!
-USED FOR DEVELOPMENT PURPOSES
+USED TO STORE COMPRESSED CSS FOR DEVELOPMENT PURPOSES
diff --git a/app/Entirety/devices/forms/__init__.py b/app/Entirety/devices/forms/__init__.py
@@ -0,0 +1,2 @@
+from .devices import *
+from .servicegroups import *
diff --git a/app/Entirety/devices/forms.py → app/Entirety/devices/forms/devices.py b/app/Entirety/devices/forms.py → app/Entirety/devices/forms/devices.py
@@ -2,8 +2,8 @@
 from crispy_forms.layout import Layout, Div, HTML
 from django import forms
 from django.forms import formset_factory
-from filip.models.ngsi_v2.iot import DataType
-
+from filip.models.ngsi_v2.iot import DataType, ServiceGroup
+from entirety.widgets import ListTextWidget
 
 ATTRIBUTES_TYPE = [
     DataType.NUMBER.value,
@@ -39,8 +39,8 @@ class DeviceBasic(forms.Form):
                 "data-bs-toggle": "tooltip",
                 # "data-bs-placement": "top ",
                 "title": "ID of the entity representing the device in the Context Broker, "
-                "e.g. urn:ngsi-ld:TemperatureSensor:001. Combination of ID and "
-                "Type must be unique.",
+                         "e.g. urn:ngsi-ld:TemperatureSensor:001. Combination of ID and "
+                         "Type must be unique.",
             }
         ),
     )
@@ -52,7 +52,7 @@ class DeviceBasic(forms.Form):
                 "data-bs-toggle": "tooltip",
                 # "data-bs-placement": "top ",
                 "title": "Type of the entity in the Context Broker. Combination of ID and "
-                "Type must be unique.",
+                         "Type must be unique.",
             }
         ),
     )
@@ -72,12 +72,24 @@ class DeviceAttributes(forms.Form):
                 "data-bs-toggle": "tooltip",
                 # "data-bs-placement": "top ",
                 "title": "Unique name to identify this attribute within the context entity in the "
-                "Context Broker",
+                         "Context Broker",
             }
         ),
     )
-    type_choices = tuple([(f"{t}", t) for i, t in enumerate(ATTRIBUTES_TYPE)])
-    type = forms.ChoiceField(label="Type", required=True, choices=type_choices)
+    type = forms.CharField(
+        required=True,
+        max_length=256,
+        label="Attribute Type",
+        widget=ListTextWidget(
+            data_list=ATTRIBUTES_TYPE,
+            name="attr-type-list",
+            attrs={
+                "data-bs-toggle": "tooltip",
+                "data-bs-placement": "top",
+                "title": "Type of the context attribute.",
+            },
+        ),
+    )
     object_id = forms.CharField(
         label="Object ID",
         required=False,
@@ -86,7 +98,7 @@ class DeviceAttributes(forms.Form):
                 "data-bs-toggle": "tooltip",
                 # "data-bs-placement": "top ",
                 "title": " (optional) Name of the attribute as coming from the device ("
-                "incoming southbound traffic).",
+                         "incoming southbound traffic).",
             }
         ),
     )
@@ -139,5 +151,48 @@ def __init__(self, *args, **kwargs):
         )
 
 
+class DeviceBatchForm(forms.Form):
+    device_json = forms.JSONField(
+        required=True,
+        initial={
+            "devices":
+                [
+                    {
+                        "protocol": "IoTA-JSON",
+                        "transport": "MQTT",
+                        "explicitAttrs": False,
+                        "device_id": "device:001",
+                        "entity_name": "urn:ngsi-ld:Example:001",
+                        "entity_type": "Example",
+                        "attributes": [
+                            {
+                                "name": "attr1",
+                                "type": "Number",
+                                "object_id": "a1"
+                            }
+                        ],
+                        "ngsiVersion": "v2"
+                    },
+                    {
+                        "protocol": "IoTA-JSON",
+                        "transport": "MQTT",
+                        "explicitAttrs": False,
+                        "device_id": "device:002",
+                        "entity_name": "urn:ngsi-ld:Example:002",
+                        "entity_type": "Example",
+                        "attributes": [
+                            {
+                                "name": "attr2",
+                                "type": "Number",
+                                "object_id": "a2"
+                            }
+                        ],
+                        "ngsiVersion": "v2"
+                    },
+                ]
+        }
+    )
+
+
 Attributes = formset_factory(DeviceAttributes, extra=0)
 Commands = formset_factory(DeviceCommands, extra=0)
diff --git a/app/Entirety/devices/forms/servicegroups.py b/app/Entirety/devices/forms/servicegroups.py
@@ -0,0 +1,44 @@
+from crispy_forms.helper import FormHelper
+from django import forms
+
+
+class ServiceGroupBasic(forms.Form):
+    resource = forms.CharField(
+        label="Resource",
+        required=True,
+        widget=forms.TextInput(
+            attrs={
+                "data-bs-toggle": "tooltip",
+                "title": "A string representing the southbound resource "
+                         "that will be used to provision a device, e.g. /iot/json",
+            }
+        ),
+    )
+    apikey = forms.CharField(
+        label="API Key",
+        required=True,
+        widget=forms.TextInput(
+            attrs={
+                "data-bs-toggle": "tooltip",
+                "title": "API key that authenticates the devices of a service group",
+            }
+        ),
+    )
+    entity_type = forms.CharField(
+        label="Entity Type",
+        max_length=256,
+        required=False,
+        widget=forms.TextInput(
+            attrs={
+                "data-bs-toggle": "tooltip",
+                "title": "Type of the entity in the Context Broker",
+            }
+        ),
+    )
+    explicit_attrs = forms.BooleanField(label="Explicit Attributes", required=False, initial=False)
+    autoprovision = forms.BooleanField(label="Auto Provision", required=False, initial=True)
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.helper = FormHelper(self)
+        self.helper.form_tag = False
diff --git a/app/Entirety/devices/models.py b/app/Entirety/devices/models.py
@@ -1,4 +1,9 @@
-from django.db import models
+from filip.models.ngsi_v2.iot import ServiceGroup
 
 
-# Create your models here.
+class _ServiceGroup:
+    def __init__(self, service_group: ServiceGroup):
+        self.resource = service_group.resource
+        self.apikey = service_group.apikey
+        self.entity_type = service_group.entity_type
+        self.id = None