Merge pull request #1210 from open-zaak/release/1.7.0

Release 1.7.0

.bumpversion.cfg

@@ -1,7 +1,7 @@
 [bumpversion]
 commit = False
 tag = False
-current_version = 1.6.0
+current_version = 1.7.0
 parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(\.dev(?P<dev>\d+))?
 serialize = 
 	{major}.{minor}.{patch}.dev{dev}

CHANGELOG.rst

@@ -1,9 +1,87 @@
 Changelog
 =========
 
-1.7.0 (2022-0?-??)
+1.7.0 (2022-07-08)
 ------------------
 
+Open Zaak 1.7.0 is a rather big feature release.
+
+The biggest changes are:
+
+* Updated Zaken API from 1.0.3 to 1.1.2
+* Updated Catalogi API from 1.0.0 to 1.1.1
+* Admin UI improvements
+
+**New features**
+
+* [#1109 and #1157] Implemented Zaken API 1.1.2 - please check the upstream VNG API standards for
+  more information
+* [#1109] Implemented Catalogi API 1.1.1 - please check the upstream VNG API standards
+  for more information
+* [#1145] the log level is now configurable through environment variables
+* [#1105 and #1182] Improved performance of catalogus imports
+* [#510] allow filtering zaaktypen on geldigheid and publish status
+* [#970] improved the handling of selectielijst in zaaktypen/resultaattypen - the admin
+  now protects you better from making invalid configurations
+* [#1030] The selectielijst procestypes are now refreshed when the selectielijst-year
+  is changed and the selectielijstklasse choices for a resultaattype are now updated
+  if the zaaktype is changed or set
+* [#1085] the admin now runs more extensive validation on zaaktype publish to prevent
+  misconfiguration:
+
+    - checks that there is at least one roltype
+    - checks that there is at least one resultaattype
+    - checks that there are at least two status types (initial + closing)
+* [#1119] the Open Zaak version number is now displayed in the in admin footer
+* [#1183] updated EN -> NL translations
+
+**Bugfixes**
+
+* [#1130] added missing error documents
+* [#1107] aligned admin validation of resultaattype-archiefprocedure with API validation
+* [#979] Prevent cascading deletes when deleting a zaaktype, which would delete related
+  zaken before
+* [#983] allow concept zaaktype updates with published documenttypes
+* [#981] allow null for eindeGeldigheid in Catalogi API
+* [#992] run deelzaaktype validation for zaak.hoofdzaak.zaaktype
+* [#1023] fixed zaak list returning duplicated zaken
+* [#1080] fixed displaying authorization (specs) if there are no related objects
+  (zaaktype/documenttype/besluittype) yet
+* [#1081] Added test to confirm autorisaties are deleted when documenttypes are deleted
+* [#1169] Ensure the selectielijst procestype year is derived and stored when importing
+  zaaktypen
+* [#1042] Fixed a number of bypasses that allowed you to edit published zaaktypen
+* [#1108] Fixed crash while validating document archival status on Zaak create
+
+**Documentation**
+
+* Documented the API parity policy - there are now procedures for adding experimental
+  features to Open Zaak
+* [#1001] restructurd deployment documentation
+* Documented buildkit requirement in docker-compose install
+* Updated documentation for which API versions Open Zaak implements
+
+**Project maintenance**
+
+* [#1129] Fixed the failing api-test.nl build
+* [#1136 and #1207] Bump to the latest security releases of Django
+* [#1139] Refactor ADFS/AAD usage to generic OIDC library
+* Update to Python 3.9
+* Improved test isolation in CI build
+* Replace set_context with new context system DRF (ongoing work)
+* Replace raw requests usage with Service wrapper
+* Remove some duplicated/bad patterns in test code
+* Upgraded PyJWT dependency
+* Upgraded frontend dependencies for security issues
+* Removed the zds-client library mocking utility usage
+* Cleaned up requests mock usage to prevent real HTTP calls from being made
+* Refactored API spec mocking in tests to remove duplication and custom code
+* API spec references for data validation are now pinned to release tags rather than
+  commit hashes
+* Reduced docker build context and image size
+* Upgraded to gemma-zds-client 1.0.1
+* [#1099] Added ZGW OAS tests to CI pipeline
+
 .. warning::
 
    Manual intervention required for ADFS/AAD users.

README.en.md

@@ -6,7 +6,7 @@
 
 | | | |
 |-|-|-|
-| **Version:** | 1.6.0 |
+| **Version:** | 1.7.0 |
 | **Source:** | https://github.com/open-zaak/open-zaak |
 | **Keywords:** | zaken, zaakgericht werken, zaken-api, catalogi-api, besluiten-api, documenten-api |
 | **PythonVersion:** | 3.9 |

README.rst

@@ -7,7 +7,7 @@ Open Zaak
 
 .. _`Read this in English`: README.en.md
 
-:Version: 1.6.0
+:Version: 1.7.0
 :Source: https://github.com/open-zaak/open-zaak
 :Keywords: zaken, zaakgericht werken, zaken-api, catalogi-api, besluiten-api, documenten-api
 :PythonVersion: 3.9

deployment/kubernetes/apps.yml

@@ -14,7 +14,7 @@
   roles:
     - role: open_zaak_k8s
       vars:  # these override vars_files
-        openzaak_version: openzaak_version: '1.6.0'
+        openzaak_version: openzaak_version: '1.7.0'
         openzaak_db_host: "{{ db.privateIp }}"
         openzaak_db_port: "{{ db.port }}"
       tags:

deployment/single-server/open-zaak.yml

@@ -61,7 +61,7 @@
 
     - role: open_zaak_docker
       vars:
-        openzaak_version: '1.6.0'  # see https://hub.docker.com/r/openzaak/open-zaak/tags
+        openzaak_version: '1.7.0'  # see https://hub.docker.com/r/openzaak/open-zaak/tags
       tags:
         - replicas
 

docs/development/roadmap.rst

@@ -26,7 +26,43 @@ Feature implementation is the connection between the core group and the
 technical team. The core group may request features or consider the 
 requested (big) features from the Github issues.
 
-Current list is created from issue skimming on Github.
+**Upgrading to latest API standards**
+
+* Upgrade to Zaken API 1.3 (and 1.4 after July 2022)
+* Upgrade to Documenten API 1.1
+
+*Impact*
+
+- consumers can use the latest and greatest features
+- sizing: large
+
+**Improve domain-related robustness**
+
+(API) URLs/domains are not as stable as initially expected - especially when
+experimenting domains where the APIs are hosted tend to change (too) frequently.
+
+Additionally, when using NLX, the consumer URLs are managed on the consumer side and
+Open Zaak has no awareness of these URLs - implementing stable Open Zaak domain
+information without needing to rely on configuration/context is desired.
+
+*Impact*
+
+- Improve infrastructure flexibility / decoupling with infrastructure
+- Improve portability
+- Remove need for programmed data-migrations by providing controls to modify service
+  configuration in the admin
+- sizing: large
+
+**Notification robustness**
+
+Better tracking of notifications that are sent and received/missed + controls to
+guarantee at-least-once delivery.
+
+*Impact*
+
+- Better data integrity guarantees
+- Improve observability of complete solution
+- sizing: large
 
 **Post-installation configuration**
 
@@ -107,18 +143,6 @@ can stay closed by giving the option to use NLX.
 Documentation
 -------------
 
-**Provide documentation for the Ansible collection**
-
-The Ansible collection is really the heart of the deployment infrastructure/code and
-currently a magical black box. The collection itself should be documented, with target
-audiences: developers, devops, sysadmins.
-
-*Impact*
-
-- affects possible contributors
-- may provide a reference for service providers on how to correctly install OZ
-- sizing: medium
-
 **User manual: add selectielijst configuration**
 
 The user-interface for the configuration of selectielijsten must be included in the
@@ -185,29 +209,6 @@ service config and point to a particular service instead of storing the API root
 - affects complexity of codebase (makes it less complex/confusing)
 - sizing: small
 
-**Include newer Postgres versions in CI**
-
-Currently Open Zaak is tested against Postgres 10, while 11 and 12 are out. A test
-matrix for all versions of Open Zaak seems appropriate.
-
-*Impact*
-
-- demonstrates compatibility
-- explicit support gives more possible target deploy platforms
-- sizing: small
-
-**Prepare update to Django 3.x**
-
-Recently Django 3.1 was released, after 3.0. Open Zaak is on Django 2.2 (LTS). We plan
-to jump from LTS to LTS - Django 3.2 (LTS) should be released around April 2021.
-
-*Impact*
-
-- security & future security
-- affects: developers, contributors, users (API clients), municipalities with a deployed
-  version
-- sizing: medium
-
 **Structurally check security updates**
 
 Django publishes patch releases at the beginning of each month. Open Zaak should include

package.json

@@ -1,6 +1,6 @@
 {
   "name": "openzaak",
-  "version": "1.6.0",
+  "version": "1.7.0",
   "description": "Open Zaak",
   "main": "src/index.js",
   "directories": {

src/openzaak/components/autorisaties/tests/admin/test_autorisaties_admin.py

@@ -11,6 +11,7 @@
 from django.contrib.sites.models import Site
 from django.test import TransactionTestCase, override_settings, tag
 from django.urls import reverse
+from django.utils.translation import ugettext_lazy as _
 
 import requests_mock
 from django_webtest import WebTest
@@ -742,7 +743,7 @@ def test_add_autorisatie_zaaktypen_overlap(self):
         self.assertEqual(Autorisatie.objects.count(), 0)
         self.assertEqual(
             response.context_data["formset"]._non_form_errors[0],
-            "zaaktypen may not have overlapping scopes.",
+            _("{field} may not have overlapping scopes.").format(field="zaaktypen"),
         )
 
     def test_add_autorisatie_overlap_without_types(self):
@@ -764,7 +765,7 @@ def test_add_autorisatie_overlap_without_types(self):
         self.assertEqual(Autorisatie.objects.count(), 0)
         self.assertEqual(
             response.context_data["formset"]._non_form_errors[0],
-            "Scopes in ztc may not be duplicated.",
+            _("Scopes in {component} may not be duplicated.").format(component="ztc"),
         )
 
     @tag("gh-1080")

src/openzaak/components/catalogi/api/serializers/eigenschap.py

@@ -32,7 +32,7 @@ class EigenschapSerializer(
     NestedCreateMixin, NestedUpdateMixin, serializers.HyperlinkedModelSerializer
 ):
     specificatie = EigenschapSpecificatieSerializer(
-        source="specificatie_van_eigenschap", required=False,
+        source="specificatie_van_eigenschap"
     )
 
     class Meta:
@@ -49,3 +49,10 @@ class Meta:
                 queryset=Eigenschap.objects.all(), fields=["zaaktype", "naam"],
             ),
         ]
+
+    def _get_serializer_for_field(self, field, **kwargs):
+        # workaround for drf-writable-nested. it looks up the instance by PK, but we don't
+        # expose that in the serializer at all.
+        if field.field_name == "specificatie" and self.instance:
+            kwargs["instance"] = self.instance.specificatie_van_eigenschap
+        return super()._get_serializer_for_field(field, **kwargs)

src/openzaak/components/catalogi/api/serializers/zaaktype.py

@@ -180,6 +180,7 @@ class Meta:
             "begin_geldigheid": {"source": "datum_begin_geldigheid"},
             "einde_geldigheid": {"source": "datum_einde_geldigheid"},
             "concept": {"read_only": True},
+            "producten_of_diensten": {"required": True},
             "selectielijst_procestype": {
                 "validators": [
                     ResourceValidator("ProcesType", settings.REFERENTIELIJSTEN_API_SPEC)

src/openzaak/components/catalogi/openapi.yaml

@@ -4519,6 +4519,7 @@ components:
       required:
       - naam
       - definitie
+      - specificatie
       - zaaktype
       type: object
       properties:
@@ -5155,6 +5156,7 @@ components:
       - opschortingEnAanhoudingMogelijk
       - verlengingMogelijk
       - publicatieIndicatie
+      - productenOfDiensten
       - referentieproces
       - catalogus
       - besluittypen

src/openzaak/components/catalogi/resources.md

@@ -235,7 +235,7 @@ Uitleg bij mogelijke waarden:
 | publicatieIndicatie | Aanduiding of (het starten van) een ZAAK dit ZAAKTYPE gepubliceerd moet worden. | boolean | ja | C​R​U​D |
 | publicatietekst | De generieke tekst van de publicatie van ZAAKen van dit ZAAKTYPE. | string | nee | C​R​U​D |
 | verantwoordingsrelatie | De relatie tussen ZAAKen van dit ZAAKTYPE en de beleidsmatige en/of financiële verantwoording. | array | nee | C​R​U​D |
-| productenOfDiensten | Het product of de dienst die door ZAAKen van dit ZAAKTYPE wordt voortgebracht. | array | nee | C​R​U​D |
+| productenOfDiensten | Het product of de dienst die door ZAAKen van dit ZAAKTYPE wordt voortgebracht. | array | ja | C​R​U​D |
 | selectielijstProcestype | URL-referentie naar een vanuit archiveringsoptiek onderkende groep processen met dezelfde kenmerken (PROCESTYPE in de Selectielijst API). | string | nee | C​R​U​D |
 | catalogus | URL-referentie naar de CATALOGUS waartoe dit ZAAKTYPE behoort. | string | ja | C​R​U​D |
 | statustypen | URL-referenties naar de STATUSTYPEN die mogelijk zijn binnen dit ZAAKTYPE. | array | nee | ~~C~~​R​~~U~~​~~D~~ |

src/openzaak/components/catalogi/swagger2.0.json

@@ -5759,6 +5759,7 @@
             "required": [
                 "naam",
                 "definitie",
+                "specificatie",
                 "zaaktype"
             ],
             "type": "object",
@@ -6274,6 +6275,7 @@
                 "opschortingEnAanhoudingMogelijk",
                 "verlengingMogelijk",
                 "publicatieIndicatie",
+                "productenOfDiensten",
                 "referentieproces",
                 "catalogus",
                 "besluittypen",

src/openzaak/components/catalogi/tests/admin/test_admin_publish.py

@@ -225,7 +225,9 @@ def test_publish_zaaktype_related_to_concept_besluittype_fails(self, m):
 
         # Check that the error is shown on the page
         error_message = response.html.find("li", {"class": "error"})
-        self.assertIn("should be published", error_message.text)
+        self.assertIn(
+            _("All related resources should be published"), error_message.text
+        )
 
         # Verify that the publish button is still visible and enabled.
         publish_button = response.html.find("input", {"name": "_publish"})
@@ -287,7 +289,9 @@ def test_publish_zaaktype_related_to_concept_informatieobjecttype_fails(self, m)
 
         # Check that the error is shown on the page
         error_message = response.html.find("li", {"class": "error"})
-        self.assertIn("should be published", error_message.text)
+        self.assertIn(
+            _("All related resources should be published"), error_message.text
+        )
 
         # Verify that the publish button is still visible and enabled.
         publish_button = response.html.find("input", {"name": "_publish"})