Workflow licenses #2973

aatkin · 2022-06-30T11:10:33Z

relates #2158

Add support for creating/editing licenses into workflows. Workflow licenses have existed before in some form ( workflow_licenses table), so this needed some migration work. Application model enrichment is updated to use workflow licenses.

some larger changes:

removed workflow_licenses table and related queries
migrated data to workflow table json column

api changes:

remove licenses from workflow (it is now under [:workflow :licenses])

feature:

update workflow DB layer to serialize/deserialize licenses
update workflow service layer to join license dependencies
create UI to allow administration to create and edit workflows to include licenses
update workflow API & related commands to accept and validate licenses
update application model to enrich workflow licenses
update dependencies to compute workflow licenses

Checklist for author

Remove items that aren't applicable, check items that are done.

Reviewability

Link to issue
Consider adding screenshots for ease of review

Backwards compatibility

API is backwards compatible or completely new
Feature appears correctly in PDF print

Documentation

Update changelog if necessary
API is documented and shows up in Swagger UI

Testing

Complex logic is unit tested
Valuable features are integration / browser / acceptance tested automatically

Follow-up

New tasks are created for pending or remaining tasks

Screenshots

edit workflow

Previously workflow licenses have existed (as legacy feature) in workflow_licenses table as id,wfid,licid columns. As part of effort to unify code conventions the workflow_licenses table is due for removal, and the data needs to be migrated into workflow table under workflowBody json column.

Migration to remove workflow_licenses table is done separately from data migration for simpler SQL. Removing table would break integration tests awfully if down migration was missing to restore the removed table.

Fetching catalogue item and workflow licenses was previously handled by a single SQL query. This is now separated, so rems.db.applications/get-catalogue-item-licenses relies on rems.db.workflow/get-workflow to enrich licenses.

Macroz

Looks good to me, sorry for the review taking a while.

Also based on looking at the UI, I think there's a couple small points to address here or if you can create a new task:

Could we disable editing of the licenses for workflows? We don't allow the editing of forms, because this could affect existing applications. A better version would allow editing of these if the workflow is not in use yet (not in use in any application or so).
The margins / spacing in these Licenses looks a bit off. Perhaps the same with the Forms. There is extra margin compared to e.g. Handlers, and also slightly more in bottom than top?

Macroz · 2022-08-29T12:35:04Z

src/clj/rems/api/services/workflow.clj

+      (let [id (workflow/create-workflow! (-> cmd
+                                              (update :licenses #(map :license/id %))))]


Something that we have perhaps not specified yet in "layer architecture" is how the ids of various things should be passed on. Could be nice to have {:license/id 123} format at this level but it could also work to have the 123 for the DB layer and keep the map for API and service layers.

Macroz · 2022-08-29T12:42:01Z

src/clj/rems/api/workflows.clj


 (s/defschema EditWorkflowCommand
  {:id s/Int
   (s/optional-key :organization) schema-base/OrganizationId
   ;; type can't change
   (s/optional-key :title) s/Str
-   (s/optional-key :handlers) [schema-base/UserId]})
+   (s/optional-key :handlers) [schema-base/UserId]
+   (s/optional-key :licenses) [schema-base/LicenseId]})


We should probably remove the option of setting licenses in edit as it could affect existing applications. It's the same with forms and there we don't allow editing.

Macroz · 2022-08-29T12:46:18Z

src/clj/rems/db/licenses.clj

+(defn license-exists? [id]
+  (some? (db/get-license {:id id})))


This could potentially be very expensive (it's a query per license). I'm sure we have some other these kinds of checks around. We could add a task to go over them and see what approach is best. Could be for example cached.

Macroz · 2022-08-29T15:47:25Z

resources/migrations/20220711142631-remove-workflow-licenses-table.down.sql

@@ -0,0 +1,7 @@
+CREATE TABLE workflow_licenses (


Should this be IF NOT EXISTS? 🤔

Macroz · 2022-08-30T10:38:10Z

src/cljc/rems/common/util.cljc

@@ -608,3 +608,12 @@
    (is (= "id_123-element" (escape-element-id "123-element")))
    (is (= "id__123-element" (escape-element-id " 123-element")))))

+(defn keep-keys [f coll]


A docstring would be good for a utility fn?

Macroz · 2022-08-30T10:39:44Z

src/clj/rems/db/applications.clj

+                              workflow/get-workflow
+                              (get-in [:workflow :licenses]))]
+    (->> (licenses/get-licenses {:items [catalogue-item-id]})
+         (map #(keep-keys {:id :license/id} %))


Since we do this and (map #(select-keys ...)) quite much, there might be a place for some helper that contains the map in it too. The regular keep works on sequences so should this keep-keys too? And it could do the map?

Macroz · 2022-08-30T16:40:46Z

src/cljs/rems/administration/create_workflow.cljs

-                            :handlers (get workflow :handlers)})))
+                            :handlers (get workflow :handlers)
+                            :licenses (->> (:licenses workflow)
+                                           (map #(replace-key % :license/id :id)))})))


Maybe here also, something like replace-keys that maps and contains a map to potentially rename multiple keys?

src/clj/rems/middleware.clj

…actor keep-keys

aatkin added 5 commits July 14, 2022 14:23

feat: remove workflow_licenses table migration

1c20d51

Migration to remove workflow_licenses table is done separately from data migration for simpler SQL. Removing table would break integration tests awfully if down migration was missing to restore the removed table.

feat: enrich application model with workflow licenses

1fe38eb

Fetching catalogue item and workflow licenses was previously handled by a single SQL query. This is now separated, so rems.db.applications/get-catalogue-item-licenses relies on rems.db.workflow/get-workflow to enrich licenses.

feat: UI to create/edit workflow licenses

7907712

refactor: minor renamings

d6577e6

aatkin force-pushed the 2158/workflow-licenses branch from 9cd64fd to d6577e6 Compare July 14, 2022 11:43

aatkin added 3 commits July 15, 2022 16:33

refactor: revert and reorganize

0074023

refactor: revert and reorganize part 2

01416d9

refactor: render read-only wf licenses like forms

b681b20

aatkin marked this pull request as ready for review July 18, 2022 15:31

aatkin changed the title ~~WIP workflow licenses~~ Workflow licenses Jul 18, 2022

aatkin added 3 commits July 19, 2022 14:52

chore: fix browser test

2552fb6

chore: fix browser test part 2

71df995

doc: update CHANGELOG

262551e

aatkin force-pushed the 2158/workflow-licenses branch from 038c595 to 262551e Compare July 19, 2022 11:53

Macroz approved these changes Sep 5, 2022

View reviewed changes

aatkin added 14 commits September 6, 2022 15:05

Merge remote-tracking branch 'origin/master' into 2158/workflow-licenses

cabcd36

feat: disable editing workflow licenses

4412120

refactor: remove licenses from edit workflow, tweak licenses rendering

f999d23

refactor: add if not exists to workflow_licenses down migration

1134926

refactor: update fe tests

fc8a298

refactor: remove rems.common.util/distinct-by in favor of medley, ref…

b053b9c

…actor keep-keys

refactor: add missing distinct-by import

a5f2cd2

refactor: browser-test edit workflow cannot edit licenses

3b1e514

refactor: remove license change from edit workflow tests

e240ffa

refactor: update edit-workflow test

52b5296

refactor: fix typo in test

148f3c8

Merge remote-tracking branch 'origin/master' into 2158/workflow-licenses

91e041b

Merge remote-tracking branch 'origin/master' into 2158/workflow-licenses

9f64e04

chore: fix edit workflow test

c7a46ac

Macroz merged commit a384f86 into master Sep 9, 2022

Macroz deleted the 2158/workflow-licenses branch September 9, 2022 14:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Workflow licenses #2973

Workflow licenses #2973

aatkin commented Jun 30, 2022 •

edited

Loading

Macroz left a comment

Macroz Aug 29, 2022

Macroz Aug 29, 2022

Macroz Aug 29, 2022

Macroz Aug 29, 2022

Macroz Aug 30, 2022

Macroz Aug 30, 2022

Macroz Aug 30, 2022

		(let [id (workflow/create-workflow! (-> cmd
		(update :licenses #(map :license/id %))))]

		(defn license-exists? [id]
		(some? (db/get-license {:id id})))

Workflow licenses #2973

Workflow licenses #2973

Conversation

aatkin commented Jun 30, 2022 • edited Loading

Checklist for author

Reviewability

Backwards compatibility

Documentation

Testing

Follow-up

Screenshots

Macroz left a comment

Choose a reason for hiding this comment

Macroz Aug 29, 2022

Choose a reason for hiding this comment

Macroz Aug 29, 2022

Choose a reason for hiding this comment

Macroz Aug 29, 2022

Choose a reason for hiding this comment

Macroz Aug 29, 2022

Choose a reason for hiding this comment

Macroz Aug 30, 2022

Choose a reason for hiding this comment

Macroz Aug 30, 2022

Choose a reason for hiding this comment

Macroz Aug 30, 2022

Choose a reason for hiding this comment

aatkin commented Jun 30, 2022 •

edited

Loading