Merge remote-tracking branch 'origin/production' into issue-3905

specify · Sep 29, 2023 · 9c156b2 · 9c156b2
2 parents 077f015 + 987ecb8
commit 9c156b2
Show file tree

Hide file tree

Showing 98 changed files with 3,033 additions and 1,218 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -10,20 +10,59 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 
+## [7.9.0](https://github.com/specify/specify7/compare/v7.8.13...v7.9.0) (25 September 2023)
 
-## Unreleased
-
-
-
-Coming in the next few months:
-
-- [Statistics page](https://discourse.specifysoftware.org/t/statistics-panel-for-specify-7/828)
+Specify 7.9 is a major update, introducing the new **Statistics Page**, **Agent Merging**, **Basic Query View**, and numerous behind the scenes improvements and bug fixes. [Learn more](https://discourse.specifysoftware.org/t/specify-7-9-release-announcement/1308).
 
-- [Duplicate record merging tool](https://discourse.specifysoftware.org/t/record-merging-in-specify-7/939/9)
+### Added
+- [Statistics page](https://discourse.specifysoftware.org/t/statistics-panel-for-specify-7/828) ([#501](https://github.com/specify/specify7/issues/501), [#3760](https://github.com/specify/specify7/issues/3760), [#3662](https://github.com/specify/specify7/issues/3662), [#3687](https://github.com/specify/specify7/issues/3687), [#3722](https://github.com/specify/specify7/issues/3722), [#3670](https://github.com/specify/specify7/issues/3670) *– Requested by CSIRO, CSIC, NHMD, SDSU, The University of Michigan, TERN, KU, Muséum d'histoire naturelle Genève, Cleveland Museum, and many others*)
+- [Agent merging](https://discourse.specifysoftware.org/t/record-merging-in-specify-7/939/9) ([#3864](https://github.com/specify/specify7/pull/3864), [#3887](https://github.com/specify/specify7/issues/3887), [#3832](https://github.com/specify/specify7/pull/3832), [#3818](https://github.com/specify/specify7/pull/3818), [#3894](https://github.com/specify/specify7/pull/3894), [#3855](https://github.com/specify/specify7/pull/3855), [#3846](https://github.com/specify/specify7/pull/3846), [#3822](https://github.com/specify/specify7/issues/3822), [#3818](https://github.com/specify/specify7/pull/3818), [#3842](https://github.com/specify/specify7/pull/3842), [#3838](https://github.com/specify/specify7/pull/3838), [#3825](https://github.com/specify/specify7/pull/3825), [#3809](https://github.com/specify/specify7/pull/3809), [#3474](https://github.com/specify/specify7/pull/3474) *– Requested by RBGE, AAFC-AAC, CSIRO, CSIC, The University of Michigan, and many others*)
+- Simple query interface ([#2479](https://github.com/specify/specify7/issues/2479) *– Requested by Muséum d'histoire naturelle Genève*)
+- The field mapper can now be hidden in query dialogs ([#3745](https://github.com/specify/specify7/issues/3745))
+- The Specify logo now reflects the custom color chosen in User Preferences ([#2210](https://github.com/specify/specify7/issues/2210))
+- Dismissible errors are now shown as 'toasts', allowing the user to dismiss warnings ([#2957](https://github.com/specify/specify7/issues/2957))
+- Major improvements and refactoring of all business rules ([#2924](https://github.com/specify/specify7/issues/2924))
+	- Implemented a uniqueness rule system that is respected by the frontend and backend
+	- Created a new `type` for Uniqueness Rules
+	- Allow users to safely dismiss business rule errors
+	- Created and improved business rule automatic tests
+- Added the ability for users to change the attachment preview mode between full resolution and thumbnails ([#3391](https://github.com/specify/specify7/issues/3391) *– Requested by New Mexico State University Herbarium*)
+- Subviews visualized as buttons will now have a highlighted ring around the button to indicate records exist ([#2326](https://github.com/specify/specify7/issues/2326) *– Requested by Muséum d'histoire naturelle Genève*)
+- Results can now be exported to CSV from the query dialog ([#3616](https://github.com/specify/specify7/issues/3616))
+- Miscellaneous localization improvements on behalf of Weblate ([#4003](https://github.com/specify/specify7/pull/4003))
+- The deletion blocker dialog has been overhauled, allowing uses to review a comprehensive list of records obstructing deletion ([80087a2](https://github.com/specify/specify7/commit/80087a29f4547c5c9fbd9ca37fd40309241af3fb))
 
-- [User Interface for editing Data Object Formatters, Forms, and other XML resources](https://github.com/specify/specify7/pull/2796)
+### Changed
+- MariaDB 10.11 is now the recommended DBMS and the `docker-compose` file has been updated accordingly ([#3743](https://github.com/specify/specify7/issues/3743))
+- The button to go to the top of a query has been changed from `Edit Sp Query` to an arrow icon
+- The navigation menu now is dark in both dark and light mode ([#3554](https://github.com/specify/specify7/issues/3554))
+- The Specify logo has been updated with a transparent background ([#2210](https://github.com/specify/specify7/issues/2210))
+- App resources can now be edited, saved, and created in full screen view ([#3768](https://github.com/specify/specify7/issues/3768))
+- Query lines now are scrollable horizontally on narrow windows ([#3945](https://github.com/specify/specify7/pull/3945))
+- Query item ordering buttons have been removed on narrow windows, instead, drag and drop is encouraged ([#3945](https://github.com/specify/specify7/pull/3945))
+- Query items in detailed view now are wrapped on narrow windows ([#3877](https://github.com/specify/specify7/pull/3877))
+- Atomic save is now used to check for integrity errors when merging records ([#3802](https://github.com/specify/specify7/issues/3802))
+- The 'Browse In Forms', 'Create Record Set', and 'GeoMap' buttons will no longer display if the query returns no results ([#3796](https://github.com/specify/specify7/issues/3796))
+- Improved instructions for running Django migrations ([#3626](https://github.com/specify/specify7/issues/3626))
+- Subviews are no longer centered vertically by default and customization options have been added ([#2108](https://github.com/specify/specify7/issues/2108))
+- Buttons are now right-aligned in toolbars instead of centered ([#3681](https://github.com/specify/specify7/issues/3681))
 
-- [And many more features](https://github.com/specify/specify7/pulls)
+### Fixed
+- Solved an issue with display formatters and aggregators that resulted in an incorrect count when calculating statistics ([db7f014](https://github.com/specify/specify7/commit/db7f0140c5d47cf714f76a1a8ea39f6024e79195))
+- The button that creates new record sets when working in a temporary set of records now shows 'Creating new record' on hover rather than 'newRecordSet'
+- Taxon records now will be marked as accepted if there is no accepted taxon given when saved ([2c2faa9e9](https://github.com/specify/specify7/commit/2c2faa9e9))
+- Fixed Loan Return Preparation counts being calculated incorrectly ([#3981](https://github.com/specify/specify7/issues/3981) *– Reported by CSIRO*)
+- Business rule automatic tests have been rewritten to ensure the order of operations are correct ([#3792](https://github.com/specify/specify7/pull/3792))
+- The `uniqueIdentifier` field is now globally unique in the Collection Object, Collecting Event, and Locality tables.
+- Proper scoping is now used for pick lists defined on a table or field from a table ([#3901](https://github.com/specify/specify7/issues/3901) *– Reported by The University of Michigan*)
+- The `ordinal` field now is automatically set by a backend business rule if none is provided ([#3788](https://github.com/specify/specify7/issues/3788))
+- Added uniqueness constraints for Accession Agent on Repository Agreement ([#133](https://github.com/specify/specify7/issues/133)).
+- Resolved an issue where navigating between records in record sets displayed a warning indicating that the record was not saved. ([#3259](https://github.com/specify/specify7/issues/3259))
+- Fixed hidden locality fields being added when sorting columns via the query results table ([#3725](https://github.com/specify/specify7/issues/3725), [#3733](https://github.com/specify/specify7/issues/3733), [#3683](https://github.com/specify/specify7/issues/3683))
+- Button sizes in the WorkBench are now unified ([#3727](https://github.com/specify/specify7/pull/3727))
+- Fixed a bug encountered when querying just the (formatted) table record in some cases ([#3721](https://github.com/specify/specify7/issues/3721))
+- Fixed some cases when buttons are not centered ([b07c6d5](https://github.com/specify/specify7/commit/b07c6d5644ecde060c8a0a82c1c3ac8fe288f011))
+- Text in dark mode buttons no longer is too bright upon hover ([eac4e7e](https://github.com/specify/specify7/commit/eac4e7e589af48efe1c7b1563dd8ccf592fcb685))
 
 ## [7.8.13](https://github.com/specify/specify7/compare/v7.8.12...v7.8.13) (5 July 2023)
 

diff --git a/README.md b/README.md
@@ -366,14 +366,18 @@ contenttypes, and sessions) but does not need the corresponding tables
 to be added to the database. Running `make django_migrations` will
 apply only those migrations needed for Specify 7 to operate.
 
-### The Specify 7 worker
-
-Beginning with v7.6.0 the Specify WorkBench upload and validate
-operations are carried out by a separate worker process using a
-[Celery](https://docs.celeryproject.org/en/master/index.html) job
-queue with
-[Reddis](https://docs.celeryproject.org/en/master/getting-started/backends-and-brokers/redis.html)
-as the broker. The worker process can be started from the commandline
+### The Specify 7 Worker
+
+Starting from version `v7.6.0`, the Specify WorkBench utilizes this 
+dedicated worker process to handle the upload and validation operations. 
+
+Starting from version `v7.9.0`, the record merging functionality employs the worker to handle all record merging activities.
+
+This worker process utilizes [Celery](https://docs.celeryproject.org/en/master/index.html), a job queue 
+management system, with [Redis](https://docs.celeryproject.org/en/master/getting-started/backends-and-brokers/redis.html) 
+serving as the broker.
+
+The worker process can be started from the commandline
 by executing:
 
 ```shell

diff --git a/specifyweb/businessrules/attachment_rules.py b/specifyweb/businessrules/attachment_rules.py
@@ -42,7 +42,8 @@ def attachment_save(attachment):
 @orm_signal_handler('post_delete', 'Attachment')
 def attachment_deletion(attachment):
     from specifyweb.attachment_gw.views import delete_attachment_file
-    delete_attachment_file(attachment.attachmentlocation)
+    if attachment.attachmentlocation is not None:
+        delete_attachment_file(attachment.attachmentlocation)
 
 def get_attachee(jointable_inst):
     main_table_name = JOINTABLE_NAME_RE.match(jointable_inst.__class__.__name__).group(1)

diff --git a/specifyweb/businessrules/tests/division.py b/specifyweb/businessrules/tests/division.py
@@ -8,7 +8,12 @@ def test_name_unique_in_institution(self):
             institution=self.institution,
             name='foobar')
 
-        with self.assertRaises(BusinessRuleException):
+        with self.assertRaises(BusinessRuleException) as business_rule_exception:
             models.Division.objects.create(
                 institution=self.institution,
                 name='foobar')
+
+        naive_insitution = business_rule_exception.exception.args[1].get('parentField', None)
+        self.assertEquals(naive_insitution is not None and naive_insitution.lower() == 'institution', True)
+
+
diff --git a/specifyweb/businessrules/uniqueness_rules.py b/specifyweb/businessrules/uniqueness_rules.py
@@ -57,8 +57,7 @@ def get_exception(conflicts, matchable, field_map):
                            }
 
         if len(parent_fields) > 0:
-            error_message += ' in {}'.format(join_with_and(parent_fields)) \
-                if len(parent_fields) > 0 else ''
+            error_message += ' in {}'.format(join_with_and(parent_fields))
             response.update({
                 "parentField": ','.join(parent_fields),
                 "parentData": serialize_multiple_django(matchable, field_map, parent_fields)
@@ -162,7 +161,7 @@ def resolve_child_parent(field, rule_instance):
         parent.append(rule_instance['field'])
         child.extend(rule_instance['otherFields'])
     else:
-        if rule_instance is not None and rule_instance != 'institution':
+        if rule_instance is not None:
             parent.append(rule_instance)
     child.sort()
     return tuple(child), tuple(parent)

diff --git a/specifyweb/context/views.py b/specifyweb/context/views.py
@@ -65,6 +65,14 @@ def set_users_collections_for_sp6(cursor, user, collectionids):
         # in collectionids. (I think the principal represents the
         # user's capacity wrt to a collection.)
 
+        # Unset all collections for the user if collectionids is empty
+        if not collectionids:
+            cursor.execute("delete from specifyuser_spprincipal where SpecifyUserID = %s", [user.id])
+            cursor.execute("delete from spprincipal_sppermission where spprincipalid in ("
+                           "select spprincipalid from specifyuser_spprincipal)")
+
+            return
+
         # First delete the mappings from the user to the principals.
         cursor.execute("delete specifyuser_spprincipal "
                        "from specifyuser_spprincipal "
@@ -106,6 +114,46 @@ class Sp6CollectionAccessPT(PermissionTarget):
     read = PermissionTargetAction()
     update = PermissionTargetAction()
 
+@openapi(schema={
+    "get": {
+        "responses" : {
+            "200" : {
+                "description": "Gets the list of collections a user has permissions for in Specify 6",
+                "content" : {
+                    "application/json": {
+                    "schema": {
+                        "type": "array",
+                        "items": {
+                            "type": "integer",
+                            "description": "The Collection IDs for which to a user has Specify 6 permission's for"
+                        }
+                    }
+                }
+                }
+            }
+        }
+    },
+    "put": {
+        "requestBody": {
+            "required": True,
+            "description": "Sets the Specify 6 permissions of a user for a list of collections",
+            "content": {
+                "application/json": {
+                    "schema": {
+                        "type": "array",
+                        "items": {
+                            "type": "integer",
+                            "description": "The Collection IDs for which to set a user's permussions in Specify 6 for"
+                        }
+                    }
+                }
+            }
+        },
+        "responses": {
+            "200": {"description": "Specify 6 permissions for user set."}
+        }
+    }
+})
 @login_maybe_required
 @require_http_methods(['GET', 'PUT'])
 @never_cache
@@ -318,7 +366,7 @@ def domain(request):
                 "description": "Flag to indicate that if the AppResource does not exist, return response with code 204 instead of 404"
             }
         ],
-    "get" : {
+    "get": {
         "responses": {
             "404": {
                 "description": "'name' parameter was not provided, or App Resource was not found"
@@ -418,7 +466,7 @@ def schema_localization(request):
                 "description": "Flag to indicate that if the view does not exist, return response with code 204 instead of 404"
             }
         ],
-    "get" : {
+    "get": {
         "responses": {
             "404": {
                 "description": "'name' parameter was not provided, or view was not found"

diff --git a/specifyweb/frontend/js_src/lib/components/AppResources/Editor.tsx b/specifyweb/frontend/js_src/lib/components/AppResources/Editor.tsx
@@ -234,14 +234,14 @@ export function AppResourceEditor({
         <AppResourcesTabs
           appResource={appResource}
           data={resourceData.data}
+          footer={footer}
           headerButtons={headerButtons}
           isFullScreen={isFullScreen}
           isReadOnly={isReadOnly}
           label={formatted}
           resource={resource}
           showValidationRef={showValidationRef}
           onChange={(data): void => setResourceData({ ...resourceData, data })}
-          footer={footer}
         />
       </Form>
       {isFullScreen ? null : footer}

diff --git a/specifyweb/frontend/js_src/lib/components/Atoms/Icons.tsx b/specifyweb/frontend/js_src/lib/components/Atoms/Icons.tsx
@@ -79,6 +79,9 @@ export const icons = {
   externalLink: <svg aria-hidden className={iconClassName} fill="currentColor" viewBox="0 0 20 20" xmlns="http://www.w3.org/2000/svg"><path d="M11 3a1 1 0 100 2h2.586l-6.293 6.293a1 1 0 101.414 1.414L15 6.414V9a1 1 0 102 0V4a1 1 0 00-1-1h-5z" /><path d="M5 5a2 2 0 00-2 2v8a2 2 0 002 2h8a2 2 0 002-2v-3a1 1 0 10-2 0v3H5V7h3a1 1 0 000-2H5z" /></svg>,
   eye: <svg aria-hidden className={iconClassName} fill="currentColor" viewBox="0 0 20 20" xmlns="http://www.w3.org/2000/svg"><path d="M10 12a2 2 0 100-4 2 2 0 000 4z" /><path clipRule="evenodd" d="M.458 10C1.732 5.943 5.522 3 10 3s8.268 2.943 9.542 7c-1.274 4.057-5.064 7-9.542 7S1.732 14.057.458 10zM14 10a4 4 0 11-8 0 4 4 0 018 0z" fillRule="evenodd" /></svg>,
   fingerPrint: <svg aria-hidden className={iconClassName} fill="currentColor" viewBox="0 0 20 20" xmlns="http://www.w3.org/2000/svg"><path clipRule="evenodd" d="M6.625 2.655A9 9 0 0119 11a1 1 0 11-2 0 7 7 0 00-9.625-6.492 1 1 0 11-.75-1.853zM4.662 4.959A1 1 0 014.75 6.37 6.97 6.97 0 003 11a1 1 0 11-2 0 8.97 8.97 0 012.25-5.953 1 1 0 011.412-.088z" fillRule="evenodd" /><path clipRule="evenodd" d="M5 11a5 5 0 1110 0 1 1 0 11-2 0 3 3 0 10-6 0c0 1.677-.345 3.276-.968 4.729a1 1 0 11-1.838-.789A9.964 9.964 0 005 11zm8.921 2.012a1 1 0 01.831 1.145 19.86 19.86 0 01-.545 2.436 1 1 0 11-1.92-.558c.207-.713.371-1.445.49-2.192a1 1 0 011.144-.83z" fillRule="evenodd" /><path clipRule="evenodd" d="M10 10a1 1 0 011 1c0 2.236-.46 4.368-1.29 6.304a1 1 0 01-1.838-.789A13.952 13.952 0 009 11a1 1 0 011-1z" fillRule="evenodd" /></svg>,
+  gallery:<svg aria-hidden className={iconClassName} fill="currentColor" viewBox="0 0 20 20" xmlns="http://www.w3.org/2000/svg">
+  <path clipRule="evenodd" d="M5 3a2 2 0 00-2 2v2a2 2 0 002 2h2a2 2 0 002-2V5a2 2 0 00-2-2H5zM5 11a2 2 0 00-2 2v2a2 2 0 002 2h2a2 2 0 002-2v-2a2 2 0 00-2-2H5zM11 5a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2h-2a2 2 0 01-2-2V5zM11 13a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2h-2a2 2 0 01-2-2v-2z" fillRule="evenodd" />
+</svg>,
   hashtag: <svg aria-hidden className={iconClassName} fill="currentColor" viewBox="0 0 20 20" xmlns="http://www.w3.org/2000/svg"><path clipRule="evenodd" d="M9.243 3.03a1 1 0 01.727 1.213L9.53 6h2.94l.56-2.243a1 1 0 111.94.486L14.53 6H17a1 1 0 110 2h-2.97l-1 4H15a1 1 0 110 2h-2.47l-.56 2.242a1 1 0 11-1.94-.485L10.47 14H7.53l-.56 2.242a1 1 0 11-1.94-.485L5.47 14H3a1 1 0 110-2h2.97l1-4H5a1 1 0 110-2h2.47l.56-2.243a1 1 0 011.213-.727zM9.03 8l-1 4h2.938l1-4H9.031z" fillRule="evenodd" /></svg>,
   // This icon is not from Heroicons. It was drawn by @grantfitzsimmons
   history: <svg aria-hidden className={iconClassName} fill="currentColor" viewBox="0 0 20 20" xmlns="http://www.w3.org/2000/svg"><path d="M17.043,6.984c-0.382-0.935-0.942-1.753-1.665-2.433c-0.767-0.722-1.572-1.249-2.459-1.611   c-0.911-0.391-1.94-0.597-2.979-0.597c-1.147,0-2.207,0.251-3.239,0.768C5.87,3.526,5.136,4.05,4.512,4.671V3.746   c0-0.47-0.397-0.867-0.867-0.867c-0.486,0-0.867,0.381-0.867,0.867v2.866c0,0.577,0.469,1.046,1.045,1.046h2.825   c0.51,0,0.909-0.399,0.909-0.908c0-0.471-0.355-0.826-0.826-0.826H5.752C6.191,5.5,6.688,5.131,7.258,4.802   c0.797-0.46,1.688-0.71,2.579-0.723c1.683-0.052,3.085,0.532,4.291,1.698c1.178,1.139,1.775,2.532,1.775,4.141   c0,1.729-0.555,3.125-1.696,4.267c-1.18,1.18-2.616,1.778-4.267,1.778c-1.647,0-3.016-0.568-4.181-1.733   c-0.983-1.022-1.542-2.259-1.661-3.685C4.041,10.084,3.677,9.75,3.233,9.75c-0.257,0-0.499,0.104-0.662,0.288   c-0.158,0.176-0.23,0.406-0.205,0.638c0.127,1.823,0.858,3.413,2.171,4.727c1.496,1.495,3.313,2.254,5.402,2.254   c0.977,0,1.947-0.18,2.983-0.558c0.977-0.443,1.805-1.001,2.46-1.655c0.74-0.739,1.282-1.547,1.659-2.469   c0.395-0.966,0.595-1.979,0.595-3.016C17.638,8.937,17.443,7.964,17.043,6.984z"/> <path d="M10.022,5.595H9.94c-0.426,0-0.785,0.359-0.785,0.784v3.333c0,0.411,0.158,0.796,0.445,1.084l2.345,2.303   c0.175,0.175,0.402,0.264,0.637,0.264c0.2,0,0.405-0.064,0.587-0.194c0.258-0.181,0.328-0.4,0.341-0.553   c0.021-0.251-0.09-0.519-0.31-0.737l-2.31-2.231V6.462C10.89,5.976,10.509,5.595,10.022,5.595z"/></svg>,