Merge branch 'develop'

FirelyTeam · Aug 8, 2023 · 9e0482b · 9e0482b
2 parents 4d101d2 + e80ec8e
commit 9e0482b
Show file tree

Hide file tree

Showing 13 changed files with 205 additions and 19 deletions.
diff --git a/.readthedocs.yaml b/.readthedocs.yaml
@@ -0,0 +1,22 @@
+# .readthedocs.yaml
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Set the version of Python and other tools you might need
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.7"
+
+# Build documentation in the docs/ directory with Sphinx
+sphinx:
+  configuration: ./conf.py
+
+# We recommend specifying your dependencies to enable reproducible builds:
+# https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html
+# python:
+#   install:
+#   - requirements: docs/requirements.txt
diff --git a/_static/css/style.css b/_static/css/style.css
@@ -70,9 +70,8 @@ a.icon-home:hover {
 
 }
 .wy-nav-content {
-	max-width: 800px;
+	max-width: 95%;
 	width: 100%;
-
 	word-wrap: break-word;
 }
 

diff --git a/features_and_tools/custom_operations/erase.rst b/features_and_tools/custom_operations/erase.rst
@@ -7,7 +7,13 @@ Description
 -----------
 When Firely Server receives a DELETE request for a resource, it marks it as deleted in the database which makes it hidden from search results. However, the data is still present in the database. This approach is known as *soft deletion*. This comes in handy in scenarios when you want to recover accidentally deleted data. However, there are also scenarios when you *actually* want the data to be erased from the database. For that purpose, Firely Server provides the $erase operation.
 
-The $erase operation can be executed on a resource instance level and a resource version level.
+The `$erase` operation permanently deletes a single resource or one or more historical revisions of a resource from the database. It can be executed on a resource instance level and a resource version level.
+
+The `$purge` operation permanently deletes all resources within a patient compartment. The operation can be executed at a Patient instance level.
+
+.. note::
+
+  The `$purge` operation is not available when SQLite is used as data storage.
 
 Examples
 ^^^^^^^^
@@ -22,7 +28,13 @@ Use the following request to erase the specified version ``xyz`` and all the old
 
 ::
 
-  POST <base-url>/Patient/example/_history/xyz/$erase
+  POST <base-url>/Observation/example/_history/xyz/$erase
+
+Use the following request to erase resources within the patient compartment of the Patient with id 'example'. Note that AuditEvent and Provenance resources won't get erased by default. Additionally, resources that have been soft deleted before are not being purged. See the configuration details in the :ref:`Appsettings` section, more specifically the ExcludeFromPatientPurge option, on how to exclude more resources from being purged.
+
+::
+
+  POST <base-url>/Patient/example/$purge
 
 Appsettings
 -----------
@@ -43,7 +55,33 @@ To enable the $erase operation you will first have to make sure the plugin ``Von
           ...
         ]
       }, ...etc...
+    ]
+  },
+  "EraseOperation": {
+      "ExcludeFromPatientPurge": [ ] // AuditEvents and Provenances will never be deleted 
+  }
+
+
+Use `ExcludeFromPatientPurge` to list resource types that are included in the Patient compartment but should not get deleted on patient `$purge` operation. By default, it contains only `AuditEvent` and `Provenance`.
+
+Many resources in the Patient compartment reference resources outside the compartment. For example, a DeviceRequest might reference a Device. As Device itself is not in the Patient compartment, the resource would normally not be erased on `$purge`.
+
+AuditEvent & Provenance resources
+---------------------------------
+- It is not allowed to erase AuditEvents using `$erase`
+- It is not allowed to permanentely delete AuditEvent and Provenance resources using `$purge`
+- AuditEvents that are created for the `$erase` and `$purge` operations will contain the list of deleted items
+
+SMART on FHIR
+-------------
+To work with SMART on FHIR plugin of Firely Server, you need following custom scopes when requesting an access token
+
+- Scope ``http://server.fire.ly/auth/scope/erase-operation`` for `$erase`
+- Scope ``http://server.fire.ly/auth/scope/purge-operation`` for `$purge`
+
+.. note::
 
+  When the above custom scopes are used, the other SMART on FHIR scopes will be ignored by Firely Server. Due to this limitation, scopes for `$erase` and `$purge` should only be granted to admin users.
 
 AuditEvents
 -----------
@@ -52,4 +90,4 @@ AuditEvents
 
 License
 -------
-The $erase operation is part of the core Firely Server functionality. However, to use it, you may need to request an updated license from Firely. You can use your current license file if it contains ``http://fire.ly/vonk/plugins/erase``.
+The `$erase` and `$purge` operations are part of the core Firely Server functionality. However, to use it, you may need to request an updated license from Firely. You can use your current license file if it contains ``http://fire.ly/vonk/plugins/erase``.
diff --git a/features_and_tools/customresources.rst b/features_and_tools/customresources.rst
@@ -57,7 +57,3 @@ Now use your favorite editor to create a resource that conforms to the Foo Struc
 All the operations on specification-defined resourcetypes are also available for custom resources. You can also use them in a batch or transaction bundle. 
 Custom Resources can also be validated. This also means that :ref:`feature_prevalidation` can be used in conjunction with Custom Resources.
 
-Search parameters on a custom resource
---------------------------------------
-
-In Firely Server you can define your own custom search parameters on any type of resource (see :ref:`feature_customsp`). This includes Custom Resources. Just use the type name of the Custom Resource in the SearchParameter.base.
diff --git a/getting_started/binary.rst b/getting_started/binary.rst
@@ -65,5 +65,6 @@ You will see the following homepage:
 
 .. image:: ../images/localhost_home.png
   :align: center
+  :width: 760px
 
 The next step is to explore Firely Server functionality using Postman. The section :ref:`postman_tutorial` will guide you through this.
diff --git a/getting_started/postman_tutorial.rst b/getting_started/postman_tutorial.rst
@@ -48,11 +48,13 @@ Using Postman
     .. image:: ../images/postman_tutorial_variables.png
        :align: center
 
-#. Test the first request, metadata, as seen below by clicking "Send". This will return the server Capability Statement.
+#. Test the first request, metadata, as seen below by clicking "Send".
 
     .. image:: ../images/postman_tutorial_metadata.png
        :align: center    
 
+#.  This will return the server Capability Statement:
+
     ::
 
         {

diff --git a/index.rst b/index.rst
@@ -6,6 +6,7 @@ Firely Server is the answer to the growing need for a stable server that can be
 
 .. image:: ./images/firelyserver_home.png
   :align: center
+  :width: 760px
 
 On these pages we provide you with the documentation you need to get up and running with your own standard Firely Server
 installation, as well as information on how to contact us when you have additional needs, such as a custom implementation

diff --git a/releasenotes/releasenotes_v5.rst b/releasenotes/releasenotes_v5.rst
@@ -3,6 +3,36 @@
 Current Firely Server release notes (v5.x)
 ==========================================
 
+.. _vonk_releasenotes_5_2_0:
+
+
+Release 5.2.0, August 8th, 2023
+-------------------------------
+
+Configuration
+^^^^^^^^^^^^^
+#. Firely Server now raises a configuration error if the https port is bound to the same port as http
+
+Features
+^^^^^^^^
+
+#. An informational message is now logged for auditing pruposes if authorization for a request was successful. Previously only authorization failures were logged.
+#. Improved compartment checks for writing resources to a Patient compartment with a patient-level access token. All compartment references need to refer to the same compartment. This is important for resources that have multiple compartment references which may refer to different Patients (e.g. AllergyIntolerance.recorder and AllergyIntolerance.patient).
+#. Added support for permanently deleting all resources within a Patient compartment using the $purge operation. See :ref:`erase` for more details.
+#. Enable FS to write logs to AWS CloudWatch, see :ref:`configure_log_sinks`.
+#. We upgraded Firely Server to the latest SDK 5.2.0, see its `releasenotes <https://github.com/FirelyTeam/firely-net-sdk/releases/tag/v5.2.0>`_. 
+
+Fixes
+^^^^^
+
+#. The ``_count`` argument was not marked as handled in the case of an HTTP 401 - Unauthorized status code, leading to a superfluous warning message in the resulting OperationOutcome.
+#. modifierExtensions without a matching StructureDefinition in the administration database are no longer rejected when the validation level is set to "Core".
+#. Improved transaction handling by making sure that accidentally providing a versionId in a resource within a transaction does not lead to versioned references.
+#. Fixed a bug in ``$everything`` running on SQL server that resulted in contained resources being returned as individual resources outside of their container.
+#. The SearchAnonymization plugin now also anonymizes URLs in a history bundle.
+#. The FHIR specification does not allow the use of arbitrary search parameters on the ``_history`` operation. Firely Server now enforces this and rejects those parameters.
+#. Simplifier projects specified under the AdministrationImportOptions were not imported on start-up
+
 .. _vonk_releasenotes_5_1_1:
 
 Release 5.1.1, June 29th, 2023

diff --git a/setting_up_firely_server/configuration/logsettings.rst b/setting_up_firely_server/configuration/logsettings.rst
@@ -221,6 +221,26 @@ Seq server::
 * Change ``serverUrl`` to the URL of your Seq server
 * ``restrictedToMinimumLevel``: as described for `Console`_.
 
+AWS Cloudwatch
+^^^^^^^^^^^^^^
+Firely Server can also log to AWS Cloudwatch. What you need to do:
+
+#. Create a user with restricted privilages in AWS that can write to Cloudwatch
+#. Configure the machine with the Firely Server instance with the credentials of this AWS account 
+#. These 2 steps are described `here <https://docs.aws.amazon.com/sdk-for-net/v3/developer-guide/creds-assign.html>`_
+#. Add the correct sink to the logsettings.json::
+
+		"WriteTo": [
+			{
+				"Name": "AmazonCloudWatch",
+				"Args": {
+					"logGroup": "<the name of your log group>",
+					"logStreamPrefix": "<the description to prefix your log stream>", 
+					"restrictedToMinimumLevel": "Verbose" //Or a higher level
+				}
+			},
+		],
+
 .. _configure_log_database:
 
 Database details

diff --git a/setting_up_firely_server/configuration/sqlserver/database_security.rst b/setting_up_firely_server/configuration/sqlserver/database_security.rst
@@ -0,0 +1,39 @@
+Database security
+=================
+
+The following explores some of the various security measures that exist in Microsoft SQL Server and their applicability to a Firely Server SQL database.
+
+Transparent Data Encryption (TDE)
+---------------------------------
+After enabling TDE, the data stored on disk will be encrypted, using a certificate to protect the keys used for encryption.
+This prevents copies of the database to be read properly without the certificate.
+The performance impact of using this security measure will be on the database server, as the data is encrypted/decrypted during write/read to/from disk activities.
+
+`More information <https://learn.microsoft.com/en-us/sql/relational-databases/security/encryption/transparent-data-encryption?view=sql-server-ver16>`_
+
+Data Masking
+------------
+Data masking will obfuscate (parts of) database columns on a database user level. Granting 'unmasked' privileges to the application user and stricter masking to other users will limit the exposure of sensitive data while querying the database.
+This security measure should be used in conjunction with other security measures.
+The performance impact for Firely Server will be minimal since it is required that the database user configured for Firely Server be set to fully 'unmasked'.
+
+`More information <https://learn.microsoft.com/en-us/sql/relational-databases/security/dynamic-data-masking?view=sql-server-ver16>`_
+
+Row Level Security
+------------------
+Row level security is not supported by Firely Server.
+
+`More information <https://learn.microsoft.com/en-us/sql/relational-databases/security/row-level-security?redirectedfrom=MSDN&view=sql-server-ver16>`_
+
+Encrypted Connections
+---------------------
+With encrypting connections the data traffic between the database and the Firely Server will be encrypted by using certificates.
+The performance impact will be minimal, similar to the difference between http and https.
+
+`More information <https://learn.microsoft.com/en-us/sql/database-engine/configure-windows/configure-sql-server-encryption?view=sql-server-ver16>`_
+
+Always Encrypted
+----------------
+Always encrypted is a client side operation before storing the data in the database. This is currently not supported by Firely Server.
+
+`More information <https://learn.microsoft.com/en-us/sql/relational-databases/security/encryption/always-encrypted-database-engine?view=sql-server-ver16>`_
diff --git a/setting_up_firely_server/configuration/sqlserver/db_sql.rst b/setting_up_firely_server/configuration/sqlserver/db_sql.rst
@@ -252,3 +252,4 @@ Performance tuning of SQL Server is a topic on its own. Firely Server manages th
    backup_maintenance_planning
    index_maintenance
    sqlserver_statistics
+   database_security
diff --git a/setting_up_firely_server/deployment/azureWebApp.rst b/setting_up_firely_server/deployment/azureWebApp.rst
@@ -25,6 +25,7 @@ Deployment
 
    .. image:: ../../images/Azure_02_ChooseName.png
       :align: center
+      :width: 760px
 
 #. Add the trial license file (firelyserver-trial-license.json) to the firely-server-latest.zip by dragging the license file into the zipfile.
 #. Open a webbrowser, navigate to ``https://<webapp>.scm.azurewebsites.net/ZipDeployUI`` and drag vonk_distribution.zip into the browser window. 
@@ -35,6 +36,7 @@ Deployment
 
    .. image:: ../../images/Azure_05_WebRoot.png
       :align: center
+      :width: 900px
 
 #. Open a browser and go to the site ``https://<webapp>.azurewebsites.net/`` . This will show the Firely Server home page.
 
@@ -46,6 +48,7 @@ with the settings for either :ref:`SQL Server<configure_sql>` or :ref:`MongoDB<c
 
 .. image:: ../../images/Azure_04_Settings.png
    :align: center
+   :width: 900px
 
 More information
 ----------------

diff --git a/setting_up_firely_server/deployment/docker.rst b/setting_up_firely_server/deployment/docker.rst
@@ -143,22 +143,56 @@ Powershell
 
 You should see a ``vonkdata.db`` appear in the ``./resourcedata`` folder, and a log file in the ``./log`` folder. From here you can experiment with other settings. You can also easily keep different settings files side-by-side, mapping the one you want to test into the container, e.g. ``-v ${PWD}/some-weird-settings.json:/app/appsettings.instance.json``.
 
-Mounting a your custom plugins folder into the container
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-In a similar way as described above, you can mount your custom plugins into the container. However, this will replace the default plugings; they need to be added manually.
+Loading additional conformance resources in Firely Server on Docker
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-- Download the binaries of your server version from https://downloads.simplifier.net/firely-server/versions/
-- Create a plugin folder, including the default plugins (extracted from the downloaded archive)
-- Create ``appsettings.instance.json`` with the appropriate Pipeline section (see :ref:`vonk_plugins_config`)
-- Mount the files and folder as shown below
+If you want to load custom conformance resources in Firely Server, you can use the ``vonk-import`` folder, as is described in the section `Managing Conformance Resources <../maintenance/conformanceresources.html>`_.
+When running Firely Server on Docker, it is necessary to create this folder in your working directory, copy the conformance resources into this folder and mount a volume. 
+Be sure to also mount a volume for the ``vonk-imported`` folder, otherwise your conformance resources will be reloaded upon each startup of Firely Server and this can take up some time:
+
+.. code-block::
+   
+   docker run -d -p 8080:4080 --name firely.server `
+   -v ${PWD}/appsettings.instance.json:/app/appsettings.instance.json `
+   -v ${PWD}/vonk-import:/app/vonk-import `
+   -v ${PWD}/vonk-imported:/app/vonk-imported `
+   firely/server
+
+Be sure to mount your appsettings.instance.json as well, and make sure to point the ``AdministrationImportOptions`` to the ``vonk-import`` and ``vonk-imported`` folder:
+
+.. code-block::
+
+  "AdministrationImportOptions": {
+		"ImportDirectory": "./vonk-import",
+    "ImportedDirectory": "./vonk-imported"
+  }
+
+By default, Firely server will assume the resources placed in the vonk-import folder are R3 resources. If you want to load R4 or R5 resources, you need to alter the name of your import folder and volume accordingly:
 
 .. code-block::
    
    docker run -d -p 8080:4080 --name firely.server `
-   -v ${PWD}/firelyserver-license.json:/app/firelyserver-license.json `
    -v ${PWD}/appsettings.instance.json:/app/appsettings.instance.json `
-   -v ${PWD}/plugins:/app/plugins `
+   -v ${PWD}/vonk-import.R4:/app/vonk-import.R4 `
+   -v ${PWD}/vonk-imported:/app/vonk-imported `
+   firely/server
+
+You can leave the ``AdministrationImportOptions`` in the appsettings.instance.json as is, there is no need to point these settings to a separate vonk-import.R4 or vonk-import.R5 folder.
+
+Mounting your custom plugins folder into the container
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In a similar way as described above, you can mount your custom plugins into the container.
+In your working directory, create a folder 'plugins', copy your plugin files to this folder and mount this folder in the docker container. You can add a folder structure inside the plugin folder if you want.
+
+.. warning:: Do not change the target folder to '/app/plugins' as it will overwrite the existing plugins folder in the docker image.
+
+.. code-block::
+   
+   docker run -d -p 8080:4080 --name firely.server `
+   -v ${PWD}/firelyserver-license.json:/app/firelyserver-license.json `
+   -v ${PWD}/plugins:/app/plugins/custom `
    firely/server
 
 The server is now accessible on ``http://localhost:8080/``.