Support for configurable list of licenses #7920
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,119 @@ | ||||||
| ### Multiple License Support | ||||||
|
|
||||||
| Users can now select from a set of configured licenses in addition to or instead of the current Creative Commons CC0 choice or provide custom terms of use (if configured) for their datasets. Administrators can configure their Dataverse instance via API to allow any desired license as a choice and can enable/disable the option to allow custom terms. Administrators can also mark licenses as 'inactive' to disallow future use while keeping that license for existing datasets. By default, only the CC0 license will be preinstalled. Examples in the Guides show how to add additional licenses and specific examples are given for several Creative Commons licenses. **Note: Datasets in existing installations will automatically be updated to conform to new requirements that custom terms cannot be used with a standard license and that custom terms cannot be empty. Administrators may wish to manually update datasets with these conditions if they do not like the automated migration choices. See the Notes for Dataverse Installation Administrators and Additional Release Steps sections for further information.** | ||||||
|
|
||||||
|
|
||||||
| ## Major Use Cases and Infrastructure Enhancements | ||||||
|
|
||||||
|
|
||||||
| - When creating/updating datasets, users can select from a set of standard licenses configured by the administrator or provide custom terms (if the installation is configured to allow them). | ||||||
|
|
||||||
| ## Notes for Dataverse Installation Administrators | ||||||
|
|
||||||
| ### Updating for multiple license support | ||||||
|
|
||||||
| As part of installing/upgrading an existing installation, administrators may wish to add additional license choices and/or configure Dataverse to allow custom terms. Adding additional licenses is managed via API. Licenses are described via a JSON structure providing a name, URL, short description, and optional icon URL. Additionally licenses may be marked as active (selectable for new/updated datasets) or inactive (only allowed on existing datasets) and one license can be marked as the default. Custom Terms are allowed by default (backward compatible with the current option to select 'No' to using CC0) and can be disabled by setting `:AllowCustomTermsOfUse` to false. | ||||||
|
|
||||||
| Further, administrators should review the following automated migration of existing licenses and terms into the new license framework and, if desired, should manually find and update any datasets for which the automated update is problematic. | ||||||
| To understand the migration process, it is useful to understand how the multiple license feature works in this release: | ||||||
|
|
||||||
| 'Custom Terms', aka a custom license, are defined through entries in the following fields of the dataset "Terms" tab: | ||||||
| - Terms of Use | ||||||
| - Confidentiality Declaration | ||||||
| - Special Permissions | ||||||
| - Restrictions | ||||||
| - Citation Requirements | ||||||
| - Depositor Requirements | ||||||
| - Conditions | ||||||
| - Disclaimer | ||||||
|
|
||||||
| 'Custom Terms' require, at a minimum, a non-blank entry in the "Terms of Use" field. Entries in other fields are optional. | ||||||
|
|
||||||
| Since these fields are intended for terms/conditions that would potentially conflict with or modify the terms in a standard license, they are no longer shown when a standard license is selected. | ||||||
|
|
||||||
| In earlier Dataverse releases, it was possible to select the CC0 license and have entries in the fields above. It was also possible to say 'No' to using CC0 and leave all of these terms fields blank. | ||||||
|
|
||||||
| The automated process will update existing datasets as follows. | ||||||
|
|
||||||
| - 'CC0 Waiver' and no entries in the fields above -> CC0 License (no change) | ||||||
|
There was a problem hiding this comment.
Suggested change
In the past we've made emphasized that CC0 is a waiver rather license. I assume we want to continue doing this. There was a problem hiding this comment. This is a tricky one. I think Waiver made sense when the only alternative was to add custom terms/not accept the waiver, but now with the overall option being 'Licenses' having one of them labelled as a Waiver may be more confusing than it is worth. There are more comments about naming below, but I'll note here that the intent was not to replace screen text 'CC0 Waiver' with 'CC0 License' - it would just be 'CC0' as the value where the License/DUA choice is shown (DUA == data user agreement). Nominally the important connection for a license is the URL since that gets you to the text of the license, so the name could potentially be non-standard, i.e. it could be 'CC0 Waiver' if we wanted that as a default or if individual sites wanted more backward compatibility. (Not sure if one would then add 'CC-By License' instead of 'CC-By' etc.) I think where the code is now is trying to use the standard community name (perhaps confused with identifier still in some places - more below), which would presumably be used if/when licenses get tied to an external service and/or if internationalization is added. |
||||||
| - No CC0 Waiver and an entry in the Terms of Use field and possibly others fields listed above -> 'Custom Terms' with the same entries in these fields (no change) | ||||||
|
|
||||||
| - CC0 Waiver and an entry in some of the fields listed -> 'Custom Terms' with the following text preprended in the "Terms of Use" field: "This dataset is made available under a Creative Commons CC0 license with the following additional/modified terms and conditions:" | ||||||
|
There was a problem hiding this comment. Installations that enter metadata in non-English language may not like this. I can imagine a mix of English at the top (prepended) and French below. I guess they can do a SQL query for datasets that were modified in this way and replace the English with French. There was a problem hiding this comment. Yes. I'm not sure that anyone will think that is a good default, but we added something just so that a naïve update won't lose info or result in an invalid/legacy case where a dataset now has a CC0 license and custom terms - something you can't create with the new UI. (Not sure if it has been said explicitly anywhere yet but in the design and review of this feature, the idea that one should not be able to add terms of use (the specific termsOfUse field and others that also present a way to modify/counter license terms) was agreed to, as was the idea that having a custom license and a blank/empty terms of use field (essentially having no terms at all) should no longer be allowed. These have created the question about how to handle legacy datasets that violate those restrictions. From looking at Harvard and some other sites, it seems like these cases should be rare, so the idea of providing queries/a process for finding and fixing them prior to upgrading (could be done way before upgrading too) and then providing some default that would catch any remaining cases in a way that works technically and arguably doesn't change the meaning, seems like an acceptable approach. FWIW: to help clarify things for the community, I was also hoping to do a recorded segment in a community meeting that would show off the new multi-license feature and explain what would happen to existing data/how to avoid the default upgrades if desired.)) There was a problem hiding this comment. Sure, I definitely think showing off the new feature during a community call would be interesting and helpful. Makes sense what you're saying about converting invalid/legacy cases to valid ones. |
||||||
| - No CC0 Waiver and an entry in a field(s) other than the Terms of Use field -> 'Custom Terms' with the following "Terms of Use" added: "This dataset is made available with limited information on how it can be used. You may wish to communicate with the Contact(s) specified before use." | ||||||
| - No CC0 Waiver and no entry in any of the listed fields -> 'Custom Terms' with the following "Terms of Use" added: "This dataset is made available without information on how it can be used. You should communicate with the Contact(s) specified before use." | ||||||
|
|
||||||
| Administrators who have datasets where CC0 has been selected along with additional terms, or datasets where the Terms of Use field is empty, may wish to modify those datasets prior to upgrading to avoid the automated changes above. The Additonal Release Steps provides information on how to find and modify any such datasets. | ||||||
|
|
||||||
| ## New JVM Options and DB Settings | ||||||
|
|
||||||
| - `:AllowCustomTermsOfUse` (default: true) allow users to provide Custom Terms instead of choosing one of the configured standard licenses. | ||||||
|
|
||||||
| See the [Database Settings](https://guides.dataverse.org/en/5.10/installation/config.html) section of the Guides for more information. | ||||||
|
|
||||||
| ## Additional Release Steps | ||||||
|
|
||||||
| In most Dataverse installations, one would expect the vast majority of Datasets to either use the CC0 Waiver or have non-empty Terms of Use. As noted above, these will be migrated without any issue. Administrators may however wish to find and manually update datasets that specified a CC0 license but also had terms (no longer allowed) or had no license and no terms of use (also no longer allowed) rather than accept the default migrations for these datasets listed above. | ||||||
|
|
||||||
| ### To find Datasets with a CC0 license and non-empty terms: | ||||||
|
|
||||||
| select CONCAT('doi:', dvo.authority, '/', dvo.identifier), v.alias as dataverse_alias, case when versionstate='RELEASED' then concat(dv.versionnumber, '.', dv.minorversionnumber) else versionstate END as version, dv.id as datasetversion_id, t.id as termsofuseandaccess_id, t.termsofuse, t.confidentialitydeclaration, t.specialpermissions, t.restrictions, t.citationrequirements, t.depositorrequirements, t.conditions, t.disclaimer from dvobject dvo, termsofuseandaccess t, datasetversion dv, dataverse v where dv.dataset_id=dvo.id and dv.termsofuseandaccess_id=t.id and dvo.owner_id=v.id and t.license='CC0' and not (t.termsofuse is null and t.confidentialitydeclaration is null and t.specialpermissions is null and t.restrictions is null and citationrequirements is null and t.depositorrequirements is null and t.conditions is null and t.disclaimer is null); | ||||||
|
|
||||||
| The datasetdoi column will let you find/view the affected dataset in the Dataverse web interface. The version column will indicate which version(s) are relevant. The dataverse_alias will tell you which Dataverse collection the dataset is in (and may be useful if you want to adjust all datasets in a given collection). The termsofuseandaccess_id column indicates which specific entry in that table is associated with the dataset/version. The remaining columns show the values of any terms fields. | ||||||
|
|
||||||
| There are two choices to migrate such datasets: | ||||||
|
|
||||||
| - Set all terms fields to null: | ||||||
|
|
||||||
|
|
||||||
| update termsofuseandaccess set termsofuse=null, confidentialitydeclaration=null, t.specialpermissions=null, t.restrictions=null, citationrequirements=null, depositorrequirements=null, conditions=null, disclaimer=null where id=<id>; | ||||||
|
|
||||||
| or to change several at once: | ||||||
|
|
||||||
| update termsofuseandaccess set termsofuse=null, confidentialitydeclaration=null, t.specialpermissions=null, t.restrictions=null, citationrequirements=null, depositorrequirements=null, conditions=null, disclaimer=null where id in (<comma separated list of termsanduseofaccess_ids>); | ||||||
|
|
||||||
| - Alternately, change the Dataset version(s) to not use the CCO waiver and modify the Terms of Use (and/or other fields) as you wish to indicate that the CC0 waiver was previously selected: | ||||||
|
|
||||||
|
|
||||||
| update termsofuseandaccess set license='NONE', termsofuse=concat('New text. ', termsofuse) where id=<id>; | ||||||
|
|
||||||
| or | ||||||
|
|
||||||
| update termsofuseandaccess set license='NONE', termsofuse=concat('New text. ', termsofuse) where id in (<comma separated list of termsanduseofaccess_ids>); | ||||||
|
|
||||||
| ### To find datasets without CC0 and having an empty Terms of Use field: | ||||||
|
|
||||||
| select CONCAT('doi:', dvo.authority, '/', dvo.identifier), v.alias as dataverse_alias, case when versionstate='RELEASED' then concat(dv.versionnumber, '.', dv.minorversionnumber) else versionstate END as version, dv.id as datasetversion_id, t.id as termsofuseandaccess_id, t.termsofuse, t.confidentialitydeclaration, t.specialpermissions, t.restrictions, t.citationrequirements, t.depositorrequirements, t.conditions, t.disclaimer from dvobject dvo, termsofuseandaccess t, datasetversion dv, dataverse v where dv.dataset_id=dvo.id and dv.termsofuseandaccess_id=t.id and dvo.owner_id=v.id and t.license='NONE' and t.termsofuse is null; | ||||||
|
|
||||||
| These datasets could be updated to use CC0: | ||||||
|
|
||||||
| update termsofuseandaccess set license='CC0', confidentialitydeclaration=null, t.specialpermissions=null, t.restrictions=null, citationrequirements=null, depositorrequirements=null, conditions=null, disclaimer=null where id=<id>; | ||||||
|
|
||||||
| or Terms of Use could be added: | ||||||
|
|
||||||
| update termsofuseandaccess set termsofuse='New text. ' where id=<id>; | ||||||
|
|
||||||
| In both cases, the same where id in (`<comma separated list of termsanduseofaccess_ids>`); ending could be used to change multiple datasets/versions at once. | ||||||
|
|
||||||
| ### Standardizing Custom Licenses: | ||||||
|
|
||||||
| If many datasets use the same set of Custom Terms, it may make sense to create and register a standard license including those terms. Doing this would include: | ||||||
| - Creating and posting an external document that includes the custom terms, i.e. an HTML document with sections corresponding to the terms fields that are used. | ||||||
| - Defining a name, short description, URL (where it is posted), and optionally an icon URL for this license | ||||||
| - Using the Dataverse API to register the new license as one of the options available in your installation | ||||||
| - Using the API to make sure the license is active and deciding whether the license should also be the default | ||||||
| - Once the license is registered with Dataverse, making an SQL update to change datasets/versions using that license to reference it instead of having their own copy of those custom terms. | ||||||
|
|
||||||
| The benefits of this approach are: | ||||||
| - usability: the license can be selected for new datasets without allowing custom terms and without users having to cut/paste terms or collection administrators having to configure templates with those terms | ||||||
| - efficiency: custom terms are stored per dataset whereas licenses are registered once and all uses of it refer to the same object and external URL | ||||||
| - security: with the license terms maintained external to Dataverse, users cannot edit specific terms and curators do not need to check for edits | ||||||
|
Comment on lines
+97
to
+109
There was a problem hiding this comment. I love this write up of the benefits of custom licenses and how to create them but I'm afraid it'll get lost in a one time release note. Should we copy this into the guides? There was a problem hiding this comment. Sounds good. I'll look for a spot. |
||||||
|
|
||||||
| Once a standardized version of you Custom Terms are registered as a license, an SQL update like the following can be used to have datasets use it: | ||||||
|
|
||||||
| UPDATE termsofuseandaccess | ||||||
| SET license_id = (SELECT license.id FROM license WHERE license.name = '<Your License Name>'), termsofuse=null, confidentialitydeclaration=null, t.specialpermissions=null, t.restrictions=null, citationrequirements=null, depositorrequirements=null, conditions=null, disclaimer=null | ||||||
| WHERE termsofuseandaccess.termsofuse LIKE '%<Unique phrase in your Terms of Use>%'; | ||||||
|
|
||||||
|
|
||||||
|
|
||||||
| select CONCAT('doi:', dvo.authority, '/', dvo.identifier), v.alias as dataverse_alias, case when versionstate='RELEASED' then concat(dv.versionnumber, '.', dv.minorversionnumber) else versionstate END as version, dv.id as datasetversion_id, t.id as termsofuseandaccess_id, t.termsofuse, t.confidentialitydeclaration, t.specialpermissions, t.restrictions, t.citationrequirements, t.depositorrequirements, t.conditions, t.disclaimer from dvobject dvo, termsofuseandaccess t, datasetversion dv, dataverse v where dv.dataset_id=dvo.id and dv.termsofuseandaccess_id=t.id and dvo.owner_id=v.id and t.license_id=1 and not (t.termsofuse is null and t.confidentialitydeclaration is null and t.specialpermissions is null and t.restrictions is null and citationrequirements is null and t.depositorrequirements is null and t.conditions is null and t.disclaimer is null); | ||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| { | ||
| "name": "CC-BY-4.0", | ||
|
There was a problem hiding this comment. Should this be "CC BY 4.0" (without the hyphens)? That's what it says at https://creativecommons.org/licenses/by/4.0/
Update: I talked to @4tikhonov and he pointed me to https://spdx.org/licenses/ which leads me to believe that "name" should be "identifer" since "CC-BY-4.0" seems to be an identifier. However, now I'm concerned that we're showing these machine-readable identifiers in the UI. If I saw "MIT-0" in the GUI below I wouldn't know what it means but "MIT No Attribution" is clear.
There was a problem hiding this comment. I think the URL is really the primary identifier and the 'name' is intended for display rather than as a second less specific way to identify the license globally. So - I think I agree that the non-hyphenated versions make more sense and we can change those (unless someone else has a concern about that?). (Note - since these are configured via API, installations that prefer some other user-visible label (or want to have them in another language, etc.), they can still do that. The interoperability is intended to be by URL - if that matches, it's the same license. There was a problem hiding this comment. I have no objection to using a URL as a primary identifier but are our URLs aligned with SPDX, for example? It seems like not (CC-BY-4.0 example): In short, if we're going to use URLs as identifiers it would be nice to see a canonical list of these URLs somewhere. There was a problem hiding this comment. The URLs used now are what Creative Commons has assigned to their licenses. Their page metadata include attributes like |
||
| "uri": "http://creativecommons.org/licenses/by/4.0", | ||
| "shortDescription": "Creative Commons Attribution 4.0 International License.", | ||
| "iconUrl": "https://i.creativecommons.org/l/by/4.0/88x31.png", | ||
| "active": true | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3717,3 +3717,28 @@ Recursively applies the role assignments of the specified Dataverse collection, | |
| GET http://$SERVER/api/admin/dataverse/{dataverse alias}/addRoleAssignmentsToChildren | ||
|
|
||
| Note: setting ``:InheritParentRoleAssignments`` will automatically trigger inheritance of the parent Dataverse collection's role assignments for a newly created Dataverse collection. Hence this API call is intended as a way to update existing child Dataverse collections or to update children after a change in role assignments has been made on a parent Dataverse collection. | ||
|
|
||
|
There was a problem hiding this comment. This is a geneal comment about updates to the guides. We should probably update the "Terms" section in the User Guide from what's currently there: https://guides.dataverse.org/en/5.9/user/dataset-management.html#terms |
||
|
|
||
| Manage Available Standard License Terms | ||
|
There was a problem hiding this comment. We might want to move these outside "admin" so regular API users can learn what licenses are available. There was a problem hiding this comment. Makes sense - for the get licenses call anyway. Probably good if API users can directly tell if custom licenses are allowed too. Is there a precedent for a split API like that where get is open and the change/delete functionality is restricted? (Looking at how to construct the URLs and where to place the code - I know there are some /api/admin/* calls not in the Admin class, but I don't know what best practice is.) There was a problem hiding this comment. Perhaps we should have a new top level That way, any API client could get a list of licenses even without authenticating. But only superusers would be able to add licenses. |
||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| View the list of standard license terms that can be selected for a dataset:: | ||
|
|
||
| curl http://$SERVER/api/admin/licenses | ||
|
|
||
| View the details of the standard license with the database ID specified in ``$ID``:: | ||
|
|
||
| export ID=1 | ||
| curl http://$SERVER/api/admin/licenses/$ID | ||
|
|
||
| Add a new license by posting a JSON file adapted from this example :download:`add-license.json <../_static/api/add-license.json>`. The ``name`` and ``uri`` of the new license must be unique. :: | ||
|
|
||
| curl -X POST -H 'Content-Type: application/json' --data-binary @add-license.json http://$SERVER/api/admin/licenses | ||
|
|
||
| Overwrite the license with the database specified in ``$ID``:: | ||
|
|
||
| curl -X PUT -H 'Content-Type: application/json' --data-binary @edit-license.json http://$SERVER/api/admin/licenses/$ID | ||
|
|
||
| Delete the license with with the database specified in ``$ID``:: | ||
|
|
||
| curl -X DELETE http://$SERVER/api/admin/licenses/$ID | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm glad this says "By default, only the CC0 license will be preinstalled" because I found it surprising when I deployed the code first and looked at this release note later. For a new installation of Dataverse, should we have several popular Creative Commons licenses already enabled?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We didn't want to add for existing installations since that's a policy change, but having more (e.g. the basic CC choices) for new installations could make sense, although I'm not sure - even then admins may want to review before allowing anything beyond fully public/CC0.
(I'm also not sure how best to do it - would it be a change in the installer to call the API after Dataverse is running? Or is there a better way? (Can the mechanism that autocreates tables for new installs also add rows (and not affect existing tables?). Ansible? (It might be nice to preconfigure with the standard GDCC previewers too - another set of API calls that could/should only run on new installs))
In any case, if there's consensus on changes we can do that here or as part of a new issue/PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, exactly. If we want the product to ship with half a dozen CC licenses, we'd create the JSON files and have the installer create each license. This would be for new installations.
For existing installations, we'd just say in the release note that new licenses are possible and probably point out the JSON files that new installations wlll have loaded if they want to have the same licenses as out of the box Dataverse. I don't think we should populate existing installations with licenses. Let them opt in.
Obviously, yes, this could all be done after this pull request is merged in a new pull request, but I thought I'd raise it now. Otherwise, it feels like a bit of a hidden feature that installations can have a nice dropdown of popular licenses.