[Need QA Prep] Feature/The Amicable Node-Preprint Divorce [PLAT-555][PLAT-480][PLAT-559][PLAT-561]

[pattisdr]

Purpose

Removes the dependency of Preprints on Nodes. Nodes can house supplemental files for a preprint, but beyond that, Preprints are completely independent.

Ticket contains:

• Backend changes for node-preprint divorce
• Schema and data migrations for legacy preprints
• v1/Non-Ember front-end changes
• API changes for preprints
• ❗️ Doc changes haven't been made yet. Support pages will also need to be updated (Becca?). This can wait until front-end is ready?

How to create a preprint in the API

Step 1: Start Preprint

POST http://localhost:8000/v2/preprints/
Required fields are title and provider

request body
 {
     "data": {
       "type": "preprints",
       "attributes": {
        "title": "My first preprint",
        "tags": ["coffee", "cinnamon"]
       },
       "relationships": {
          "provider": {
             "data": {
                "type": "providers",
                "id": "osf"
             
             }
          }
    		
       }
     }
}

Step 2 Upload new file to preprint, or copy file from existing node:

Uploading new file

GET http://localhost:8000/v2/preprints/nmqp6/files/ and then

curl -X "PUT" "http://localhost:7777/v1/resources/nmqp6/providers/osfstorage/?kind=file&name=my_preprint_file.txt" \
--
-H "Authorization: Bearer my-token-goes-here" \
-H "Content-Type: text/plain" \
-d "This is a text file"

Copying file from supplementary node

Get "move" link from supplementary node and then POST to move link where resource is the preprint id, action is copy, and provider is osfstorage.
GET http://localhost:8000/v2/nodes/4u3mk/files/osfstorage/ and then

curl -X "POST" "http://localhost:7777/v1/resources/4u3mk/providers/osfstorage/5b2803b04c9d8502e5aea0cf" \
--
-H "Authorization: Bearer my-token-goes-here " \
-H "Content-Type: application/json" \
-d '{"action": "copy", "path": "/", "resource": "nmqp6", "provider": "osfstorage", "conflict": "replace"}'

Step 3 - Update preprint to set newly uploaded file as primary file, add subjects if not there already, (supplementary node optional) and publish

PATCH http://localhost:8000/v2/preprints/nmqp6/
 {
     "data": {
       "id": "nmqp6",
       "type": "preprints",
       "attributes": {
        "subjects": [["5b2419881f4ae00001670157"]],
        "is_published": "true"
       },
       "relationships": {
          "primary_file": {
             "data": {
                "type": "files",
                "id": "5b2809294c9d8502e5aea0e1"
             }
          },
          "node": {
             "data": {
                 "type": "nodes",
                 "id": "4u3mk"
             }
          }
    		
       }
     }
}

Changes

Too many to list, here are some of them.

Admin App

• Preprints are their own thing in the admin app, not relying on node fields
• You can modify contributors directly, flag preprints as spam, and reindex preprints in elastic search
• Several node templates/views made more generic to work with nearly-identical preprints

API

• Preprint endpoints are looking at preprint fields, not node fields.
• PreprintProviderList endpoint and PreprintFiles list endpoint added (to expose OSFStorage Waterbutler endpoints for uploading primary file
• New process for creating a preprint 1) POST to v2/preprints/ (title, provider are required fields) 2) Use WB links to upload or move file to created preprint 3) PATCH to v2/preprints/<preprint_id>/ to finish up process.
• New preprints scopes added now that preprints/nodes are separate

Elastic Search

• Legacy nodes attached to preprints are reindexed as nodes in a one-off script (scripts/remove_after_use/node_preprint_es.py)
• Legacy preprints are added to the index as preprints
• Moving forward, nodes are indexed as nodes, preprints are indexed as preprints
• Modifications to search migration to not include QuickFIlesNode in Project list (bug fix?)
• Modifications to search migration to add indexing for preprints and preprint files, and to stop indexing nodes as preprints

Modeling

• New PreprintLog and PreprintContributor models
• New ContributorMixin to share contributor-related methods between Nodes and Preprints
• New SpamOverrideMixin to share spam-methods between Nodes and Preprints
• Preprints uses Django-guardian for contributor permissions
• Write contributors have been upgraded to have more permissions - can edit most things on a submitted preprint except publishing, withdrawing, etc.
• Preprints do not have addons, so a few OSFStorage methods have been moved to the preprint.
• UploadMixin added to Preprint that creates a root folder for storage

Preprint File

• Stored on preprint instead of node
• Some OSFStorage methods modified to handle permission checking on preprint in addition to node
• Preprints have regions (by default, same region as UserSettings, in migration, pulls region off of node).

Legacy UI

• Dashboard My Preprints fetches from users/preprints/ instead of users/nodes/?filter['preprint']=True. Activity logs removed
• Preprint file restrictions removed from fangorn
• Language changed on project page to reflect that project is a supplemental file for a project

Supplemental project attached to withdrawn preprint - status says Withdrawn

Withdrawn preprint still shows up for a user on My Projects page. Is marked as withdrawn.

Published preprints show up on My Projects page

My private preprints still show up on My Projects page

Selecting multiple preprints on My Projects page

Elastic search before migration - 2 preprints 3 projects

Elastic search after migration - 2 preprints 5 projects (2 projects formerly indexed as preprints were reindexed as projects)

Preprint files don't have links, but you can click on preprint link. File detail for preprint page is blocked.

QA Notes

Many many things to test. A lot of Node code was adapted to work for preprints so we want to make sure we didn't break things. Preprints front-end is not addressed here.

To check on Node

• That file logs still work, (osfstorage_file_added, osfstorage_file_copied, for example). Do all the file operations on a node, verify logs are what you expect.
• Contributor operations on nodes still work.
• Project page banners that your node is a supplementary file for a preprint are what you expect, and appear when you expect. Test for various providers.
• MyProjects page shows what you would expect for preprints. Activity log has been removed since we're not exposing preprint logs yet.
• Searching for nodes still works the way it used to
• SHARE updates for projects still work the way you would expect.
• Test nodes in admin app, removing contributors, marking node as deleted, marking nodes as spam/ham,
• Make sure deleting nodes with children on them still works

To check on preprint

• Preprints themselves, instead of nodes are indexed in elastic search
• Preprints are indexed in SHARE correctly (HAVE NOT TESTED THIS LOCALLY)
• Test preprints in admin app, removing contributors, marking preprint as deleted, marking preprints as spam/ham, reindexing preprint in elastic search
• Preprint files only show up in search if preprint is verified publishable
• If you change the preprint file, the original is removed from search
• Preprint files don't have links to file in search
• Full regression on preprint dois

Checking migration

• Make several preprints ahead of time, make sure they have a title, abstract, primary_file, multiple contributors with different permissions/bibligraphic, article_doi, some public, others private, some that have been deleted, some that have been abandoned, tags, some whose nodes have been marked as spam, modified date.
• modified date of preprint will be modified to the date of a preprint-related node log if it is a later date than the preprint modified.
• Make sure that all of these files are moved over to the preprint.

Side Effects

Ticket

[pattisdr]

Bug fix? Seems like this would make quickfile nodes show up in search

[pattisdr]

Recent updates involve responding to an in-person review w/ Steve and accommodating Erin's DOI work.

[pattisdr]

Now that the one preprint per provider constraint has been removed, project page can list the multiple preprints for which it holds supplemental information. In practice, this will probably be uncommon.

[pattisdr]

@laurenbarker, constraint of one preprint per provider per node removed.