Skip to content

Releases: CSCfi/metadata-submitter

v0.13.1

30 May 11:48
5782b4d
Compare
Choose a tag to compare

[0.13.1] - 2022-05-30

Changed

v0.13.0

07 Apr 11:34
ce026c3
Compare
Choose a tag to compare

Added

  • Submission endpoint update #371
    • Adds mandatory query parameter folder for submit endpoint POST
    • On actions add and modify object is added or updated to folder(submission) where it belongs with it's accession ID, schema, submission type, title and filename
    • Adds metax integration to submit endpoint
  • Integration with Metax service #356 #387
    • Adds new local container for testing against mocked Metax API
    • Introduces new env vars: METAX_USER, METAX_PASS, METAX_URL for connection to Metax service
    • Introduces new env var DISCOVERY_URL for creating link to dataset inside Fairdata SD catalog
    • Adds new key metaxIdentifier to Study and Dataset collections containing metax id returned from Metax API
    • Adds new handler MetaxServiceHandler to take care of mapping Submitter metadata to Metax metadata and to connect to Metax API
    • Adds new mapper class to adjust incoming metadata to Metax schema
  • Add patching of folders after object save and update operations #354
    • Adds mandatory query parameter folder for objects endpoint POST
    • Object is added or updated to folder(submission) where it belongs with it's accession ID, schema, submission type, title and filename in the case of CSV and XML upload
    • Adds configuration for mypy linting to VScode devcontainer setup
  • Templates API #256
    • use ujson as default json library
  • Creating draft Datacite DOI for folders #257 #332
    • created a mock web app, which would act similarly to DataCite REST API
    • altered publish_folder endpoint so that extraInfo containing the DOI data is added upon publishing
    • added datePublished key to folders which takes in the date/time, when folder is published
  • DOI Publishing and deletion to Datacite #332 #369
    • create draft DOIs for both Study and Datasets and add them to the folder extraInfo when published
    • delete draft DOIs on object delete
    • update DOI info at Datacite when folder is published
  • VScode Dev environment #287
    • Add VS Code development container
    • Update docker for development
  • Docker-compose and docker-compose-tls files changed to use variables from .env file. #301
  • Add folder querying by name #305
    • Add indexing on database initialization
    • Add new field text_name to folder collection
    • Python scripts for database operations. mongo_indexes.py for collections and indexes creation to be run if the database is destroyed and clean_db.py script with new functionality to only delete documents from collections
    • update github actions
  • Add folder querying by date #308
  • Add description to JSON schemas #323
    • add JSON schema spelling checker to pyspelling github action
    • optimise wordlist by adding regex ignore patterns
    • added pyspelling to pre-commit hooks (fixed syntax for scripts according to https://github.com/koalaman/shellcheck )
    • enum are sorted alphabetically, with the exception of other and unspecified values which are left at the end of the list
    • allow for accession key in referenceAlignment & process sequence as array, previously all accession keys were converted to accessionId which is not correct
    • add default gender as unknown
  • Project ownership #346
    • added new collection project
    • added new key projects to user
    • added new key projectId to folder and template-* collections
    • new mandatory /userinfo value from AAI at login time sdSubmitProjects
      • user is redirected to an info page by AAI if key is missing
    • new mandatory query parameter projectId in GET /folders
    • new mandatory JSON key projectId in POST /folders and POST /templates
    • new endpoint GET /templates to replace GET /users/current {"templates":[...]}
    • new JSON keys index and tags to PATCH /templates/schema/templateId, same values as were previously used in PATCH /user which is now removed
    • WARNING: breaking change that requires fresh database, because "project" is new information that did not exist before, and it can't be migrated to existing user-owned hierarchy
  • Multilevel add patch objects to support /extraInfo/datasetIdentifiers/- which needs dot notation for mongodb to work e.g. extraInfo.datasetIdentifiers #332

Changed

  • Refactor auth.py package by removing custom OIDC code and replacing it with https://github.com/IdentityPython/JWTConnect-Python-OidcRP. #315
    • New mandatory ENV OIDC_URL
    • New optional ENVs OIDC_SCOPE, AUTH_METHOD
    • Added oidcrp dependency
  • Use node 16+ #345
  • VScode Dev environment #287
    • Adds requirements-dev.in/txt files. Now pip dependencies can be managed with pip-tools
    • README updated with tox command, development build instructions, and prettify Dockerfile.
  • Update ENA XML and JSON schemas #299
  • Github actions changed the use of https://git.io/misspell to rojopolis/spellcheck-github-actions #316
  • Separated most of the handlers to own files inside the handlers folder #319
  • allow inserting only one study in folder #332
  • JSON schemas #332
    • introduce keywords required for Metax in doiInfo
    • dataset description and study studyAbstract are now mandatory
  • keywords will be comma separated values, that will require splitting when adding to Metax API

Fixed

  • Coveralls report #267
  • Typos for functions and tests #279
  • Fix spelling mistakes for JSON schemas #323
  • Oidcrp does not allow empty values, prefill them in mockauth so front-end can start #333
  • Fix development environment #336
    • Add env vars OIDC_URL and OIDC_URL_TEST to mock auth container
    • Adds logging configs for mock auth
    • Updates mock auth api's token endpoint with expiration configs
    • Adds config .pre-commit-config.yaml file required by pre-commit library
    • Redirect url in docker-compose is now default
    • Adds logging for doi mock api

Removed

  • Removed Authlib dependency #315
  • Project ownership #346
    • deprecated folders and templates keys from GET /users/current
      • as a side effect, deprecated items query parameter from the same endpoint
    • deprecated PATCH /user

Deprecated

  • Deprecated ENVs ISS_URL, AUTH_URL, AUTH_REFERER, JWK_URL #315

metadata-submitter v0.11.0

31 Aug 08:48
62f6747
Compare
Choose a tag to compare
  • package updates
  • Feature/sort folders #249
  • Include DOI information in the folder schema #246

metadata-submitter v0.10.0

12 Aug 07:01
4d486fa
Compare
Choose a tag to compare
  • PRs: #217 #223 #224 #243 #235 #227 #240 #233 #231
  • package updates
  • Fix misses for DAC, experiment and policy processing of XML
  • fix misses in JSON Schema
  • add integration tests for misses in dataset, experiment, policy
  • refactor analysis and experiment schemas to adhere to XML schema
  • EGA XML schemas version:1.8.0

metadata-submitter v0.9.0

22 Mar 10:57
88423b9
Compare
Choose a tag to compare

PR #203 #206 #207 #208

  • Refactor JSON schema Links
  • refactor handlers to be more streamlined
  • validate patch requests for JSON content
  • use dependabot
  • switch to python 3.8
  • support simultaneous sessions

metadata-submitter v0.8.1

15 Feb 07:31
cde5d50
Compare
Choose a tag to compare

bugfix for error pages. See #202

metadata-submitter v0.8.0

12 Feb 14:57
9353218
Compare
Choose a tag to compare

see #180 #181 #183 #185 #186 #187 #189 #190 #193 #194 #195 #197 #198 #199

  • TLS support
  • use sub as alternative to eppn to identify users
  • 'PATCHfor objects andPUT` for XML objects enabled
  • delete folders and objects associated to user on user delete
  • fix replace on json patch
  • redirect to error pages
  • general bug and fixes
  • extended integration tests

metadata-submitter v0.7.1

19 Jan 15:45
b61b7c6
Compare
Choose a tag to compare

hotfix release, see #176

Changes made

  • check objects and folders exist before any operation
  • integration check to see if deleted object or folder are still registered in db

metadata-submitter v0.7.0

06 Jan 13:07
e7d44d9
Compare
Choose a tag to compare

PRs #150 #151 #158 #162 #163 #164 #165 #166 #168 #169 #171 #172 #173

changes:

  • health endpoint
  • updated documentation
  • map users to folders
  • fix race condition in db operations
  • restrict endpoints to logged in users
  • fixes for idp and location headers redirects
  • switch to using uuids for accession ids
  • integration tests and increase unit test coverage

metadata-submitter v0.6.1

23 Nov 11:52
3864695
Compare
Choose a tag to compare

PRs #144 #142 #141 #140 #139
Includes:

  • bugfixes for login redirect
  • refactor gh actions
  • patch publish
  • #136