Create / Draft a DOI object capability

[tloubrieu-jpl]

See detailed design documentation https://github.com/NASA-PDS-Incubator/pds-doi-service/blob/master/docs/design/pds-doi-service-srd.md

This only include this function as a standalone python command based on a pds-data-doi module.

Applicable requirements
🦄 #5
🦄 #15

[tloubrieu-jpl]

Some inputs to start:

1. To setup the dev environment on your laptop:
   https://docs.python-guide.org/starting/install3/osx/

2. Get the project from github
   git clone https://github.com/NASA-PDS-Incubator/pds-doi-service.git

3. Install a Integrated Development Environment, some people like to have this environment. I believe this is quite important when the project you are working on is getting big: pycharm.
   You can get it from https://opencae.jpl.nasa.gov/portal/#/community/software

4. Set the python environment. This will create an isolated environment for your development:
   cd pds-doi-service
   pip install virtualenv
   python -m venv venv
   source venv/bin/activate

5. Create the structure for your code. See for example https://github.com/NASA-PDS-Incubator/pds-github-util
   Your code should be in a directory named after the module you want to develop (for example pds_dataset_doi)

6. install the python libraries you will need: lxml, pystache, ...
   pip install lxml
   pip install pystache

[tloubrieu-jpl]

The pds4 test dataset can found in https://pds.nasa.gov/datastandards/documents/examples/
With the latest version 1.13, subdirectory dph_example_archive
We will use the xml files.

[tloubrieu-jpl]

Some details on the use case we want to implement for this sprint (from Jordan's inputs):

A) Input test datasets from these PDS4 Collection and Bundle label via URLs

• Bundle - https://pds-imaging.jpl.nasa.gov/data/nsyt/insight_cameras/bundle.xml
• Data Collection - https://pds-imaging.jpl.nasa.gov/data/nsyt/insight_cameras/data/collection_data.xml
• Browse Collection - https://pds-imaging.jpl.nasa.gov/data/nsyt/insight_cameras/browse/collection_browse.xml
• Calibration Collection - https://pds-imaging.jpl.nasa.gov/data/nsyt/insight_cameras/calibration/collection_calibration.xml
• Document Collection - https://pds-imaging.jpl.nasa.gov/data/nsyt/insight_cameras/document/collection_document.xml

B) Use the latest templates provided in aaaDOI_templateFiles/DOI_IAD2_template_20181025.xml (zip file sent by Ron)

C) Defaults – some values you will see in the output are currently “defaults” set in Ron’s config file: aaaConfig_IAD2_IMG_InSight_20191216.xml

Ron used to have default configurations for metadata which he did not found in the labels, we should find how we can automatically get them or make them mandatory in the PDS4 label as part of the submission.

D) Output:
Once you translate the bundle . collection labels above to the IAD template, it should look something like the files in aaaDOI_GeneratedFiles//aaDOI_files_20191216-1/

With this inputs we don't need to run Ron's code and this is very straightforward to start the development from scratch and get us going, we can then build off this from here:

• PDS collection / bundle products on file system instead of URLs
• CSV of metadata similar to aaDOI_production_submitted_labels/GEO_reserve_shocked_feldspar_20200330/aaaSubmitted_by_GEO_20200316/DOI_Reserved_GEO_200318.xlsx
• Build the API request modules

[tloubrieu-jpl]

1. read pds4 label (lxml)
2. extract field relevant to create the OSTI record (xpath)
3. apply the OSTI template (code on slack with pystache)

[tloubrieu-jpl]

The project is one module: pds-doi-service

Submodules are:

• doi-core
• cli
• web-service

[tloubrieu-jpl]

After the meeting with Ron, I understand that the PDS4 to DOI should work with these different sub-cases:

• inputs: bundle, collection or document
• output: osti doi or osti doi reservation

We can have one reader function for each input type, for example read_pds4_bundle, read_pds4_collection, ...
They will all create the same dictionnary {"title": "some title", ...}.
The produced dictionnary can then be used with either the osti doi or the osti doi reservation template.

[jordanpadams]

closed per c1f77ab