Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initialize production deployment with pre-existing dois #71

Closed
tloubrieu-jpl opened this issue Sep 17, 2020 · 7 comments
Closed

initialize production deployment with pre-existing dois #71

tloubrieu-jpl opened this issue Sep 17, 2020 · 7 comments
Assignees
Labels
enhancement New feature or request Epic

Comments

@tloubrieu-jpl
Copy link
Member

No description provided.

@tloubrieu-jpl tloubrieu-jpl added enhancement New feature or request triage-needed labels Sep 17, 2020
@jordanpadams jordanpadams changed the title intialize production deployment with pre-existing dois initialize production deployment with pre-existing dois Sep 17, 2020
@qchaupds
Copy link

Do we have an example input for the Python code to parse from?

Currently have a one-off script to parse an input from the 'reserve' action as an exercise and code to write to database but it is unlikely the true format of the input for this ticket.

@qchaupds
Copy link

After chatting with everyone, the recommendation is to query OSTI (test and operation) for current DOI, parse the output and insert every record into database. More details coming as implementation goes.

@tloubrieu-jpl
Copy link
Member Author

To initiate node use data curator convert if necessary, otherwise use ENG.

For submitter, let it empty or if mandatory could use (pds-operator@...).

@qchaupds
Copy link

As of Tuesday afternoon of 09/29/2020 the test and operational servers at DOI has the following count of DOIs:

https://www.osti.gov/iad2test/api/records 198 DOIs
https://www.osti.gov/iad2/api/records 602 DOIs

The one-off script is able to query both servers and parse the output for metadata required for writing to local database.
Doing this allow us to flush out issues with the parsing of content coming out from the server(s) as the content varies depends on who is doing the submission.

@qchaupds
Copy link

As of this morning 09/30/2020 at 11:05 AM, the number of DOIs on test server is actually
6789

@tloubrieu-jpl
Copy link
Member Author

@qchaupds can you do a pull request for that today or early tomorrow ? Otherwise we can move this issue to next sprint, that is not a problem. Thanks

qchaupds pushed a commit that referenced this issue Oct 8, 2020
…querying for more than 25 records, fix bug with osti_web_parser when authors are not available, add logic to parse contributors and lidvid when not available, add one-off script to perform import from OSTI server to local database.
@tloubrieu-jpl
Copy link
Member Author

Delayed because the ticket still need to be reviewed and merged.

tloubrieu-jpl pushed a commit that referenced this issue Oct 9, 2020
…querying for more than 25 records, fix bug with osti_web_parser when authors are not available, add logic to parse contributors and lidvid when not available, add one-off script to perform import from OSTI server to local database.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Epic
Projects
None yet
Development

No branches or pull requests

3 participants