[SHARE-633][Fix] Parse datacite description and tags #650
Conversation
* Update datacite source.yaml * Fix description and tags parsing
share/transformers/org_datacite.py
Outdated
| description = tools.RunPython( | ||
| force_text, | ||
| tools.Try(ctx.record.metadata['oai_datacite'].payload.resource.descriptions.description[0]) | ||
| tools.Try(ctx.record.metadata['oai_datacite'].payload.resource.descriptions) |
There was a problem hiding this comment.
You might want to use tools.Join here. If there are multiple descriptions it will join them all into a single string.
* Merge branch 'develop' of https://github.com/CenterForOpenScience/SHARE into bug/SHARE-633 * Use tools.Join for joining multiple descriptions
laurenbarker
left a comment
laurenbarker
left a comment
There was a problem hiding this comment.
This look like it fixes the tags and description issue :) However, the issue might be more widespread than it originally looked like. It might be best to move the text_list and force_text functions to share/transform/chain/utils.py and write a test for them. That way the same functions can be used in multiple transformers. Right now, the transformer for datacite, oai, mods, and clinicaltrial all use some variation of force_text and it seems like that's where the issue was originating from. What do you think @chrisseto? It doesn't have to be part of this ticket but it would probably be a good idea to do before we re-transform datacite records.
| tools.Maybe(tools.Maybe(ctx.record, 'metadata')['oai_datacite'], 'type'), | ||
| tools.RunPython( | ||
| 'text_list', | ||
| (tools.Concat(tools.Try(ctx.record.metadata['oai_datacite'].payload.resource.subjects.subject))) |
There was a problem hiding this comment.
If Concat was the problem for tags, it should be removed for the subjects as well (https://github.com/CenterForOpenScience/SHARE/pull/650/files#diff-bedec94698aaa5a42507c84db3ae44d1R549).
|
https://openscience.atlassian.net/browse/SHARE-848 has been added to address the broader concerns. As soon as the subjects field is addressed this should be good to go 🎉 |
|
@laurenbarker thanks for your feedback. I updated the subject field as requested. 🐼 |
3 participants
Ticket
https://openscience.atlassian.net/browse/SHARE-633
Problem
This is a potential problem with every record that we are getting from Datacite because of an issue with the datacite transformer.
Solution: