Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support harvest dataset source identifiers that ends with a '/' #2785

Conversation

rhabbachi
Copy link
Contributor

Harvest sources with datasets identifier fields that ends with "/" (for example http://dx.doi.org/10.7927/H4PZ56R2/) will result in an empty generated identifier during the Harvest.

This PR tweaks the dkan_harvest_datajson_prepare_item_id function to account for this case and return the last non empty segment of the identifier.

How to reproduce

  1. Harvest a source that have a dataset with an identifier that is a URL that ends with a "/" (in the following example the identifier is "http://demo.getdkan.com/90a2b708-7fea-4b92-8aee-43c4cfdd5f48/").
        {
            "@type": "dcat:Dataset",
            "accessLevel": "public",
            "contactPoint": {
                "fn": "admin",
                "hasEmail": "mailto:admin@example.com"
            },
            "description": "<p>Polling places in the state of Wisconsin</p>",
            "distribution": [
                {
                    "@type": "dcat:Distribution",
                    "description": "No description provided",
                    "downloadURL": "https://s3.amazonaws.com/dkan-default-content-files/phpunit/Polling_Places_Madison_0.csv",
                    "format": "csv",
                    "mediaType": "text/csv"
                }
            ],
            "identifier": "http://demo.getdkan.com/90a2b708-7fea-4b92-8aee-43c4cfdd5f48/",
            "keyword": [
                "election"
            ],
            "language": [
                "en"
            ],
            "license": "http://opendefinition.org/licenses/cc-by/",
            "modified": "2015-07-11",
            "publisher": {
                "@type": "org:Organization",
                "name": "Geospatial Data Explorer Examples"
            },
            "title": "Wisconsin Polling Places TEST"
        },

Expected Behavior:
Harvest works.

Current Behavior:
Harvest fail and identifier recorded is empty.

QA Steps

  • Following the How to reproduce section steps should result in the source harvested with no errors.

Merge process

  • N/A.

Reminders

  • There is test for the issue.
  • Coding standards checked.
  • Review docs.getdkan.com (or in /docs) to see if it still covers the scope of the PR and update if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants