Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

estabishmentMeans facet labels in API are JSON objects instead of strings #525

Open
nielsklazenga opened this issue Jul 25, 2022 · 1 comment
Assignees

Comments

@nielsklazenga
Copy link

This is what the facet result for establishmentMeans looks like now:

{
  "facetResults": [
    {
      "fieldName": "establishment_means",
      "fieldResult": [
        {
          "label": "{\"concept\": \"Introduced\", \"lineage\": [\"Introduced\"]}",
          "i18nCode": "establishment_means.{\"concept\": \"Introduced\", \"lineage\": [\"Introduced\"]}",
          "count": 814485,
          "fq": "establishment_means:\"{\\\"concept\\\": \\\"Introduced\\\", \\\"lineage\\\": [\\\"Introduced\\\"]}\""
        },
        {
          "label": "{\"concept\": \"Native\", \"lineage\": [\"Native\"]}",
          "i18nCode": "establishment_means.{\"concept\": \"Native\", \"lineage\": [\"Native\"]}",
          "count": 82270,
          "fq": "establishment_means:\"{\\\"concept\\\": \\\"Native\\\", \\\"lineage\\\": [\\\"Native\\\"]}\""
        },
        {
          "label": "{\"concept\": \"Uncertain\", \"lineage\": [\"Uncertain\"]}",
          "i18nCode": "establishment_means.{\"concept\": \"Uncertain\", \"lineage\": [\"Uncertain\"]}",
          "count": 459,
          "fq": "establishment_means:\"{\\\"concept\\\": \\\"Uncertain\\\", \\\"lineage\\\": [\\\"Uncertain\\\"]}\""
        },
        {
          "label": "Not supplied",
          "i18nCode": "establishment_means.novalue",
          "count": 2855500,
          "fq": "-establishment_means:*"
        }
      ]
    }
  ],
}

Both in the label and the fq are JSON strings instead of the clean values.

The same happens in the download through the API:

image

It was not possible to download establishment means before, so this is progress, but having the JSON there instead of the value makes that you cannot, for example, use the PostgreSQL COPY to load the records, but have to load the records one by one, which will be prohibitive for larger downloads (my downloads for the maps in our flora are around 3.7 million records). Downloads through the UI do not seem to have this problem.

@qifeng-bai
Copy link
Contributor

qifeng-bai commented Aug 15, 2022

https://biocache-ws-databox.ala.org.au/ws/occurrences/3afe9be0-f4fd-41d7-b6df-2a5117e08757
Valued of processed->occurrence->establishmentMeans is an json: 'establishmentMeans: "{"concept": "vagrant", "lineage": ["vagrant"]}"'

Our current EstablishmentMean model has two fields: concept and lineage, but EstablishmentMean in the latest DWC core is defined as a controlled value string
refer: https://dwc.tdwg.org/em/

It has the same issue on our Biocache prod

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants