Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JSON-LD representations #246

Merged
merged 37 commits into from Dec 19, 2019
Merged

Conversation

alpha-beta-soup
Copy link
Contributor

This is something I've been considering for a little while. It's great for SEO that pygeoapi includes microdata read from the config in the HTML templates. However I think we could do one better by supporting JSON-LD representations of the API itself, and also extensible GeoJSON-LD representations of actual data.

I had a few goals in mind here:

  • Keep to the existing microdata as much as possible; I want this to be a minimal addition. I also like and wish to continue the use of a common vocabulary: schema.org
  • Inject JSON-LD into the HTML head, so that spatial data is indexable by search engines.
  • Achieve this injection via call to a ?f=jsonld format endpoint, so the JSON-LD data can also be obtained independently, with a response of type application/ld+json.
  • Be able to describe not just the metadata about the API instance and its collections at a catalogue level, but also reach further into the items (geojson:Feature) and collections (geojson:FeatureCollection) themselves.
    • To allow the API configuration to provide additional @context annotation for describing features unambiguously. e.g. in the sample collections/obs collection features, what do value, datetime, and stn_id represent? At present there's no way to describe these within pygeoapi. Can we allow a data provider to describe this without touching their input data?
  • I wanted to describe the bounds of each dataset

This could be taken a lot further, and I'm not proposing that that's a good idea for the core of pygeoapi. However, I think these goals are good, and generic, and that what I have here is a good first attempt at achieving them. I hope this generates some useful discussion and thank you for the consideration.

I really wasn't sure the best way to implement this, and am not at all attached to the implementation. I'm willing to re-implement it in whichever way you advise, if indeed you think conceptually this is, in whole or in part, a good idea.


API root

FireShot Capture 001 - pygeoapi default instance - Home - 172 19 0 2

{
  "@context": "http://www.schema.org",
  "@type": "DataCatalog",
  "@id": "http://172.19.0.2:5000",
  "url": "http://172.19.0.2:5000",
  "name": "pygeoapi default instance",
  "description": "pygeoapi provides an API to geospatial data",
  "keywords": [
    "geospatial",
    "data",
    "api"
  ],
  "termsOfService": null,
  "license": "https://creativecommons.org/licenses/by/4.0/",
  "provider": {
    "@type": "Organization",
    "name": "Organization Name",
    "url": "https://pygeoapi.io",
    "address": {
      "@type": "PostalAddress",
      "streetAddress": "Mailing Address",
      "postalCode": 1234,
      "addressLocality": "City",
      "addressRegion": "Administrative Area",
      "addressCountry": "Country"
    },
    "contactPoint": {
      "@type": "Contactpoint",
      "email": "you@example.org",
      "telephone": "+1-555-555-5555",
      "faxNumber": "+1-555-555-555",
      "url": "https://example.com",
      "hoursAvailable": {
        "opens": "Mo-Su",
        "description": "Open all day, everyday."
      },
      "contactType": "pointOfContact",
      "description": "Position Title"
    }
  }
}

/collections

FireShot Capture 003 - pygeoapi default instance - Collections - 172 19 0 2

JSON-LD representation
{
  "@context": "http://www.schema.org",
  "@type": "DataCatalog",
  "@id": "http://172.19.0.2:5000",
  "url": "http://172.19.0.2:5000",
  "name": "pygeoapi default instance",
  "description": "pygeoapi provides an API to geospatial data",
  "keywords": [
    "geospatial",
    "data",
    "api"
  ],
  "termsOfService": null,
  "license": "https://creativecommons.org/licenses/by/4.0/",
  "provider": {
    "@type": "Organization",
    "name": "Organization Name",
    "url": "https://pygeoapi.io",
    "address": {
      "@type": "PostalAddress",
      "streetAddress": "Mailing Address",
      "postalCode": 1234,
      "addressLocality": "City",
      "addressRegion": "Administrative Area",
      "addressCountry": "Country"
    },
    "contactPoint": {
      "@type": "Contactpoint",
      "email": "you@example.org",
      "telephone": "+1-555-555-5555",
      "faxNumber": "+1-555-555-555",
      "url": "https://example.com",
      "hoursAvailable": {
        "opens": "Mo-Su",
        "description": "Open all day, everyday."
      },
      "contactType": "pointOfContact",
      "description": "Position Title"
    }
  },
  "dataset": [
    {
      "@type": "Dataset",
      "@id": "http://172.19.0.2:5000/collections/obs",
      "name": "Observations",
      "description": "Observations",
      "license": "https://creativecommons.org/licenses/by/4.0/",
      "keywords": [
        "observations",
        "monitoring"
      ],
      "spatial": {
        "geo": {
          "@type": "GeoShape",
          "box": "-180,-90 180,90"
        }
      },
      "temporalCoverage": "2000-10-30T18:24:39/2007-10-30T08:57:29",
      "url": "http://172.19.0.2:5000/collections/obs",
      "distribution": [
        {
          "@type": "DataDownload",
          "contentURL": "https://github.com/mapserver/mapserver/blob/branch-7-0/msautotest/wxs/data/obs.csv",
          "encodingFormat": "text/csv",
          "name": "data",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "https://raw.githubusercontent.com/mapserver/mapserver/branch-7-0/msautotest/wxs/data/obs.csv",
          "encodingFormat": "text/csv",
          "name": "data",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/obs/items?f=json",
          "encodingFormat": "application/geo+json",
          "name": "Features as GeoJSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/obs/items?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "Features as RDF (GeoJSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/obs/items?f=html",
          "encodingFormat": "text/html",
          "name": "Features as HTML",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/obs?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "This document as RDF (JSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/obs?f=json",
          "encodingFormat": "application/json",
          "name": "This document as JSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/obs?f=html",
          "encodingFormat": "text/html",
          "name": "This document as HTML",
          "inLanguage": "en-US"
        }
      ]
    },
    {
      "@type": "Dataset",
      "@id": "http://172.19.0.2:5000/collections/lakes",
      "name": "Large Lakes",
      "description": "lakes of the world, public domain",
      "license": "https://creativecommons.org/licenses/by/4.0/",
      "keywords": [
        "lakes"
      ],
      "spatial": {
        "geo": {
          "@type": "GeoShape",
          "box": "-180,-90 180,90"
        }
      },
      "temporalCoverage": "2011-11-11/..",
      "url": "http://172.19.0.2:5000/collections/lakes",
      "distribution": [
        {
          "@type": "DataDownload",
          "contentURL": "http://www.naturalearthdata.com/",
          "encodingFormat": "text/html",
          "name": "information",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/lakes/items?f=json",
          "encodingFormat": "application/geo+json",
          "name": "Features as GeoJSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/lakes/items?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "Features as RDF (GeoJSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/lakes/items?f=html",
          "encodingFormat": "text/html",
          "name": "Features as HTML",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/lakes?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "This document as RDF (JSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/lakes?f=json",
          "encodingFormat": "application/json",
          "name": "This document as JSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/lakes?f=html",
          "encodingFormat": "text/html",
          "name": "This document as HTML",
          "inLanguage": "en-US"
        }
      ]
    },
    {
      "@type": "Dataset",
      "@id": "http://172.19.0.2:5000/collections/countries",
      "name": "Countries in the world (SpatialLite Provider)",
      "description": "Countries of the world (SpatialLite)",
      "license": "https://creativecommons.org/licenses/by/4.0/",
      "keywords": [
        "countries",
        "natural eart"
      ],
      "spatial": {
        "geo": {
          "@type": "GeoShape",
          "box": "-180,-90 180,90"
        }
      },
      "temporalCoverage": "../..",
      "url": "http://172.19.0.2:5000/collections/countries",
      "distribution": [
        {
          "@type": "DataDownload",
          "contentURL": "http://www.naturalearthdata.com/",
          "encodingFormat": "text/html",
          "name": "information",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/countries/items?f=json",
          "encodingFormat": "application/geo+json",
          "name": "Features as GeoJSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/countries/items?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "Features as RDF (GeoJSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/countries/items?f=html",
          "encodingFormat": "text/html",
          "name": "Features as HTML",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/countries?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "This document as RDF (JSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/countries?f=json",
          "encodingFormat": "application/json",
          "name": "This document as JSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/countries?f=html",
          "encodingFormat": "text/html",
          "name": "This document as HTML",
          "inLanguage": "en-US"
        }
      ]
    },
    {
      "@type": "Dataset",
      "@id": "http://172.19.0.2:5000/collections/dutch_georef_stations",
      "name": "Dutch Georef Stations via OGR WFS",
      "description": "Locations of RD/GNSS-reference stations from Dutch Kadaster PDOK a.k.a RDInfo. Uses MapServer WFS v2 backend via OGRProvider.",
      "license": "https://creativecommons.org/licenses/by/4.0/",
      "keywords": [
        "Netherlands",
        "GNSS",
        "Surveying",
        "Holland",
        "RD"
      ],
      "spatial": {
        "geo": {
          "@type": "GeoShape",
          "box": "50.7539,7.21097 53.4658,3.37087"
        }
      },
      "temporalCoverage": "../..",
      "url": "http://172.19.0.2:5000/collections/dutch_georef_stations",
      "distribution": [
        {
          "@type": "DataDownload",
          "contentURL": "http://www.nationaalgeoregister.nl/geonetwork/srv/dut/catalog.search#/metadata/3ebe56dc-5f09-4fb3-b224-55c2db4ca2fd?tab=general",
          "encodingFormat": "text/html",
          "name": "information",
          "inLanguage": "nl-NL"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/dutch_georef_stations/items?f=json",
          "encodingFormat": "application/geo+json",
          "name": "Features as GeoJSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/dutch_georef_stations/items?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "Features as RDF (GeoJSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/dutch_georef_stations/items?f=html",
          "encodingFormat": "text/html",
          "name": "Features as HTML",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/dutch_georef_stations?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "This document as RDF (JSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/dutch_georef_stations?f=json",
          "encodingFormat": "application/json",
          "name": "This document as JSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/dutch_georef_stations?f=html",
          "encodingFormat": "text/html",
          "name": "This document as HTML",
          "inLanguage": "en-US"
        }
      ]
    },
    {
      "@type": "Dataset",
      "@id": "http://172.19.0.2:5000/collections/utah_city_locations",
      "name": "Cities in Utah via OGR WFS",
      "description": "Data from the state of Utah. Standard demo dataset from the deegree WFS server that is used as backend WFS.",
      "license": "https://creativecommons.org/licenses/by/4.0/",
      "keywords": [
        "USA",
        "deegree",
        "Utah",
        "Demo data"
      ],
      "spatial": {
        "geo": {
          "@type": "GeoShape",
          "box": "-112.108489,39.854053 -111.028628,40.460098"
        }
      },
      "temporalCoverage": "../..",
      "url": "http://172.19.0.2:5000/collections/utah_city_locations",
      "distribution": [
        {
          "@type": "DataDownload",
          "contentURL": "http://download.deegree.org/documentation/3.3.20/html/lightly.html#example-workspace-2-utah-webmapping-services",
          "encodingFormat": "text/html",
          "name": "information",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/utah_city_locations/items?f=json",
          "encodingFormat": "application/geo+json",
          "name": "Features as GeoJSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/utah_city_locations/items?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "Features as RDF (GeoJSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/utah_city_locations/items?f=html",
          "encodingFormat": "text/html",
          "name": "Features as HTML",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/utah_city_locations?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "This document as RDF (JSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/utah_city_locations?f=json",
          "encodingFormat": "application/json",
          "name": "This document as JSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/utah_city_locations?f=html",
          "encodingFormat": "text/html",
          "name": "This document as HTML",
          "inLanguage": "en-US"
        }
      ]
    },
    {
      "@type": "Dataset",
      "@id": "http://172.19.0.2:5000/collections/unesco_pois_italy",
      "name": "Unesco POIs in Italy via OGR WFS",
      "description": "Unesco Points of Interest in Italy. Using GeoSolutions GeoServer WFS demo-server as backend WFS.",
      "license": "https://creativecommons.org/licenses/by/4.0/",
      "keywords": [
        "Italy",
        "Unesco",
        "Demo"
      ],
      "spatial": {
        "geo": {
          "@type": "GeoShape",
          "box": "36.0,17.0 46.0,18.0"
        }
      },
      "temporalCoverage": "../..",
      "url": "http://172.19.0.2:5000/collections/unesco_pois_italy",
      "distribution": [
        {
          "@type": "DataDownload",
          "contentURL": "https://mapstore2.geo-solutions.it/mapstore/#/dashboard/5593",
          "encodingFormat": "text/html",
          "name": "information",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/unesco_pois_italy/items?f=json",
          "encodingFormat": "application/geo+json",
          "name": "Features as GeoJSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/unesco_pois_italy/items?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "Features as RDF (GeoJSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/unesco_pois_italy/items?f=html",
          "encodingFormat": "text/html",
          "name": "Features as HTML",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/unesco_pois_italy?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "This document as RDF (JSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/unesco_pois_italy?f=json",
          "encodingFormat": "application/json",
          "name": "This document as JSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/unesco_pois_italy?f=html",
          "encodingFormat": "text/html",
          "name": "This document as HTML",
          "inLanguage": "en-US"
        }
      ]
    },
    {
      "@type": "Dataset",
      "@id": "http://172.19.0.2:5000/collections/ogr_gpkg_poi",
      "name": "Portuguese Points of Interest via OGR GPKG",
      "description": "Portuguese Points of Interest obtained from OpenStreetMap. Dataset includes Madeira and Azores islands. Uses GeoPackage backend via OGR provider.",
      "license": "https://creativecommons.org/licenses/by/4.0/",
      "keywords": [
        "Portugal",
        "POI",
        "Point of Interrest",
        "Madeira",
        "Azores",
        "OSM",
        "Open Street Map",
        "NaturaGIS"
      ],
      "spatial": {
        "geo": {
          "@type": "GeoShape",
          "box": "-31.2687,32.5898 -6.18992,42.152"
        }
      },
      "temporalCoverage": "../..",
      "url": "http://172.19.0.2:5000/collections/ogr_gpkg_poi",
      "distribution": [
        {
          "@type": "DataDownload",
          "contentURL": "https://wiki.openstreetmap.org/wiki/Points_of_interest/",
          "encodingFormat": "text/html",
          "name": "information",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_gpkg_poi/items?f=json",
          "encodingFormat": "application/geo+json",
          "name": "Features as GeoJSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_gpkg_poi/items?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "Features as RDF (GeoJSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_gpkg_poi/items?f=html",
          "encodingFormat": "text/html",
          "name": "Features as HTML",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_gpkg_poi?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "This document as RDF (JSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_gpkg_poi?f=json",
          "encodingFormat": "application/json",
          "name": "This document as JSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_gpkg_poi?f=html",
          "encodingFormat": "text/html",
          "name": "This document as HTML",
          "inLanguage": "en-US"
        }
      ]
    },
    {
      "@type": "Dataset",
      "@id": "http://172.19.0.2:5000/collections/ogr_geojson_lakes",
      "name": "Large Lakes OGR GeoJSON Driver",
      "description": "lakes of the world, public domain",
      "license": "https://creativecommons.org/licenses/by/4.0/",
      "keywords": [
        "lakes"
      ],
      "spatial": {
        "geo": {
          "@type": "GeoShape",
          "box": "-180,-90 180,90"
        }
      },
      "temporalCoverage": "2011-11-11/..",
      "url": "http://172.19.0.2:5000/collections/ogr_geojson_lakes",
      "distribution": [
        {
          "@type": "DataDownload",
          "contentURL": "http://www.naturalearthdata.com/",
          "encodingFormat": "text/html",
          "name": "information",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_geojson_lakes/items?f=json",
          "encodingFormat": "application/geo+json",
          "name": "Features as GeoJSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_geojson_lakes/items?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "Features as RDF (GeoJSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_geojson_lakes/items?f=html",
          "encodingFormat": "text/html",
          "name": "Features as HTML",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_geojson_lakes?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "This document as RDF (JSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_geojson_lakes?f=json",
          "encodingFormat": "application/json",
          "name": "This document as JSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_geojson_lakes?f=html",
          "encodingFormat": "text/html",
          "name": "This document as HTML",
          "inLanguage": "en-US"
        }
      ]
    },
    {
      "@type": "Dataset",
      "@id": "http://172.19.0.2:5000/collections/ogr_addresses_sqlite",
      "name": "Dutch addresses (subset Otterlo). OGR SQLite Driver",
      "description": "Dutch addresses subset.",
      "license": "https://creativecommons.org/licenses/by/4.0/",
      "keywords": [
        "Netherlands",
        "addresses",
        "INSPIRE"
      ],
      "spatial": {
        "geo": {
          "@type": "GeoShape",
          "box": "50.7539,7.21097 53.4658,3.37087"
        }
      },
      "temporalCoverage": "../..",
      "url": "http://172.19.0.2:5000/collections/ogr_addresses_sqlite",
      "distribution": [
        {
          "@type": "DataDownload",
          "contentURL": "http://www.nationaalgeoregister.nl/geonetwork/srv/dut/catalog.search#/metadata/4074b3c3-ca85-45ad-bc0d-b5fca8540z0b",
          "encodingFormat": "text/html",
          "name": "information",
          "inLanguage": "nl-NL"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_addresses_sqlite/items?f=json",
          "encodingFormat": "application/geo+json",
          "name": "Features as GeoJSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_addresses_sqlite/items?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "Features as RDF (GeoJSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_addresses_sqlite/items?f=html",
          "encodingFormat": "text/html",
          "name": "Features as HTML",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_addresses_sqlite?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "This document as RDF (JSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_addresses_sqlite?f=json",
          "encodingFormat": "application/json",
          "name": "This document as JSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_addresses_sqlite?f=html",
          "encodingFormat": "text/html",
          "name": "This document as HTML",
          "inLanguage": "en-US"
        }
      ]
    },
    {
      "@type": "Dataset",
      "@id": "http://172.19.0.2:5000/collections/ogr_addresses_gpkg",
      "name": "Dutch addresses (subset Otterlo). OGR GeoPackage Driver",
      "description": "Dutch addresses subset.",
      "license": "https://creativecommons.org/licenses/by/4.0/",
      "keywords": [
        "Netherlands",
        "addresses",
        "INSPIRE"
      ],
      "spatial": {
        "geo": {
          "@type": "GeoShape",
          "box": "50.7539,7.21097 53.4658,3.37087"
        }
      },
      "temporalCoverage": "../..",
      "url": "http://172.19.0.2:5000/collections/ogr_addresses_gpkg",
      "distribution": [
        {
          "@type": "DataDownload",
          "contentURL": "http://www.nationaalgeoregister.nl/geonetwork/srv/dut/catalog.search#/metadata/4074b3c3-ca85-45ad-bc0d-b5fca8540z0b",
          "encodingFormat": "text/html",
          "name": "information",
          "inLanguage": "nl-NL"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_addresses_gpkg/items?f=json",
          "encodingFormat": "application/geo+json",
          "name": "Features as GeoJSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_addresses_gpkg/items?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "Features as RDF (GeoJSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_addresses_gpkg/items?f=html",
          "encodingFormat": "text/html",
          "name": "Features as HTML",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_addresses_gpkg?f=jsonld",
          "encodingFormat": "application/ld+json",
          "name": "This document as RDF (JSON-LD)",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_addresses_gpkg?f=json",
          "encodingFormat": "application/json",
          "name": "This document as JSON",
          "inLanguage": "en-US"
        },
        {
          "@type": "DataDownload",
          "contentURL": "http://172.19.0.2:5000/collections/ogr_addresses_gpkg?f=html",
          "encodingFormat": "text/html",
          "name": "This document as HTML",
          "inLanguage": "en-US"
        }
      ]
    }
  ]
}

/collections/obs

Note that here I have included an additional property in the dataset configuration, which, in its entirety, looks like:

    obs:
        title: Observations
        description: Observations
        keywords:
            - observations
            - monitoring
        crs:
            - CRS84
        context:
            - datetime: https://schema.org/DateTime
            - vocab: https://example.com/vocab#
              stn_id: "vocab:stn_id"
              value: "vocab:value"
        links:
            - type: text/csv
              rel: canonical
              title: data
              href: https://github.com/mapserver/mapserver/blob/branch-7-0/msautotest/wxs/data/obs.csv
              hreflang: en-US
            - type: text/csv
              rel: alternate
              title: data
              href: https://raw.githubusercontent.com/mapserver/mapserver/branch-7-0/msautotest/wxs/data/obs.csv
              hreflang: en-US
        extents:
            spatial:
                bbox: [-180,-90,180,90]
            temporal:
                begin: 2000-10-30T18:24:39Z
                end: 2007-10-30T08:57:29Z
        provider:
            name: CSV
            data: tests/data/obs.csv
            id_field: id
            geometry:
                x_field: long
                y_field: lat

The only additional part is the context. It produces a JSON-LD representation like this:

{
  "@context": [
    "https://geojson.org/geojson-ld/geojson-context.jsonld",
    {
      "datetime": "https://schema.org/DateTime"
    },
    {
      "vocab": "https://domain.com/vocab#",
      "stn_id": "vocab:stn_id",
      "value": "vocab:value"
    }
  ],
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "id": "http://172.19.0.2:5000/collections/obs/items/371",
      "geometry": {
        "type": "Point",
        "coordinates": [
          -75,
          45
        ]
      },
      "properties": {
        "stn_id": "35",
        "datetime": "2001-10-30T14:24:55Z",
        "value": "89.9"
      }
    },
    {
      "type": "Feature",
      "id": "http://172.19.0.2:5000/collections/obs/items/377",
      "geometry": {
        "type": "Point",
        "coordinates": [
          -75,
          45
        ]
      },
      "properties": {
        "stn_id": "35",
        "datetime": "2002-10-30T18:31:38Z",
        "value": "93.9"
      }
    }
  ],
  "numberMatched": 2,
  "numberReturned": 2,
  "links": [
    {
      "type": "application/geo+json",
      "rel": "alternate",
      "title": "This document as GeoJSON",
      "href": "http://172.19.0.2:5000/collections/obs/items?f=json"
    },
    {
      "rel": "self",
      "type": "application/ld+json",
      "title": "This document as RDF (JSON-LD)",
      "href": "http://172.19.0.2:5000/collections/obs/items?f=jsonld"
    },
    {
      "type": "text/html",
      "rel": "alternate",
      "title": "This document as HTML",
      "href": "http://172.19.0.2:5000/collections/obs/items?f=html"
    },
    {
      "type": "application/geo+json",
      "rel": "prev",
      "title": "items (prev)",
      "href": "http://172.19.0.2:5000/collections/obs/items/?startindex=0"
    },
    {
      "type": "application/geo+json",
      "rel": "next",
      "title": "items (next)",
      "href": "http://172.19.0.2:5000/collections/obs/items/?startindex=10"
    },
    {
      "type": "application/json",
      "title": "Observations",
      "rel": "collection",
      "href": "http://172.19.0.2:5000/collections/obs"
    }
  ],
  "timeStamp": "2019-09-24T03:46:21.503676Z",
  "id": "http://172.19.0.2:5000/collections/obs/items"
}

The GeoJSON default vocabulary is included compulsorily.

If we compact that with the JSON-LD Playground, we get:

{
  "@id": "http://172.19.0.2:5000/collections/obs/items",
  "@type": "https://purl.org/geojson/vocab#FeatureCollection",
  "https://purl.org/geojson/vocab#features": [
    {
      "@id": "http://172.19.0.2:5000/collections/obs/items/371",
      "@type": "https://purl.org/geojson/vocab#Feature",
      "https://purl.org/geojson/vocab#geometry": {
        "@type": "https://purl.org/geojson/vocab#Point",
        "https://purl.org/geojson/vocab#coordinates": {
          "@list": [
            -75,
            45
          ]
        }
      },
      "https://purl.org/geojson/vocab#properties": {
        "https://domain.com/vocab#stn_id": "35",
        "https://domain.com/vocab#value": "89.9",
        "https://schema.org/DateTime": "2001-10-30T14:24:55Z"
      }
    },
    {
      "@id": "http://172.19.0.2:5000/collections/obs/items/377",
      "@type": "https://purl.org/geojson/vocab#Feature",
      "https://purl.org/geojson/vocab#geometry": {
        "@type": "https://purl.org/geojson/vocab#Point",
        "https://purl.org/geojson/vocab#coordinates": {
          "@list": [
            -75,
            45
          ]
        }
      },
      "https://purl.org/geojson/vocab#properties": {
        "https://domain.com/vocab#stn_id": "35",
        "https://domain.com/vocab#value": "93.9",
        "https://schema.org/DateTime": "2002-10-30T18:31:38Z"
      }
    }
  ]
}

Thus we can see finally that what was originally datetime in the input CSV is now fully elaborated as https://schema.org/DateTime.

@alpha-beta-soup
Copy link
Contributor Author

Oh, and I haven't considered testing yet.

@tomkralidis tomkralidis self-requested a review September 24, 2019 10:59
@alpha-beta-soup
Copy link
Contributor Author

Just an example of one place this is interesting to me.

With a configuration like:

datasets:
    obs:
        title: Observations
        description: Observations
        keywords:
            - observations
            - monitoring
        crs:
            - CRS84
        context:
            - datetime: https://schema.org/DateTime
            - vocab: https://example.com/vocab#
              stn_id: "vocab:stn_id"
              value: "vocab:value"
            - schema: "https://schema.org/"
              name:
                "@id": "schema:name"
                "@language": "en"
              name-fr:
                "@id": "schema:name"
                "@language": "fr"
        links:
            - type: text/csv
              rel: canonical
              title: data
              href: https://github.com/mapserver/mapserver/blob/branch-7-0/msautotest/wxs/data/obs.csv
              hreflang: en-US
            - type: text/csv
              rel: alternate
              title: data
              href: https://raw.githubusercontent.com/mapserver/mapserver/branch-7-0/msautotest/wxs/data/obs.csv
              hreflang: en-US
        extents:
            spatial:
                bbox: [-180, -90, 180, 90]
            temporal:
                begin: 2000-10-30T18:24:39Z
                end: 2007-10-30T08:57:29Z
        provider:
            name: CSV
            data: tests/data/obs.csv
            id_field: id
            geometry:
                x_field: long
                y_field: lat

You can produce output that is explicit about the languages property values are expressed in; e.g. this configuration can produce:

{
  "@context": [
    "https://geojson.org/geojson-ld/geojson-context.jsonld",
    {
      "datetime": "https://schema.org/DateTime"
    },
    {
      "vocab": "https://example.com/vocab#",
      "stn_id": "vocab:stn_id",
      "value": "vocab:value"
    },
    {
      "schema": "https://schema.org/",
      "name": {
        "@id": "schema:name",
        "@language": "en"
      },
      "name-fr": {
        "@id": "schema:name",
        "@language": "fr"
      }
    }
  ],
  "type": "Feature",
  "id": "http://172.19.0.2:5000/collections/obs/items/371",
  "geometry": {
    "type": "Point",
    "coordinates": [
      -75,
      45
    ]
  },
  "properties": {
    "stn_id": "35",
    "datetime": "2001-10-30T14:24:55Z",
    "value": "89.9",
    "name": "Foobar",
    "name-fr": "Foobaré"
  },
  "links": [
    {
      "rel": "alternate",
      "type": "application/geo+json",
      "title": "This document as GeoJSON",
      "href": "http://172.19.0.2:5000/collections/obs/items/371?f=json"
    },
    {
      "rel": "self",
      "type": "application/ld+json",
      "title": "This document as RDF (JSON-LD)",
      "href": "http://172.19.0.2:5000/collections/obs/items/371?f=jsonld"
    },
    {
      "rel": "alternate",
      "type": "text/html",
      "title": "This document as HTML",
      "href": "http://172.19.0.2:5000/collections/obs/items/371?f=html"
    },
    {
      "rel": "collection",
      "type": "application/json",
      "title": "Observations",
      "href": "http://172.19.0.2:5000/collections/obs"
    },
    {
      "rel": "prev",
      "type": "application/geo+json",
      "href": "http://172.19.0.2:5000/collections/obs/items/371"
    },
    {
      "rel": "next",
      "type": "application/geo+json",
      "href": "http://172.19.0.2:5000/collections/obs/items/371"
    }
  ]
}

Compacted:

{
  "@id": "http://172.19.0.2:5000/collections/obs/items/371",
  "@type": "https://purl.org/geojson/vocab#Feature",
  "https://purl.org/geojson/vocab#geometry": {
    "@type": "https://purl.org/geojson/vocab#Point",
    "https://purl.org/geojson/vocab#coordinates": {
      "@list": [
        -75,
        45
      ]
    }
  },
  "https://purl.org/geojson/vocab#properties": {
    "https://example.com/vocab#stn_id": "35",
    "https://example.com/vocab#value": "89.9",
    "https://schema.org/DateTime": "2001-10-30T14:24:55Z",
    "https://schema.org/name": [
      {
        "@language": "en",
        "@value": "Foobar"
      },
      {
        "@language": "fr",
        "@value": "Foobaré"
      }
    ]
  }
}

@jorgejesus jorgejesus self-requested a review September 25, 2019 06:36
@tomkralidis
Copy link
Member

@alpha-beta-soup I'll start taking a deeper look in a week or so, but this is great work! As a first step I would recommend bringing the PR up to date with recent changes in master branch (#244) - which includes moving temporal extent types from None to null per #237 (comment)).

@pvgenuchten
Copy link
Contributor

Hi @alpha-beta-soup, thank you for this big work. At geonetwork we also decided to move to embedded json-ld. Maintaining schema.org annotations within html has quite regression risk and is probably also less flexible. Now that json-ld gets fortunately more common in the geo apis, this is a sensible step.

Some comments on the work:

  • would it be helpful to use a json-ld framework to render json-ld from python data structures?
  • would a python method be able to introduce the json-ld directly in the output, in stead of loading it with ajax (google supports ajax, but do others also?)
  • you suggest the use of concepts such as features and geojson, but for this use case would schema.org/place and schema.org/coordinates not be more relevant? I wonder if it would be possible in json-ld to tag for example a coordinate value with multiple concepts from multiple ontologies (and if search engines would still understand that).

@alpha-beta-soup
Copy link
Contributor Author

alpha-beta-soup commented Sep 25, 2019

@pvgenuchten

would it be helpful to use a json-ld framework to render json-ld from python data structures?

I didn't even consider that, but I think that's an excellent idea. I haven't worked with structured/linked data for long, so I really appreciate that pointer. I'll look at making that adjustment, once I've heard some other feedback also.

would a python method be able to introduce the json-ld directly in the output, in stead of loading it with ajax (google supports ajax, but do others also?)

Yes, and I experimented with that successfully. I was uneasy about the trade-off: an asynchronous call to the API for the JSON-LD representation should mean the Time to Interactive is smaller than if the server also had to wait for the JSON-LD before returning anything. I figured that since Googlebot
(and presumably other robots) now also executes JavaScript, that the benefit of the latter was moot. For clients which ultimately only want JSON-LD, the direct request is possible.

you suggest the use of concepts such as features and geojson, but for this use case would schema.org/place and schema.org/coordinates not be more relevant? I wonder if it would be possible in json-ld to tag for example a coordinate value with multiple concepts from multiple ontologies (and if search engines would still understand that).

Yes, I'm all for the ability to include multiple representations of the geometry, for a number of reasons: generalisation, scale, topology, dimensionality, perhaps even supporting true curves. My intention at first is to simply annotate what exists rather than to extend it, and I suggest that this is something to seriously consider at a later point once we're over the first bar. That said, maybe using schema:place and schema:coordinates is still useful at this stage since we're already operating in that space, but it's still an extension over and above pygeoapi's existing use of GeoJSON from my perspective.

@jorgejesus
Copy link
Member

@alpha-beta-soup Could you join us in the gitter (https://gitter.im/geopython/pygeoapi) we started to discuss the PR

@alpha-beta-soup
Copy link
Contributor Author

alpha-beta-soup commented Oct 4, 2019

I've been considering how to represent more complex information in pygeoapi/JSON-LD, leveraging the PostgreSQL driver as a read input source.

Given a table view defined as:

CREATE OR REPLACE VIEW elfie.samples AS 
SELECT
sf.gml_identifier AS "@id",
sf.featuretype_title AS "@type",
sf.gml_name AS "name",
sf.sams_shape_xy AS geom,
json_build_object(
	'@type', 'gsp:Geometry',
	'asWKT', (ST_AsEWKT(sf.sams_shape_xy))
) AS "hasGeometry",
json_build_object(
		'@id', so.gml_identifier,
		'@type', so.featuretype_title,
		'name', null --TODO so.gml_name
) AS "isSampleOf",
json_agg(
  DISTINCT jsonb_build_object(
  	'@id', om.gml_identifier,
	'@type', om.featuretype_title,
	'phenomenonTime', json_build_object(
		'@id', om.om_phenomenontime_href,
		'name', om.om_phenomenontime_title
	),
	'resultTime', json_build_object(
		'@id', om.om_resulttime_href,
		'name', om.om_resulttime_title
	),
	'usedProcedure', json_build_object(
		'@id', om.om_procedure_href,
		'name', om.om_procedure_title
	),
	'observedProperty', json_build_object(
		'@id', om.om_observedproperty_href,
		'name', om.om_observedproperty_title
	),
	'hasFeatureOfInterest', json_build_object(
		'@id', sf.gml_identifier,
		'@type', sf.featuretype_title
	),
	'hasResult',
	    CASE WHEN om.om_result_concept_id IS NOT NULL THEN json_build_object(
			'@id', cl.gml_identifier,
			'name', cl.gml_name
		) WHEN om.om_result_string IS NOT NULL THEN json_build_object(
			'@value', om.om_result_string
		) WHEN om.om_result_measure IS NOT NULL THEN json_build_object(
			'@value', om.om_result_measure,
			'uom', om.om_result_uom
		) ELSE NULL END
  	)
) AS "isFeatureOfInterestOf",
json_agg(
	DISTINCT jsonb_build_object(
		'natureOfRelationship', json_build_object(
			'@id', 'https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample',
			'name', 'sample'
		),
		'relatedSample', json_build_object(
			'@id', sp.gml_identifier,
			'name', sp.gml_name
		)
	)
) AS "hasSampleRelationship"

FROM elfie.sf_spatialsamplingfeature AS sf

JOIN elfie.om_observation_site AS om
ON sf._id = om.sf_samplingfeature_id

JOIN elfie.so_soil AS so
ON sf._id = so.sf_samplingfeature_id

JOIN elfie.sam_samplingfeaturecomplex
AS sfc ON sf._id = sfc.sourcefeature_id

JOIN elfie.sf_specimen AS sp
ON sp._id = sfc.targetfeature_id

LEFT JOIN elfie.cl_concept AS cl
ON om.om_result_concept_id = cl._id
AND so.soclassifier_id = cl._id

WHERE sf.sams_shape_xy IS NOT NULL
GROUP BY (sf.gml_identifier, sf.featuretype_title, sf.gml_name, sf.sams_shape_xy),
(so.gml_identifier, so.featuretype_title) --isSampleOf
ORDER BY sf.gml_identifier ASC;

And pygeoapi config like:

    soilsamples:
       title: Landcare Spatial Sampling Features
       description: Spatial sampling features
       keywords:
           - Sampling
           - Samples
           - Landcare Research
           - Manaaki Whenua
           - Soil
           - New Zealand
       crs:
           - CRS84
       links:
           - type: text/html
             rel: canonical
             title: information
             href: https://lab.scinfo.org.nz
             hreflang: en-NZ
       extents:
           spatial:
               bbox: [169.890834606643, -44.6476190560794, 178.761531832564, -35.5201500812183]
               crs: http://www.opengis.net/def/crs/OGC/1.3/CRS84
           temporal:
               begin:
               end:
       context:
         - https://opengeospatial.github.io/ELFIE/json-ld/sosa.jsonld
         - https://opengeospatial.github.io/ELFIE/json-ld/soilie.jsonld
         - https://opengeospatial.github.io/ELFIE/json-ld/skos.jsonld
         - gsp: http://www.opengis.net/ont/geosparql#
           schema: "https://schema.org/"
           name: "schema:name"
           asWKT: gsp:asWKT
       provider:
           name: PostgreSQL
           data:
               host: redacted
               dbname: redacted
               user: redacted
               password: redacted
               port: redacted
               schema: public,elfie # See issue: https://github.com/geopython/pygeoapi/issues/269
           id_field: "@id"
           table: samples
           context: query

(At this point there is an apparent bug, but it's not critical: #269)

Then I can return some rather complex JSON-LD, expressing relationships as well as ordinary properties:

Click to expand long (Geo)JSON-LD FeatureCollection
{
  "@context": [
    "https://geojson.org/geojson-ld/geojson-context.jsonld",
    "https://opengeospatial.github.io/ELFIE/json-ld/sosa.jsonld",
    "https://opengeospatial.github.io/ELFIE/json-ld/soilie.jsonld",
    "https://opengeospatial.github.io/ELFIE/json-ld/skos.jsonld",
    {
      "gsp": "http://www.opengis.net/ont/geosparql#",
      "schema": "https://schema.org/",
      "name": "schema:name",
      "asWKT": "gsp:asWKT"
    }
  ],
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "geometry": {
        "type": "Point",
        "coordinates": [
          174.882920346082,
          -37.0781263435251
        ]
      },
      "properties": {
        "@id": "https://lab.scinfo.org.nz/elfie/id/sams/sf_spatialsamplingfeature/55-42-00001",
        "@type": "sams:SF_SpatialSamplingFeature",
        "name": "nsdr:55.42.00001",
        "isSampleOf": {
          "@id": "https://lab.scinfo.org.nz/elfie/id/soil/so_soil/47-55-42-00001-140253",
          "@type": "soil:SO_Soil",
          "name": null
        },
        "hasGeometry": {
          "@type": "gsp:Geometry",
          "asWKT": "SRID=4326;POINT(174.882920346082 -37.0781263435251)"
        },
        "hasSampleRelationship": [
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-42-00011",
              "name": "nsdr:82.42.00011"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-42-10001a",
              "name": "nsdr:82.42.10001a"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-42-10001b",
              "name": "nsdr:82.42.10001b"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-42-10001c",
              "name": "nsdr:82.42.10001c"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-lab-0415a180-8032-4942-8ac3-33f19fb94aa0",
              "name": "nsdr:82.45.15373.19439.348"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-lab-47178795-5fac-4cae-9a39-839e6c1df774",
              "name": "nsdr:82.45.15373.19442.039"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-lab-617520b3-a110-4348-9f88-3c06d4cbbf70",
              "name": "nsdr:82.45.15373.19439.205"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-lab-83c99cd8-18ee-4228-baae-74bdb073852f",
              "name": "nsdr:82.45.15373.19442.182"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-lab-8bcbbc4c-b9b1-4a86-9379-a0550a242c18",
              "name": "nsdr:82.45.15373.19440.091"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-lab-c0b3fb93-8f91-4a5c-ac98-e67c02d0e5cb",
              "name": "nsdr:82.45.15373.19439.952"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-lab-dcc5d412-faa2-447f-929a-c6cb55684c64",
              "name": "nsdr:82.45.15373.19439.482"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-lab-f1cfff76-d963-472c-8020-0bc0c52550f8",
              "name": "nsdr:82.45.15373.19439.811"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          }
        ],
        "isFeatureOfInterestOf": [
          {
            "@id": "https://lab.scinfo.org.nz/elfie/id/om/om_observation/47-42-00001-1664-3110",
            "@type": "om:OM_Observation",
            "hasResult": {
              "@id": "https://lab.scinfo.org.nz/elfie/def/nsdr/cl_concept/1048-140253",
              "name": "Typic Orthic Allophanic Soil"
            },
            "resultTime": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "usedProcedure": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "phenomenonTime": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "observedProperty": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/property/soclassifier",
              "name": "classifier"
            },
            "hasFeatureOfInterest": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/sams/sf_spatialsamplingfeature/55-42-00001",
              "@type": "sams:SF_SpatialSamplingFeature"
            }
          },
          {
            "@id": "https://lab.scinfo.org.nz/elfie/id/om/om_observation/78-42-00001-1726",
            "@type": "om:OM_Observation",
            "hasResult": {
              "uom": "degrees",
              "@value": 0
            },
            "resultTime": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "usedProcedure": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "phenomenonTime": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "observedProperty": {
              "@id": "https://lab.scinfo.org.nz/soil/def/property/slope-angle",
              "name": "slope angle"
            },
            "hasFeatureOfInterest": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/sams/sf_spatialsamplingfeature/55-42-00001",
              "@type": "sams:SF_SpatialSamplingFeature"
            }
          },
          {
            "@id": "https://lab.scinfo.org.nz/elfie/id/om/om_observation/88-42-00001-1669",
            "@type": "om:OM_Observation",
            "hasResult": {
              "@value": "Poor pasture, hay paddock"
            },
            "resultTime": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "usedProcedure": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "phenomenonTime": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "observedProperty": {
              "@id": "http://purl.org/dc/terms/description",
              "name": "description"
            },
            "hasFeatureOfInterest": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/sams/sf_spatialsamplingfeature/55-42-00001",
              "@type": "sams:SF_SpatialSamplingFeature"
            }
          }
        ]
      },
      "id": "https://lab.scinfo.org.nz/elfie/id/sams/sf_spatialsamplingfeature/55-42-00001"
    },
    {
      "type": "Feature",
      "geometry": {
        "type": "Point",
        "coordinates": [
          174.882920346082,
          -37.0781263435251
        ]
      },
      "properties": {
        "@id": "https://lab.scinfo.org.nz/elfie/id/sams/sf_spatialsamplingfeature/55-42-00002",
        "@type": "sams:SF_SpatialSamplingFeature",
        "name": "nsdr:55.42.00002",
        "isSampleOf": {
          "@id": "https://lab.scinfo.org.nz/elfie/id/soil/so_soil/47-55-42-00002-140253",
          "@type": "soil:SO_Soil",
          "name": null
        },
        "hasGeometry": {
          "@type": "gsp:Geometry",
          "asWKT": "SRID=4326;POINT(174.882920346082 -37.0781263435251)"
        },
        "hasSampleRelationship": [
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-42-00012",
              "name": "nsdr:82.42.00012"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-42-10002a",
              "name": "nsdr:82.42.10002a"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-42-10002b",
              "name": "nsdr:82.42.10002b"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-42-10002c",
              "name": "nsdr:82.42.10002c"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-lab-0dc39d56-453b-428f-8133-388efaebe083",
              "name": "nsdr:82.45.15373.19442.739"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-lab-3c017728-5345-4552-8d3d-6cc2b843af45",
              "name": "nsdr:82.45.15373.19442.461"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-lab-5e08e3a0-7ce5-4c34-a654-176d8af8c615",
              "name": "nsdr:82.45.15373.19442.602"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-lab-baa2fc38-80bb-4cd2-b6ed-bdb454b5a999",
              "name": "nsdr:82.45.15373.19443.031"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-lab-ca6c63dd-2ae8-4352-b694-2e136da404bf",
              "name": "nsdr:82.45.15373.19443.319"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-lab-d8682ca6-14a2-49bd-89c7-683fcfa6629d",
              "name": "nsdr:82.45.15373.19443.177"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-lab-dda07e70-86b5-41ce-ae60-c23f6e6ec664",
              "name": "nsdr:82.45.15373.19443.461"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          },
          {
            "relatedSample": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/spec/sf_specimen/82-lab-fdd2f2dd-57ae-447c-9d04-151619e5b548",
              "name": "nsdr:82.45.15373.19442.881"
            },
            "natureOfRelationship": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/voc/sf-complex-role/sample",
              "name": "sample"
            }
          }
        ],
        "isFeatureOfInterestOf": [
          {
            "@id": "https://lab.scinfo.org.nz/elfie/id/om/om_observation/47-42-00002-1664-3110",
            "@type": "om:OM_Observation",
            "hasResult": {
              "@id": "https://lab.scinfo.org.nz/elfie/def/nsdr/cl_concept/1048-140253",
              "name": "Typic Orthic Allophanic Soil"
            },
            "resultTime": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "usedProcedure": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "phenomenonTime": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "observedProperty": {
              "@id": "https://lab.scinfo.org.nz/soil-data-ie/def/property/soclassifier",
              "name": "classifier"
            },
            "hasFeatureOfInterest": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/sams/sf_spatialsamplingfeature/55-42-00002",
              "@type": "sams:SF_SpatialSamplingFeature"
            }
          },
          {
            "@id": "https://lab.scinfo.org.nz/elfie/id/om/om_observation/78-42-00002-1726",
            "@type": "om:OM_Observation",
            "hasResult": {
              "uom": "degrees",
              "@value": 0
            },
            "resultTime": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "usedProcedure": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "phenomenonTime": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "observedProperty": {
              "@id": "https://lab.scinfo.org.nz/soil/def/property/slope-angle",
              "name": "slope angle"
            },
            "hasFeatureOfInterest": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/sams/sf_spatialsamplingfeature/55-42-00002",
              "@type": "sams:SF_SpatialSamplingFeature"
            }
          },
          {
            "@id": "https://lab.scinfo.org.nz/elfie/id/om/om_observation/88-42-00002-1669",
            "@type": "om:OM_Observation",
            "hasResult": {
              "@value": "Rye grass-white clover pasture"
            },
            "resultTime": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "usedProcedure": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "phenomenonTime": {
              "@id": "http://www.opengis.net/def/nil/OGC/0/unknown",
              "name": "unknown"
            },
            "observedProperty": {
              "@id": "http://purl.org/dc/terms/description",
              "name": "description"
            },
            "hasFeatureOfInterest": {
              "@id": "https://lab.scinfo.org.nz/elfie/id/sams/sf_spatialsamplingfeature/55-42-00002",
              "@type": "sams:SF_SpatialSamplingFeature"
            }
          }
        ]
      },
      "id": "https://lab.scinfo.org.nz/elfie/id/sams/sf_spatialsamplingfeature/55-42-00002"
    }
  ],
  "links": [
    {
      "type": "application/geo+json",
      "rel": "alternate",
      "title": "This document as GeoJSON",
      "href": "http://172.19.0.2:5000/collections/sf_spatialsamplingfeatures/items?f=json"
    },
    {
      "rel": "self",
      "type": "application/ld+json",
      "title": "This document as RDF (JSON-LD)",
      "href": "http://172.19.0.2:5000/collections/sf_spatialsamplingfeatures/items?f=jsonld"
    },
    {
      "type": "text/html",
      "rel": "alternate",
      "title": "This document as HTML",
      "href": "http://172.19.0.2:5000/collections/sf_spatialsamplingfeatures/items?f=html"
    },
    {
      "type": "application/geo+json",
      "rel": "prev",
      "title": "items (prev)",
      "href": "http://172.19.0.2:5000/collections/sf_spatialsamplingfeatures/items/?startindex=0"
    },
    {
      "type": "application/geo+json",
      "rel": "next",
      "title": "items (next)",
      "href": "http://172.19.0.2:5000/collections/sf_spatialsamplingfeatures/items/?startindex=10"
    },
    {
      "type": "application/json",
      "title": "Landcare Spatial Sampling Features",
      "rel": "collection",
      "href": "http://172.19.0.2:5000/collections/sf_spatialsamplingfeatures"
    }
  ],
  "timeStamp": "2019-10-04T00:45:35.631872Z",
  "id": "http://172.19.0.2:5000/collections/sf_spatialsamplingfeatures/items"
}

(Note, the links and timestamp are still ambiguous, but that's easily addressed.)

The ordinary GeoJSON FeatureCollection view is fine, and the collection renders successfully as HTML (with embedded JSON-LD), as expected:

Screenshot from 2019-10-04 13-48-52

The response is still fast.

The one "trick" here really is that each feature has an id key and also a properties['@id'] key which have the same value. My (limited) understanding of the semantics here is that therefore any statement made as a key/value nested within properties is explicitly about the same entity. I'm not really sure about any further semantic implications of making such a statement, or even if it's necessary.


All that to say, I'm so far pretty happy with this experiment and once again thanks for a great project.

@jorgejesus
Copy link
Member

@alpha-beta-soup if you are happy with the experiment, then I am in ecstasy with the jsonl-ld SQL generation

@pvgenuchten
Copy link
Contributor

I noticed the idea of referencing an external context file to define the ontology of a featuretype. I like the idea from a perspective of introducing ontology without the complexities of managing rdf output.

On the other hand, I also like the getting-started-with-linked-data approach in ldproxy, which allows users to annotate feature attributes with concepts from an ontology (such as schema.org) and the tool generating the relevant context. But sure, embedding an external context is also far more powerfull in the long run.

A relevant question could be if we should consider to at some point also support other rdf encodings, such as rdf+xml, ttl? If so, does it make sense to use json-ld as a base and generate other encodings from that?

@alpha-beta-soup
Copy link
Contributor Author

alpha-beta-soup commented Nov 7, 2019

To-do:

  • Move script to bottom of page (but within <body>) to marginally improve loading.

@alpha-beta-soup
Copy link
Contributor Author

I'm happy with this PR now. There's more that could be considered, but I think this is a minimum loveable implementation of structured data that is compatible with the core of pygeoapi, that affords a great deal of power without sacrificing anything. I'll just go through a few remaining issues and comments that I think are worth mentioning.


I don't know how best to describe the links with a popular vocabulary. I'm not particularly concerned about this since the links are mostly "reflective" (i.e. self and alternate relations). In the JSON-LD representation, the links array is present (since I don't omit anything that's also in the JSON representation) but the property does not get expanded: a JSON-LD expansion algorithm will ignore it.


@pvgenuchten A relevant question could be if we should consider to at some point also support other rdf encodings, such as rdf+xml, ttl? If so, does it make sense to use json-ld as a base and generate other encodings from that?

I don't know whether pygeoapi should support other encodings; I also don't know how that should be supported. I would err on the side of supporting only JSON-LD for RDF. Automated converters could do the rest for a client who insists on consuming rdf+xml or ttl; the overhead and/or additional dependencies for pygeoapi seem too great. Happy to be proven wrong.


In the JSON-LD tests, so far there's only a test for the obs dataset (i.e. the CSV provider). Since the implementation is provider-agnostic, this doesn't bother me. What does bother me is that I'd like to somehow give examples of "advanced use-cases", implementing ideas such as using https://schema.org/minValue and https://schema.org/maxValue to add additional constaints/semantics to what is otherwise arbitrary quantitative values. With a fundamentally flat structure like CSV, this is not really possible (which is fine). Still, at some point I'd like to explore how we might get to using advanced JSON-LD properties like @graph, and index maps (@container and @index)—if there are compelling use-cases. I'm happy to wait and see rather than forging ahead with useless additions. The minimum implementation is just allowing an arbitrary @context, and that's achieved.


I'm still unsure whether the features in a collection should be assigned an @id property that has the value of the feature-level pygeoapi URL, or if instead we should use https://schema.org/url (or something else). There's room for use of https://schema.org/sameAs, but that could be done at the underlying data store level (e.g. with a CSV column literally https://schema.org/sameAs with an appropriate URI value—more likely you'd do this with an RDBMS provider). This implementation uses @id but I'm actually unsure of the consequences of that.

@alpha-beta-soup
Copy link
Contributor Author

Something else to be updated when/if this is merged: https://github.com/geopython/pygeoapi/wiki/SEO

@@ -56,7 +56,7 @@ metadata:
- data
- api
keywords_type: theme
terms_of_service: None
terms_of_service: null
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except for the null and context def updates, can you revert the rest of the changes in this file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but can I request that we leave the hours of service one? I wanted to emphasise that the value there is not just unstructured text, but rather will be interpreted: https://schema.org/openingHours

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also a value of Lastname, Firstname suggests a structure that isn't there, suggest changing that to Full Name as in this PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lastname, Firstname is as per ISO 19115 rules. Is there a fullname property we can model after on schema.org?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, in that case Lastname, Firstname is fine. It's merely that https://schema.org/name is just unadulterated text, doesn't imply a person, etc. Happy to leave as is.

requirements.txt Outdated Show resolved Hide resolved
@tomkralidis tomkralidis merged commit 17ed141 into geopython:master Dec 19, 2019
@tomkralidis
Copy link
Member

Thank you @alpha-beta-soup for this awesome feature! Perhaps document your description of this functionality (as you described on gitter) in the docs in a subsequent PR if you have time.

Thanks again for your patience and addressing the PR comments/iterations.

@alpha-beta-soup alpha-beta-soup mentioned this pull request Dec 23, 2019
francbartoli pushed a commit to francbartoli/pygeoapi that referenced this pull request Jul 8, 2021
* Changes to sample pygeopai-config.yml
- improved terms_of_service and contact.url for SEO
- fixes typo for contact.instructions

* whitespace

* dynamically embed JSON-LD representation in head

* json-led representation for root, collections, and collection

* updates sample configuration with metadata that won't cause JSON-LD validation issues

* adds support for feature-level JSON-LD representations of collections and items

* improves consistency between microdata and JSON-LD root metadata

* valid Dataset spatial; and better use of @id

* role →  instructions

* adds jsonld f param to open api definition

* working with new temporal extent interface

* removes comment

* identify and retain NIR/HTTP IDs rather than construct them with reference to pygeoapi

* don't pop a feature.properties id, to retain JSON-LD reference integrity

* add/update tests for inclusion of json-ld responses

* bug fixes

* fix bug where id was assumed to be string

* better url checking

* renames format variable to avoid reserved word

* removes top-level CRS in obs sample config

* add json-ld textMimeType for serverless

* adds flask-cors dependency to allow cors

* moves json-ld-requesting script to bottom of body

* corrects schema.org url

* fixes urls

* adds pyld dependency

* adds tests for json-ld representations (incomplete)

* more tests

* make pyld a dev requirement only

* linting

* changes from revision

* removes merge artifact
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants