Update /Observation in light of pending experience including datacommons.org

[danbri]

Data Commons have been using /Observation heavily, and @darobin has also been looking into using it (see https://gist.github.com/darobin/07324754f0db7f090d737b9d2616536c). The /Observation type is in "Pending" status currently which means we are looking for feedback. This issue is to make some updates based on implementor experience.

[smrgeoinfo]

Why does 'marginOfError' have expected type 'DateTime'. The marginOfError should have a numeric value with some specification of the statistic used to estimate the margin of error, thus itself is actually an observation result.

[smrgeoinfo]

Seems like it would be good to align the schema.org observation with the w3c SOSA observation "Observation - Act of carrying out an (Observation) Procedure to estimate or calculate a value of a property of a FeatureOfInterest. Links to a Sensor to describe what made the Observation and how; links to an ObservableProperty to describe what the result is an estimate of, and to a FeatureOfInterest to detail what that property was associated with.'

Looks like observedNode would map to FeatureOfInterest.
measuredProperty maps to ObservableProperty
measuredValue maps to result
Procedure is missing AFAICT in the schema.org proposal, but is important to understand and trust an observation result. For example in the gist referenced above, I would really like to know how and by who those numbers were compiled.

[darobin]

Having seen first-hand the work that the NYT folks need to do to get at such data, I believe that this could prove very useful and is worth shipping early even if it may be imperfect at first.

[dr-shorthair]

A proposed mapping between Schema.org and SOSA was prepared a couple of years ago - see https://github.com/w3c/sdw/blob/gh-pages/ssn/rdf/sosa-sdo-mapping.ttl
I note that a few other new properties are proposed here, so some of the mapping mentioned above would have to be adjusted if this proposal was accepted.

A few details:

1. I suggest moving Observation so that its parent is Action rather than Intangible (The Observation type had not been introduced to Schema.org at that time, so in the mapping we used Action imagining that was a suitable super-type.)

Note that the tension between

• observation={information-object} and
• observation={act-to-generate-information}

was the topic of much discussion in the W3C project that generated the SOSA vocabulary before we came down on the side of it being an Action, partly through analogy with PROV and BFO.

2. What is the relationship between measuredProperty and variableMeasured? They come from different origins (domain: dataset vs observation), but is there a good reason for duplicating the function with two tags?

3. In place of measuredValue (which is not yet in the Schema.org canon) could you call it result - this follows from the idea that an Observation is an action whose outcome is new information.

4. In SOSA (and its predecessors) we recognised that there was potentially more than one date/time associated with an observation. There are many, but the two most widely required are

(1) the time in the world that the result describes (which is usually what an analyst is interested in)
(2) the time that the result became available (more related to data management).

In general these are not the same. At the very least there is likely to be a data-processing and communications lag which will put the result-time later than the phenomenon-time. While this may or may not be important, having both slots explicit makes the structure also useful

• for forecasts (when the result-time is before the phenomenon-time) or
• for historical/archaeological/geological/cosmological observations (when the result-time is a long way later than the phenomenon-time).

I suspect that observationDate is intended to be the phenomenon-time, but this should at least be clear from the documentation.

5. A link to the protocol and sensor/observer is missing. This is important metadata/quality information.

[smrgeoinfo]

schema.org Action does seem like a better parent class for observation: it accounts for

• Agent-- who made the observation
• startTime, endTime-- when the observation was made
• instrument-- the sensor or other instrument used to make the observation
• location-- where was the observation made.
• object-- doesn't seem like too much of a stretch to make this 'feature of interest' (is that the intention of 'observedNode'???)
• result-- this would have to have range 'measuredValue' that would be a quantifier--numeric or categorical, with uncertainty

I agree that measuredProperty and variableMeasured are redundant. Keep variableMeasured since its already there.

Location-- Have to clarify the location of the feature of interest from the location of the observer, e.g. where was the satellite as opposed to what was the satellite imaging, or where the sample was collected in the field from the location of the lab where it was analyzed.

reiterating issues Simon brought up:

• observation time is commonly more complex that a simple calendar date/time interval
• distinguish phenomenon time from observation time (perhaps other time semantics)
• Need to have a link to the protocol/method/procedure for how the observation was made/obtained.

[ramonawalls]

I second the suggestions from @smrgeoinfo and @dr-shorthair. Particularly important is the change to make Observation a child of Action rather than Intangible.

[danbri]

Several people have suggested that we put Schema.org/Observation below /Action. I can see how this makes sense, in that observing is a kind of action. However I am wary of doing so, in that the Schema.org Actions design has its own complications (including an associated protocol) and after some years only a patchy implementation story. I am wary of complicating the picture further by mixing this work in there. Similarly, there is a case that actions are events; and yet we have chosen not to entangle those pieces of the design either, for now at least.

[dr-shorthair]

On the detailed matter that you bring up @danbri, every sosa:Observation (from W3C SSN) has a usedProcedure property which I think carries related information to your protocol. (Note that this is separate from instrument which in this context implements the procedure.) So I don't think there is necessarily such a clash. Now I can imagine that Action as originally conceived had distinct behaviour, but maybe it is not so different.

There is already quite a rich set of sub-classes of Action. I'd be comfortable seeing Observation slot in there.

[github-actions]

This issue is being tagged as Stale due to inactivity.

[dr-shorthair]

Looking at W3C/OGC sosa::Observation I had proposed the following alignment to the Schema.org/CreateAction, on the principle that an Observation is an action that creates information.

(The other 'actions' in SOSA/SSN are sosa:Sampling - which creates a Sample, and sosa:Actuation, which results in a change of state.)

sosa:Observation has a set of standard properties which could be aligned to Schema.org as follows

sosa:Observation rdfs:subClassOf schema:CreateAction . 

sosa:usedSensor rdfs:subPropertyOf schema:instrument .
sosa:usedProcedure owl:equivalentProperty schema:measurementTechnique .
sosa:hasFeatureOfInterest rdfs:subPropertyOf schema:object .
sosa:observedProperty owl:equivalentProperty schema:measuredProperty .
sosa:result rdfs:subPropertyOf schema:result .
sosa:resultTime rdfs:subPropertyOf schema:endTime .
sosa:phenomenonTime rdfs:subPropertyOf schema:temporalCoverage . 

Schema.org/Observation type is not tied to the Action hierarchy. @danbri provides some explanation above #2564 (comment) But I would argue that unless the Action hierarchy is removed, then it makes no sense for Observation not to sit there.

(Modified 2022-08-31 - added sosa:usedProcedure and amended sosa:observedProperty mappings.)

[dr-shorthair]

@danbri wrote

the Schema.org Actions design has its own complications (including an associated protocol)

I don't see the hook for protocol in https://schema.org/Action - can you explain what you meant?

[dr-shorthair]

@smrgeoinfo @ashepherd @mbjones @fils @njarboe this is possibly relevant to ESIPFed/science-on-schema.org#211

[dr-shorthair]

@pbuttigieg I wonder if you had seen this thread - attempting to bridge schema.org with sensor/observation models from W3C & OGC. Possibly relevant to your Ocean InfoHub work?

[dr-shorthair]

@danbri : in #2564 (comment) you mentioned

the Schema.org Actions design has its own complications (including an associated protocol)

I can't find any trace of the 'associated protocol'. Could you point us to that?
AFAICT this would be only blocked to making the proposed change - i.e. to move Observation to be a subclass of Action rather than Intangible.

All the contributors to this issue appear to be in agreement. I'll prepare a PR if that would help move things forward.

[danbri]

Maybe “protocol” was not best name, but see https://schema.org/docs/actions.html Similarly we didn’t entangle Action and Event

…

On Tue, 30 Aug 2022 at 07:48, Simon Cox ***@***.***> wrote: @danbri <https://github.com/danbri> : in #2564 (comment) <#2564 (comment)> you mentioned the Schema.org Actions design has its own complications (including an associated protocol) I can't find any trace of the 'associated protocol'. Could you point us to that? AFAICT this would be only blocked to making the proposed change - i.e. to move Observation to be a subclass of Action rather than Intangible. All the contributors to this issue appear to be in agreement. I'll prepare a PR if that would help move things forward. — Reply to this email directly, view it on GitHub <#2564 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABJSGPP56H35J7RLDU4BQDV3WVDLANCNFSM4M7BJDSA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[dr-shorthair]

Clarifier: what is the relationship between

1. schema:measuredProperty schema:rangeIncludes schema:Property - used on Observations
2. schema:variableMeasured schema:rangeIncludes schema:PropertyValue - used on Datasets

??

[dr-shorthair]

see https://schema.org/docs/actions.html

Would it help if I generated some additional examples that could be added to this page, showing Observation as a sub-type of Action (or CreateAction) ?

Similarly we didn’t entangle Action and Event

They are potentially related. However, the suggested properties of Event are clearly related to cultural events, like concerts and entertainments, while the suggested properties of Action include instrument which can easily be a Sensor, and result which is exactly the same language as used in W3C SOSA/SSN. Perhaps there is a common supertype, like bfo:Occurrent?

I had proposed that Observation be made a sub-type of CreateAction, on the basis that an Observation creates some information. But now I notice that the generic Action itself has result as a suggested property, and that CreateAction adds no more properties or axioms. What is CreateAction for?

[dr-shorthair]

Example of a single observation showing all observation metadata fields and result:

{
    "@context": "https://schema.org/",
    "@type": "Observation",
    "name": "O-BG-78x",
    "actionStatus": "CompletedActionStatus",
    "object" : { 
      "@type" : "Thing",
      "name" : "Boab prison tree, Derby",
      "url" : "https://www.wikidata.org/wiki/Q4931215"
    },
    "measuredProperty" : { 
      "@type" : "Property",
      "name" : "Diameter at breast height",
      "url" : "https://www.wikidata.org/wiki/Q973582"
    },
    "measurementTechnique" : "directly measure circumference then divide by pi",
    "instrument" : { 
      "@type" : "Thing",
      "name" : "Stanley Fatmax 30m tape measure",
      "url" : "https://www.bunnings.com.au/stanley-fatmax-30m-tape-measure_p5667609"
    },
    "agent" : {
      "@type": "Person",
      "name": "Frances Peel",
      "url" : "https://orcid.org/0000-0009-3899-3499"
    },
    "endTime" : "2014-03-01T11:30:00Z",
    "temporalCoverage" : "2014-03-01T11:15:00Z/2014-03-01T11:30:00Z",
    "result": {
      "@type": "PropertyValue",
      "name": "Length",
      "value": "4.7 m"
    }
  }

I could add this to the Wiki documentation as an additional example (i.e. fork and make a PR).
However, I don't see where this documentation is linked from. How do people usually find it?

[dr-shorthair]

Here's the matching SSN/SOSA

<https://example.org/obs/O-BG-78x>
  a sosa:Observation ;
  rdfs:label "O-BG-78x" ;
  sosa:hasFeatureOfInterest <https://www.wikidata.org/wiki/Q4931215> ;
  sosa:observedProperty <https://www.wikidata.org/wiki/Q973582> ;
  sosa:usedProcedure [
      a sosa:Procedure ;
      dcterms:description "directly measure circumference then divide by pi" ;
    ] ;
  sosa:madeBySensor <https://www.bunnings.com.au/stanley-fatmax-30m-tape-measure_p5667609> ;
  prov:wasAssociatedWith <https://orcid.org/0000-0009-3899-3499> ;
  sosa:resultTime "2014-03-01T11:30:00Z"^^xsd:dateTimeStamp ;
  sosa:phenomenonTime [
      a time:Interval ;
      time:hasBeginning [ a time:Instant ; time:inXSDDateTimeStamp "2014-03-01T11:15:00Z"^^xsd:dateTimeStamp ; ] ;
      time:hasEnd [ a time:Instant ; time:inXSDDateTimeStamp "2014-03-01T11:30:00Z"^^xsd:dateTimeStamp ; ] ;
    ] ;
  sosa:hasSimpleResult "4.7 m"^^cdt:ucum ;
.

[KathiSchleidt]

Meta-question on schema.org and JSON-LD - I just pasted the example to the JSON-LD Playground after I fixed the link in the context (https://schema.org resolves to the main landing page, https://schema.org/docs/jsonldcontext.json seems to be the file to include in the context), but keep getting errors.

Would be nice to see the LD in action

Any ideas why this doesn't work?

[dr-shorthair]

I cloned-and-modified from https://schema.org/docs/actions.html
But I'm not a JSON-LD person. @fils might be able to help.

[smrgeoinfo]

hmmm, when I paste dr-shorthair's JSON-LD with no changes, it seems to work fine in the playground .
"@context": {"@vocab":"http://schema.org/"}
also works.

[gkellogg]

hmmm, when I paste dr-shorthair's JSON-LD with no changes, it seems to work fine in the playground .
"@context": {"@vocab":"http://schema.org/"}
also works.

People can be confused because opening "https://schema.org/" in a browser returns a web page, and not a JSON-LD Context. The key is that a JSON-LD processor will make the request with ACCEPT: application/ld+json, which causes the context to be returned, which is why the playground yields the results you expect.

[KathiSchleidt]

I'd assumed that conneg would sort the bit of getting the context JSON vs. the HTML page.

My issue is that when I paste Simon's example to the JSON-LD Playground, I get an error

[danbri]

It used to, but added unwanted complexity to Schema.org hosting, so we have moved to the JSON-LD 1.1 link: header mechanism. This makes it possible to host the site statically, which helps with DDOS etc

…

On Wed, 7 Sep 2022 at 10:18, Kathi Schleidt ***@***.***> wrote: I'd assumed that conneg would sort the bit of getting the context JSON vs. the HTML page. My issue is that when I paste Simon's example to the JSON-LD Playground <https://json-ld.org/playground/#startTab=tab-expanded&json-ld=%7B%22%40context%22%3A%22https%3A%2F%2Fschema.org%2F%22%2C%22%40type%22%3A%22Observation%22%2C%22name%22%3A%22O-BG-78x%22%2C%22actionStatus%22%3A%22CompletedActionStatus%22%2C%22object%22%3A%7B%22%40type%22%3A%22Thing%22%2C%22name%22%3A%22Boab%20prison%20tree%2C%20Derby%22%2C%22url%22%3A%22https%3A%2F%2Fwww.wikidata.org%2Fwiki%2FQ4931215%22%7D%2C%22measuredProperty%22%3A%7B%22%40type%22%3A%22Property%22%2C%22name%22%3A%22Diameter%20at%20breast%20height%22%2C%22url%22%3A%22https%3A%2F%2Fwww.wikidata.org%2Fwiki%2FQ973582%22%7D%2C%22measurementTechnique%22%3A%22directly%20measure%20circumference%20then%20divide%20by%20pi%22%2C%22instrument%22%3A%7B%22%40type%22%3A%22Thing%22%2C%22name%22%3A%22Stanley%20Fatmax%2030m%20tape%20measure%22%2C%22url%22%3A%22https%3A%2F%2Fwww.bunnings.com.au%2Fstanley-fatmax-30m-tape-measure_p5667609%22%7D%2C%22agent%22%3A%7B%22%40type%22%3A%22Person%22%2C%22name%22%3A%22Frances%20Peel%22%2C%22url%22%3A%22https%3A%2F%2Forcid.org%2F0000-0009-3899-3499%22%7D%2C%22endTime%22%3A%222014-03-01T11%3A30%3A00Z%22%2C%22temporalCoverage%22%3A%222014-03-01T11%3A15%3A00Z%2F2014-03-01T11%3A30%3A00Z%22%2C%22result%22%3A%7B%22%40type%22%3A%22PropertyValue%22%2C%22name%22%3A%22Length%22%2C%22value%22%3A%224.7%20m%22%7D%7D>, I get an error — Reply to this email directly, view it on GitHub <#2564 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABJSGKL7KXYN2R3L6PGGODV5BMWRANCNFSM4M7BJDSA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[danbri]

I've just posted an update to this vocabulary in schema.org version 16

There's a Google doc with some more notes on the approach taken, which essentially follows the evolution of the datacommons.org vocabulary which we adopted previously. I've gone into a bit more detail there on some of the topics discussed in this thread.

[drkane]

Not sure if this is the right place, but I have a query about how observationDate and observationPeriod are supposed to interact. Is there an expected relationship between the two - e.g. does the period modify the date? So an observationDate of 2020-01-01 and an observationPeriod of P1Y would mean the observation relates to all of 2020, or all of 2019?

I guess the question is - does it need clarifying whether observationDate is the start or end date of the observationPeriod?