Determine ProvONE index fields

[mbjones]

The ProvONE model has many fields, determine which should be indexed in Solr and how.

Discussion today led us to the following new fields for the Solr index:

• prov_wasDerivedFrom
  • example: compiledData.1.1 prov_wasDerivedFrom data.1.1
  • example: figure.2.1 wasDerivedFrom data.1.1
• prov_generatedByProgram
  • inferenced via prov:wasGeneratedBy, prov:qualifiedAssociation, prov:hadPlan
  • example: compiledData.1.1 prov_generatedByProgram rScript.1.1
• prov_generatedByExecution
  • example: compiledData.1.1 prov_generatedByExecution execution.1.1 (this is the prov:wasGeneratedBy property)
    used
• prov_usedByProgram, inferred via prov:used, prov:qualifiedAssociation, and prov:hadPlan
  • example: data.1.1 prov_usedByProgram rScript.1.1
• prov_usedByExecution, which is the inverse of prov:used
  • example: data.1.1 prov_usedByExecution execution.1.1
• wasAssociatedWith has multiple fields, incuding two fields that aggregate the properties, and several for specific types of user identifiers
  • prov_usedByUser
  • prov_generatedByUser
  • specific fields will be
    • Prov_usedByOrcid
    • prov_usedByDataONEDN
    • prov_usedByFoafName
    • prov_generatedByOrcid
    • prov_generatedByDataONEDN
    • prov_generatedByFoafName
  • example: data.1.1 usedByFoafName "Matthew Jones"
  • example: data.1.1 generatedByOricid http://orcid.org/0000-0003-0077-4738
• prov_wasExecutedByExecution, which is inferred from prov:qualifiedAssociation and prov:hadPlan
  • example: rScript.1.1 prov_wasExecutedByExecution execution.1.1
    ProvONE types
• prov_instanceOfClass, multivalued, defined as a string URI,
  • similar to Ben's annotation field
  • Value would be a URI that represents an OWL class, with potential values such as the URIs for p1:User, p1:Program, p1:Visualization, prov:Plan, prov:Entity, p1:Execution, p1:Document

Discussion: Do we use DataONE account URIs for Agent properties?

• e.g., https://cn.dataone.org/cn/v1/accounts/CN=Jim%20Green%20A10401,O=Google,C=US,DC=cilogon,DC=org
• Or DataONE DNs? CN=Jim%20Green%20A10401,O=Google,C=US,DC=cilogon,DC=org
• Or ORCIDs?
• Or any of these?

Yes, any. The indexer will take any of these values, and will then look up the other values that exist in the DataONE user portal and add those to the index as well. Model would use any of: hasOrcid, hasDN, foaf:Name, etc. The RDF Subject URI could be a anonymous blank node with each of these properties.

[mbjones]

Reviewed these properties on today's sem-prov call.

I have now incorporated these into the PROVAnnotation design documents in SHA 69ceea7.

Open a new enhancement ticket if additional properties should be created, or a bug if one of these should be changed or revised.

[laurenwalker]

I propose we also index prov:used as "used."

[leinfelder]

Just a thought, but perhaps the index fields should be a little more distinguishable, like with a namespace: "prov_used"

The bare term - especially with a word like "used" - seems prone to confusion, collision, and misuse.

On Jan 15, 2015, at 4:59 PM, Lauren Walker notifications@github.com wrote:

I propose we also index prov:used as "used."

—
Reply to this email directly or view it on GitHub.

[mbjones]

Good idea. We should also be clear about the namespace in the term definition.

[csjx]

For the UI, being able to display an indication that a metadata document describing a data file has any provenance information. We decided that producing two more fields in the index would be helpful:

• prov_hasSources
• prov_hasDerivations

These would list the source pids and derivation pids for the data file this metadata describes. the UI will be able to display the total number based on the list size, and an icon that there is provenance information available. I'll open another ticket to add these.

[csjx]

To clarify where fields come from, we'll prefix them:
All fields will start with prov_ to diffrentiate them in the Solr index (and make searcing easier for provenance-enabled data packages). I'll open another ticket to add the prefix to the names.