Representation of Data node in the model

[mcdittmar]

In the kick-off meeting for Spectrum-1.2 RFE, we decided (at my request) that it would be OK to modernize the diagrams in the document to actual UML diagrams, but we want to minimize the overall changes to the document.

As I mentioned then, there are some inconsistencies between the diagrams and the model schema (xsd).
This ticket is to address one such inconsistency.

In short:

• the diagram shows Data as a set of DataAxes with arrays 1:Length of Value, Quantity, Accuracy
• the schema shows Data as an array of Points, each containing a set of DataAxes with 0:1 Value, Quantity, Accuracy
  • since UTypes are not used here, there is no real mapping of XML to the model. ie: there is no model element (UType) which the node "<Point>" represents.
• the text uses the term 'point' quite a lot, which I think can be interpreted either way.
• example instances
  • XML instance matches schema
  • VOTable instance; Groups match diagrams
  • FITS instance, very flat with just UType tags, impossible to isolate groupings.

The inconsistencies seem to be related to the 'natural' serialization in the different formats:
+ XML: easier/clearer to provide multiple instances of Point
+ VOTable: easier/clearer to point to the 'FIELD' containing all values of the same measure. In fact, it would not be possible to express an array of Points using VOTable syntax.

Proposal/Plan:
These are the issues that VODML and the annotation syntax are meant to address.
So, for the purposes of this update, I think the best course is to replace the diagrams as they currently are. Perhaps we can/should add some language to the XML section to address the structural difference in the serialization from the model.

---

Details:

• Section 3.1: Model Components
  • describes "spectrum is a set of one or more data points"... and "Specifically, a spectrum will have arrays of the following values:" listing Flux value, Spectral coordinate, Quality and Resolution as possible contents.
  • IMO: this is confusing, described as an array of 'points' and as an array of each measured property.
• Section 3.2: Use Cases and Required Fields
  • contains all the model diagrams;
  • Figure 2 shows the Data node
    • contains n 'DataAxis' elements (one for each measure - SpectralAxis, FluxAxis, etc. )
    • each DataAxis contains an array of values. ie: 1..Spectrum.Length of Value, Quality, Accuracy, Resolution
  • The text contains the minimal required content
    • the last bullet says: "For each point: the values of the spectral coordinate and flux."
• Section 7: XML schema serialization
  • embeds the model schema file
  • "In the following XML schema, we implement the model fairly directly."
  • "Within a spectrum the data points are kept together in objects called Point."
  • which indeed defines Data node
    • contains an array of Point
    • Point contains a single instance of each measure; SpectralAxis, FluxAxis, etc
      • each Axis type contains 0..1 of each Value, Accuracy, Resolution
    • there is also an alternate flavor of Point which flattens the Axis content (Value, Quality, Accuracy, Resolution) into 1 node

[lmichel]

A general remark: AFAIK, all model serializations are set in SSAP context and SSAP says that in case of conflict with SDM, it wins.

[lmichel]

A agree with not changing the XML schema. We do not have to mind about the serialization issue because SDM serializations are all based on GROUP/UTypes, thus as long as you do not change UTypes there no discrepancies with SSAP

[mcdittmar]

Contrary to my earlier message; I've updated the diagrams to a set which is closely aligned with the schema. I noticed that the VOTable serialization section mentions the the 'Point' element is implied by the table structure, so I'm thinking these diagrams are a good representation. The UTypes can all be mapped to an element in the diagrams.

I didn't make a PR for review, but would appreciate any comments on the new diagrams.
They are in wd-v1.2 branch commit 404f961 in doc/diagrams/*.png.

Notable deltas between these diagrams and the schema:

• unused elements arrayOfField and Field are not diagrammed
• I cut an abstract layer which supports 2 flavors of 'Point', arrayOfPoint and arrayOfFlatPoint. The text doesn't really discuss the different options, and it is more inline with the original diagrams
• DataSet element is not diagrammed here, it is not in the schema, but is in the earlier diagrams.. it's contents are included in the BaseSegment

[lmichel]

Mark,

It is a bit difficult to review the diagrams without any appropriate tool.

From my screen, the splitting look relevant. I'm just wondering whether you will add a low detail model overview.

[mcdittmar]

Mark,

It is a bit difficult to review the diagrams without any appropriate tool.

From my screen, the splitting look relevant. I'm just wondering whether you will add a low detail model overview.

Do you mean an overview diagram with all elements (no attributes)? I can do that.
At the time, I thought the PDF preview would be available for people to compare the diagrams and text, but there is no workflow on this repo yet.

Thanks for looking!

[lmichel]

I can try to make a workflow

[lmichel]

I wrote a workflow only triggered in the wd-v1.2 branch and which generates a Spectrum.pdf avatar.
https://github.com/ivoa-std/SpectrumDM/releases/tag/auto-pdf-preview
Thinks will have to be adapted later on to the IVOA publication steps

[mcdittmar]

I wrote a workflow only triggered in the wd-v1.2 branch and which generates a Spectrum.pdf avatar.

Thanks!
I've added doc/diagrams/Overview.png to the suite. Let me know if you were looking for something else.

[lmichel]

The only comment I had is no longer relevant once I've seen the PDF. That's all for today.