Is 'default' attribute value an absolute reference or a relative one?

[prjemian]

is it allowed for the attribute 'default' to contain absolute references or only a relative one? Shall the relative reference point to one of the subgroups or can it point to a subgroup of a subgroup of a ...?

[prjemian]

is it allowed for the attribute 'default' to contain absolute references or only a relative one?

The value of parent_group@default must be the name of a group in the parent_group. Names with path delimiters will not be found.

[sanbrock]

step by step process instead of a direct jump (should be better documented)

[woutdenolf]

This adds to the confusion: https://www.nexusformat.org/2014_How_to_find_default_data.html

add default attribute to each NXentry that states which NXdata is the default and only to resolve ambiguity when more than one NXdata exists. The value is the name of the NXdata group.

[vasole]

@woutdenolf

In 2018 NIAC it was accepted that any group could contain a group@default pointing to an NXdata inside its hierachy.

[vasole]

@prjemian

I do not know from where you obtain the step-by-step rule. All what it was said in NIAC 2018 is that the target of a the default attribute had to be an NXdata inside the hierarchy of the containing group. Nothing about a step-by-step rule because that would imply the default attribute as pointing to something not an NXentry (for NXroot) or an NXdata (any other group). That was de-facto validating relative paths down to the hierarchy of the containing group.

Are you confusing with some sort of depends on?

This is what it was reflected in the minutes.

https://www.nexusformat.org/NIAC2018Minutes.html

Extended Use of the Defaults Attribute

Currently we have a default attribute at root level which points to the default data. Currently this is restricted to NXroot, NXentry, NXsubentry.

Proposal: allow the default attribute for any NeXus group which contains a plotable NXdata.

All in favor

[rayosborn]

Perhaps it's worth showing an example just in case there is a misunderstanding of how the default attribute is used.

root:NXroot
  @default = 'e2'
  e1:NXentry
    @default = 'd2'
    d1:NXdata
    d2:NXdata
  e2:NXentry
    @default = 'd3'
    d3:NXdata
    d4:NXdata

In the above, d2 and d3 are set to be the default NXdata groups to be plotted within NXentry groups e1 and e2, respectively. So, in NeXpy, e1.plot() will plot d2 and e2.plot() will plot d3. However, since e2 is set as the default at the root level, root.plot() will plot d3.

No paths are necessary, since the default attribute refers to a unique NXdata group within it. As @vasole, says, there is no reason in principle why this can't be extended to any subgroups of NXentry groups that also contain NXdata groups. Which of those groups gets propagated back to be the default group at the root level is to be decided when implementing it. NeXpy (or rather the nexusformat API) has an over argument to dictate whether a default overrides those at a higher level, so to make d2 the default at root level, I would type:

root['e1/d2'].set_default(over=True)

All this does is set e1 to be the default at root level, since d2 is already the default at entry level.

[woutdenolf]

So there are currently two interpretations:

1. step-by-step discovery: @default should point to a direct subgroup which needs to be an a group with an NX_class that can have a @default attribute

/:NXroot
  @default = 'e1'      <<< discovery step 1
  e1:NXentry
    @default = 'p2'    <<< discovery step 2
    p1:NXprocess
      @default = 'd2'
      d1:NXdata
      d2:NXdata
    p2:NXprocess
      @default = 'd2'  <<< discovery step 3 (we found the NXdata)
      d1:NXdata
      d2:NXdata        <<< this is the default plot

2. 2-step discovery: @default should point to an NXdata, except for the root for which @default should point to an NXentry direct subgroup

/:NXroot
  @default = 'e1'         <<< discovery step 1
  e1:NXentry
    @default = 'p2/d2'    <<< discovery step 2 (we found the NXdata)
    p1:NXprocess
      @default = 'd2'
      d1:NXdata
      d2:NXdata
    p2:NXprocess
      @default = 'd2'
      d1:NXdata
      d2:NXdata           <<< this is the default plot

Note that in the first interpretation, the number of steps to find the default plot depends on how deep in the tree the default NXdata is stored. In the second interpretation the default NXdata is always found in 2 steps.

At the ESRF we have always used interpretation 2. Personally I find interpretation 1 more elegant:

• For example the final NXdata could be in an external file which structure you may not know when setting the defaults in the master file.
• Interpretation 1 is also more convenient when renaming groups.
• Finally if the Nexus standard is not tight to HDF5 then the relative path in @default = 'p2/d2' may not have a Nexus meaning (unless Nexus has defined the concept of relative paths with the "/" separator).

[woutdenolf]

@rayosborn You did not show an example where the NXdata is found deeper in the structure but from your explanation it seems nexpy is using interpretation 1 (step-by-step) from #1063 . Is that correct?

[prjemian]

The default value must be a child of the current group and that child must be a NeXus group.

Apologies if the documentation was not clear. The proposal to NIAC showed this.

[prjemian]

The value of the default attribute ...

[rayosborn]

@woutdenolf, I have always interpreted it as interpretation 1, as I think @prjemian has confirmed. I believe that NeXpy shows that the scheme works perfectly fine in practice.

[rayosborn]

@woutdenolf, here is an example of the default attribute pointing deeper in the hierarchy. I just tested this in NeXpy.

root:NXroot
  @default = 'e1'
  e1:NXentry
    @default = 's1'
    d1:NXdata
    d2:NXdata
    s1:NXsubentry
      @default = 'd6'
      d5:NXdata
      d6:NXdata
  e2:NXentry
    @default = 'd3'
    d3:NXdata
    d4:NXdata

In this example, root.get_default() returns NXdata('d6').

This currently only works for NXsubentry, which is a sub-class of NXentry, but clearly it could be extended to, e.g., NXprocess groups.

On the other hand, I would caution against propagating a default from too deep in the hierarchy back to the root level. It would be odd, it seems to me, for the default plot for the whole file to be buried where it would otherwise be difficult to find. There may be use cases where it makes sense, so I wouldn't rule it out.

[woutdenolf]

There may be use cases where it makes sense, so I wouldn't rule it out.

NXroot > NXentry > NXprocess > NXdata is a common use case when saving processed data.

[rayosborn]

I agree with that, although it's not how I use NXprocess groups. In fact, I've just noticed that there appears to be an inconsistency in the NXprocess base class definition. It already contains a default attribute, which has the misleading description "Declares which child group contains a path leading to a NXdata group." However, the base class doesn't allow for either NXdata groups or groups that could contain NXdata groups. The only allowed group is a NXnote group.

Actually, this doesn't just apply to NXprocess groups. It looks as if, at some time, we added the default attribute to pretty well every base class. I think I will raise this as a NIAC issue, because I think it can only cause confusion to allow a default attribute absolutely everywhere. Ironically, I think this is what @vasole was calling for but I don't think we have implemented it sensibly.

[vasole]

@rayosborn

Yes, I asked to allow an optional default attribute to any group containing an NXdata downstream its hierachy and yes I had in mind option 1.

That was unanimously approved in 2018. In practice it has turned put to be quite convenient. Option 1 or option 2, I do not really care, but the feature itself. is very convenient.

This is what it was reflected in the minutes.

https://www.nexusformat.org/NIAC2018Minutes.html

Extended Use of the Defaults Attribute

Currently we have a default attribute at root level which points to the default data. Currently this is restricted to NXroot, NXentry, NXsubentry.

Proposal: allow the default attribute for any NeXus group which contains a plotable NXdata.

All in favor

[vasole]

Concerning the example given by Wout, if the entry only contains an NXdata, even if it is deeper in the hierarchy I would advocate to create softlink at NXentry level pointing to the deeper encountered NXdata.

The idea behind the default attribute was to be able to have other NXdatas besides the one at top level. For instance, Each NXdetector or NXprocess could provide a representative NXdata. It has been working quite nice and I do not see a major need for option 2 except perhaps when dealing with external links as pointed out by @woutdenolf . Since it is a work to be done by a program and not by a user, please agree among you what you prefer.

[sanbrock]

Adding the attribute 'default' to every group supports the idea that every group (independently where it is in a definition hierarchy) can have its default plot (assuming that there is an NXdata inside). Hence, solution (1) offers a nice situation, that one can get a default plot if you click anywhere in the tree of hdf groups.

[vasole]

@sanbrock

Yes, that was what it was shown to the NIAC in 2018 and what was unanimously approved.

[rayosborn]

I think we still need to at least clarify the wording, since I think the use of the word 'path' has led to the confusion about which interpretation is correct. I think the NIAC also needs to clarify the situation concerning the presence of NXdata groups when they are not strictly allowed in the base class definitions. Perhaps there is already a blanket statement somewhere concerning this. It is not obvious to me that it makes sense to have a NXdata group within a NXcapillary group, but if there is a good reason, I would be interested to see if such files validate successfully.

[PeterC-DLS]

There may be use cases where it makes sense, so I wouldn't rule it out.

NXroot > NXentry > NXprocess > NXdata is a common use case when saving processed data.

In this case, we put the NXdata in the entry group (which could be an entry or subentry called processed_results or whatever analysis/processing).

[rayosborn]

That's what we do. I can see an argument for putting the NXdata group in the NXprocess group that describes how it was produced, but it doesn't really make sense in our case, because the processed data results from a sequence of processes. We didn't want users to have to tunnel into different NXprocess groups to find where the data is stored.

[benajamin]

This topic was discussed at the code camp and we worded the manual towards saying that the default attribute should be a relative reference to a child group (strict writer), but a reader might find an absolute path.

[prjemian]

Essentially a duplicate of #1003