C++: Complete OME-XML serialisation to and from OME model objects #1540

ghost · 2015-01-24T18:18:37Z

Fixes and additions to the Xerces-C wrappers
Fixes and additions to the xsd-fu generator and the C++ templates
Additional helper methods and functions added to ome-xml; parsing helpers move complexity out of the Genshi templates into real C++ templates, which is easier to edit and debug
Addition of OME-XML output and validation to showinf
Addition of model testcase to the ome-xml tests
Add helpers to MetadataTools and XMLTools from Java MetadataTools, XMLTools and OMEXMLServiceImpl
Revert model to 2013-06 for the 5.1.0 release

Note that the generated source code was changed by the template updates.

--no-rebase

Testing: Unit tests (cpp/test/ome-xml/model) will test all the 2013-06 samples except for one with timestamps we don't currently handle [currently can't store dates prior to the POSIX epoch]; I added a custom version without pre-1970 dates for the time being. Additional OME-XML could be tested by dropping it into the 2013-06 samples directory.

showinf should also display OME-XML metadata for TIFF files. Run (in the build tree):

./bf bin/showinf/showinf --omexml yes --validate yes /path/to/tiff

Or in a downloaded binary build:

cd /path/to/unpacked/zip
export BIOFORMATS_HOME=$(pwd)
export LD_LIBRARY_PATH="${BIOFORMATS_HOME}/lib"
PATH="${BIOFORMATS_HOME}/bin:$PATH"
showinf --help
showinf --omexml yes --validate yes /path/to/tiff

Use DYLD_LIBRARY_PATH on MacOS X in place of LD_LIBRARY_PATH.

Note it is likely to be missing original metadata annotations at this point, but the OME-XML should validate and contain all the correct dimensions, etc. (Multi-plane TIFFs will be timeseries and have SizeT==PlaneCount)

/cc @bramalingam @pwalczysko @emilroz

…ent methods

Used to introspect which namespace a given model object belongs to.

Primarily using weak_ptr lock() method to prevent invalid weak_ptrs from throwing bad_weak_ptr exceptions.

If it's a reference, the reference wrapper goes out of scope, while returning by value is still really just a copy of the wrapped DOMElement pointer.

Drop old properties which are no longer functional (bitrot). Add extra parent properties for use in templates.

Due to the fact that shared_from_this() isn't usable inside constructors, it's not possible to call the update() method in a constructor since taking references etc. of the object being constructed will cause problems. To avoid this, add create methods which construct the object, call update(), and then return a shared_ptr to the object.

… sources

Debug output breaks the cmake use of the xsd-fu output. This output was useless since it was in the generated sources in any case. All debug output is now expanded inline in the generated code.

…mple

…:Document

Use to check if a given model object can parse an element.

…dataTools

…e 2013-06 model

mtbc · 2015-01-28T12:08:04Z

components/xsd-fu/templates-cpp/OMEXMLModelObject.template

-        return asXMLElementInternal(document, nullelem);
+{% if klass.name != "OME" %}\
+        xerces::dom::Element element(document.createElementNS(NAMESPACE, "${klass.name}"));
+{% end if %}\


there isn't an else?

No. The OME root element is created with the document.

qidane · 2015-01-28T16:40:39Z

The .ome files look fine.

Would any of the changes to the xsd-fu files affect the java code generation?
Could reverting it for C++ cause something to revert for Java?

ghost · 2015-01-28T17:47:15Z

The xsd-fu changes are restricted to the C++ templates so won't affect Java. The exceptions are the tweaks to remove debug output but this won't affect the generated code.

joshmoore · 2015-01-29T10:41:48Z

Looks like everyone's happy. Thanks @rleigh-dundee (oh, and @rleigh-codelibre too).

C++: Complete OME-XML serialisation to and from OME model objects

joshmoore · 2015-01-29T10:43:07Z

components/xsd-fu/xsd-fu

@@ -190,26 +192,6 @@ def main(model, opts):
            content = content.render(encoding='utf-8')
            customAsXMLElementPropertyContent[propertyName] = content
        # END Custom templates for asXMLElment()
-        parents = model.resolve_parents(obj.name)


Doh, perhaps a bit belatedly could you explain why this was removed?

It's not removed; it's moved above so that the resolved parents are accessible in the ancillary template fragments.

Sorry, I intended to commend on if opts.debug

Ah, OK. In that case this was removed for a few reasons. The cmake build captures stdout to introspect what's being generated, and enabling debugging breaks things horribly by spewing rubbish. Debugging is now restricted to additional information expanded in the templates. Additionally, the debug output is mostly just a copy of what's in the actual generated code, so writing it to a file and also to stdout isn't giving us anything we don't already have. For the structured annoations, this is for debugging a single feature; this is included as a more general property list in the C++ templates, and the same can also be added to the Java templates if desired.

rleigh-codelibre and others added 30 commits January 24, 2015 14:25

cpp: ome-xerces: Add getElementsByTagName method to dom::Document

e5c71a1

cpp: ome-xerces: Add getElementsByTagName method to dom::Element

951f08d

cpp: ome-xerces: Add at method to dom::NodeList

43ed214

cpp: ome-xerces: Add empty method to dom::NodeList

8665dfb

cpp: ome-xerces: Add dom::NamedNodeMap

7136db5

cpp: ome-xerces: Add Node getAttributes, getNodeValue and getTextCont…

f67f294

…ent methods

cpp: ome-xerces: Correct String indentation

e491c63

cpp: ome-xerces: Correct dom::Document docstring

7ee81fa

cpp: ome-xml: Implement dumpXML method of OMEXMLMetadata

0418b13

cpp: ome-xml: Add getXMLNamespace method to OMEModelObject

dcf4fab

Used to introspect which namespace a given model object belongs to.

cpp: ome-xml: Correct stripNamespacePrefix

a052629

xsd-fu: Correct C++ OMEModelObject serialisation bugs

bd07562

Primarily using weak_ptr lock() method to prevent invalid weak_ptrs from throwing bad_weak_ptr exceptions.

cpp: showinf: ImageInfo shows OME-XML

dae3670

cpp: ome-xml: asXMLElement returns value, not reference

9656529

If it's a reference, the reference wrapper goes out of scope, while returning by value is still really just a copy of the wrapped DOMElement pointer.

cpp: showinf: Validate OME-XML

71d5c2e

xsd-fu: Output valid OME-XML

d5c55b9

xsd-fu: Update object.py properties

9fa2131

Drop old properties which are no longer functional (bitrot). Add extra parent properties for use in templates.

cpp: ome-xml: Correct conditional in OMEModel::addReferences

1227a79

cpp: ome-xml: Add elementName method to model objects

3d1422a

xsd-fu: Add summary of schema metadata for model objects to generated…

58c20c3

… sources

xsd-fu: Remove useless debug output

e18c98d

Debug output breaks the cmake use of the xsd-fu output. This output was useless since it was in the generated sources in any case. All debug output is now expanded inline in the generated code.

cmake: Add xsdfu-debug option

330f0b9

xsd-fu: C++ template tidying and corrections

32acc08

xsd-fu: Correctly serialise Boolean values

4b62060

xsd-fu: C++: Strip tag prefix when checking elements

dff3687

xsd-fu: C++: Handle XMLAnnotation content correctly

35a1649

xsd-fu: C++: Handle Description ordering correctly

d27c03e

cpp: ome-xml: Add XML property parsing helper functions

af4ac31

specification: Add namespace and nested elements to XML annotation sa…

6a45a43

…mple

rleigh-codelibre and others added 6 commits January 24, 2015 18:52

cpp: ome-xml: Create DOM document with default namespace

3541a84

cpp: ome-xerces: Add parsing parameters and default namespace to dom:…

0d038c1

…:Document

cpp: ome-xml: Add validElementName method to OMEModelObject

dee7b26

Use to check if a given model object can parse an element.

cpp: xsd-fu: Correct C++ template element validity check

9828ce2

cpp: test: Add ome-xml model test

684b3c3

cpp: ome-bioformats: Add OME-XML and Modulo utility functions to Meta…

23440bd

…dataTools

melissalinkert added the develop label Jan 24, 2015

Roger Leigh added 4 commits January 24, 2015 20:36

cpp: ome-xml: Revert to 2013-06 model

b283bfa

cpp: ome-xml: Correct docstring

d13ea00

xsd-fu: Make object.py acceptable to flake8

68ab553

cpp: ome-xml: Restore missing methods for metadata conversion with th…

9cd0794

…e 2013-06 model

mtbc reviewed Jan 28, 2015
View reviewed changes

joshmoore added a commit that referenced this pull request Jan 29, 2015

Merge pull request #1540 from rleigh-dundee/cpp-metadata5

74843c7

C++: Complete OME-XML serialisation to and from OME model objects

joshmoore merged commit 74843c7 into ome:develop Jan 29, 2015

joshmoore reviewed Jan 29, 2015
View reviewed changes

ghost deleted the cpp-metadata5 branch January 29, 2015 10:44

sbesson added this to the 5.1.0-m4 milestone Feb 17, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

C++: Complete OME-XML serialisation to and from OME model objects #1540

C++: Complete OME-XML serialisation to and from OME model objects #1540

ghost commented Jan 24, 2015

mtbc Jan 28, 2015

ghost Jan 28, 2015

qidane commented Jan 28, 2015

ghost commented Jan 28, 2015

joshmoore commented Jan 29, 2015

joshmoore Jan 29, 2015

ghost Jan 29, 2015

joshmoore Jan 29, 2015

ghost Jan 29, 2015

C++: Complete OME-XML serialisation to and from OME model objects #1540

C++: Complete OME-XML serialisation to and from OME model objects #1540

Conversation

ghost commented Jan 24, 2015

mtbc Jan 28, 2015

Choose a reason for hiding this comment

ghost Jan 28, 2015

Choose a reason for hiding this comment

qidane commented Jan 28, 2015

ghost commented Jan 28, 2015

joshmoore commented Jan 29, 2015

joshmoore Jan 29, 2015

Choose a reason for hiding this comment

ghost Jan 29, 2015

Choose a reason for hiding this comment

joshmoore Jan 29, 2015

Choose a reason for hiding this comment

ghost Jan 29, 2015

Choose a reason for hiding this comment