The adoption of UK GEMINI 2.3 as an open data standard #74
Comments
C-D-I
changed the title
C-D-I
changed the title
|
With the publication of The Government Data Quality Framework, we need to document the mapping from the "DAMA data quality dimensions" that it uses to the ISO 19115/INSPIRE "data quality measures" that GEMINI uses. This is not a difficult mapping, although it won't be semantically perfect (!) and can't be bidirectionally lossless e.g. the "completeness" dimension maps easily to the "completeness - omission" measure, but "uniqueness" has no clear match (perhaps some "completeness - commission" results could be related to "uniqueness" failures). DAMA "Consistency" is almost certainly GEMINI "conceptual consistency". More of a question for now: who should do this work & where should the result live, to help a consistent understanding between different data publishers & users? Should it be part of the gov.uk data quality guidance? gov.uk metadata guidance? AGI's GEMINI? If the answer is "all three", then that will need to be kept in step - always better to have one authoritative (canonical) home for such things. |
Standard Assessment GEMINI Metadata 2.3The standard assessment is part of the evaluation of this standard by the Open Standards Board. The 47 questions are a subset of the CAMSS questions UK GEMINI (GEo-spatial Metadata INteroperability Initiative)Formal specificationQ. 1. Does it address and aid interoperability between public administrations? A. Yes Q. 2. Does it address and aid the development of digital services in government? A. Yes Q. 3. Are the functional and non-functional requirements for the use and implementation of the specification clearly defined? A. Yes Q.4. Is it possible to implement the specification across different domains? A. Yes Q.5. Is it largely independent from products of single providers, either open source or proprietary? A. Yes Q. 6. Is it largely independent from specific platforms? A. Yes Q. 7. Has the standard been written so that it can be delivered or used with more than one technology (for example XML and JSON)? A. Yes Q. 8. Has the specification been sufficiently developed and existed long enough to overcome most of its initial problems? A. Yes Q. 9. Are there existing or planned mechanisms to assess its conformity and implementation - for example conformity tests, certifications and plugfests? A. Yes Q. 10. Does it have sufficient detail, consistency and completeness for the use and development of products? A. Yes Implementation of the formal specificationQ. 11. Does it provide current implementation guidelines and documentation for the implementation of products? A. Yes Q. 12. Does it provide a reference (or open source) implementation? A. Yes (with qualification) Q. 13. Does it address backwards compatibility with previous versions? A. Yes Q. 14. Are the underlying technologies for implementing it proven, stable and clearly defined? A. Yes OpennessQ. 15. Is information on the terms and policies for the establishment and operation of the standardisation organisation publicly available? A. Yes Q. 16. Is participation in the creation process of the formal specification open to all relevant stakeholders (such as organisations, companies or individuals)? A. Yes Q. 17. Is information on the standardisation process publicly available? A. Yes Q. 18. Is information on the decision-making process for approving formal specifications is publicly available? A. Yes Q. 19. Are the formal specifications approved in a decision-making process which aims at reaching consensus? A. Yes Q. 20. Are the formal specifications reviewed using a formal review process with all relevant external stakeholders (such as public consultation)? A. Yes Q. 21. Can all relevant stakeholders formally appeal or raise objections to the development and approval of formal specifications? A. Yes Q. 22. Is relevant documentation of the development and approval process of formal specifications publicly available (such as preliminary results and committee meeting notes)? A. Yes Access to the formal specificationQ. 23. Is the documentation publicly available for implementation and use at zero or low cost? A. Yes Q. 24. Is the documentation of the intellectual property rights publicly available (is there a clear and complete set of licence terms)? A. Yes Q. 25. Is it licensed on a royalty-free basis? A. Yes Versatility/flexibility of the proposed standardQ. 26. Has the formal specification been used for different implementations by different vendors/suppliers? A. Yes Q. 27. Has the formal specification been used in different industries, business sectors or functions? A. Yes Q. 28. Has interoperability been demonstrated across different implementations by different vendors/suppliers? A. Yes End user effect of the formal specificationQ. 29. Do the products that implement it have a significant market share of adoption? A. N/A Q. 30. Do the products that implement it target a broad spectrum of end-uses? A.N/A Q. 31. Does it have strong support from different interest groups? A. Yes Q. 32. Is there evidence that the adoption of it supports improving efficiency and effectiveness of organisational process? A. Yes Q. 33. Is there evidence that the adoption of it makes it easier to migrate between different solutions from different providers? A. Yes Q. 34. Is there evidence that the adoption of it positively impacts the environment? A. No Q. 35. Is there evidence that the adoption of it positively impacts financial costs? A. No Q. 36. Is there evidence that the adoption of it positively impacts security? A. No Q. 37. Is there evidence that the adoption of it can be implemented alongside enterprise security technologies? A. Yes Q. 38. Is there evidence that the adoption of it positively impacts privacy? A. N/A Q. 39. Is it largely compatible with related (not alternative) formal specifications in the same area of application? A. Yes Q. 40. Is there evidence that the adoption of it positively impacts accessibility and inclusion? Maintenance of the formal specificationQ. 41. Does it have a defined maintenance organisation? A. Yes Q. 42. Does the maintenance organisation provide sufficient finance and resource to control short-to-medium-term threats? A. Yes Q. 43. Does the maintenance organisation have a public statement on intention to transfer responsibility for maintenance of it, if the organisation were no longer able to continue? A. Yes Q. 44. Does it have a defined maintenance and support process? A. Yes Q. 45. Does it have a defined policy for version management? A. Yes Related European standardsQ. 46. Is this an existing European standard or an identified technical specification in Europe? (Note: CEN, CENELEC or ETSI are the European standards bodies. Technical specifications provided by organisations other than CEN, CENELEC or ETSI can be under consideration to become a European standard or an identified technical specification in Europe.) A. Yes Q. 47. Does this specification or standard cover an area different from those already identified or currently under consideration as an identified European standard or specification? A. Yes |
|
Towards the end of 2020, AGI closed its Standards Committee; the AGI GEMINI Working Group now reports directly to the AGI Council. This has little affect on the above (& no affect on the logic of the case), but over time, it is likely that the URL links will change to reflect that organisation change. GEMINI itself will remain at https://www.agi.org.uk/gemini; the maintenance working group will be at https://www.agi.org.uk/agi-uk-gemini/ (rather than https://www.agi.org.uk/agi-groups/standards-committee). |
|
Do we have a status update on the adoption of this? |
|
Here's my recollection of discussion after the September 2021 OSB meeting, which was unfortunately conducted in a Slack conversation that (unknown to the participants) was deleted after a set time. Note: I am the lead editor for AGI's GEMINI One issue was whether it's clear who owns GEMINI and how to get involved / see it being governed. AGI should make a better link from their site to the GitHub site on which GEMINI is managed: https://github.com/agiorguk/gemini Another issue was what licence GEMINI is published under; AGI need to make the declaration of CC-BY more obvious (it's currently "hidden" on the page that used to be seen as the introduction) - see agiorguk/gemini#67 The third issue seemed to be that it 'only solves part of the problem', which is presumably the case for an standard. |
|
Thanks for the link to the Working Group - that was indeed unfindable to me as I was reviewing the materials. It is not clear from https://www.agi.org.uk/agi-uk-gemini/ what the governance of this standards profile is, or why it would be appropriate to consider it a standard using the criteria OSB does, namely
Can you help me with this please? Thanks! |
|
@webmink : see the GitHub links in my comment a couple of days ago. Meanwhile, AGI have acknowledged that weakness in their webpage, as a result of me feeding that back after the last OSB. They will also address the criticism that it isn't particularly obvious that it's published under CC-BY. They'll add something like: UK GEMINI (Geo-spatial Metadata Interoperability Initiative) is a set of metadata elements for describing geospatial data resources. The current version, Gemini 2.3, is provided free of charge by the AGI for the benefit of the GI community, under a Creative Commons license (CC-BY). It is maintained by AGI's GEMINI Working Group who can be contacted at https://github.com/agiorguk/gemini. Either on the 'group' page (the one you mention, @webmink) or the GEMINI page. |
|
@PeterParslow: Thanks for the reply! I followed all those links before posting and I did not find evidence satisfying the three bullets above, either historically or on an ongoing maintenance basis for the profile. This is not a specialist area for me, so I unfortunately need hand-holding to understand (I suspect some others on OSB are in the same boat). While the text you indicate is a helpful addition I would like to be reassured this standard is indeed open and not just publicly available. |
I give you the minutes & membership, at https://github.com/agiorguk/gemini/wiki. Records of discussion & meetings before February 2021 are in https://github.com/agiorguk/gemini/tree/main/archive (they were previously in a public Google document - just as public, but less easy to find) Isn't handling all changes as GitHub issues & minuting all meetings in a GitHub wiki, all in a public repository transparent enough? Perhaps what's missing by using GitHub is a clear indication of the organisational affiliation of the members of the working group. I don't expect they'd mind me adding that - it's "discoverable" for most of them anyway. wg members during 2021:
Is that the missing piece? |
|
If you look at the current issues, you will notice that 'change' has stalled for a while. This is due to AGI moving their website management from one hosting provider to another, and us (the wg) working out how to manage publication of GEMINI now. Normal service should be resumed in a few months: we're not web publishing experts! |
|
Thanks, that's useful, I appreciate your time working here! The contents of https://github.com/agiorguk/gemini/tree/main/archive especially provide the historic information needed to satisfy the OSB criteria as I understand them. It's still not clear to me how changes are accepted or rejected - the minutes in the wiki show everyone working but the norms are unspoken. Also, how would someone become a member of the working group? Forgive the suggestion, but since the Github page at https://github.com/agiorguk/gemini looks like an internal tool rather than the full expression of the governance of the profile, may I suggest adding a note to the README clarifying and a resource to the root of the repo called something like GOVERNANCE.md describing briefly where the profile came from, where to find who is involved, how to participate, the basis on which decisions are made about changes and explaining the public feedback and change proposal process is Github? |
|
That's a very useful suggestion; I'll transfer it across to our GitHub repository, if you don't mind - and look to add such a GOVERNANCE.md (a new one on me, but then I've only been working on GitHub for a year or so....) "how would someone become a member of the working group?" - in practice, just ask: giving some evidence of your expertise / reason to be part of the governance group |
|
Information on previous versions & organisations involved, in the context of discussion on patent rights: 1.0, 2004: Co-branded, Cabinet Office e-Government Unit & AGI The UK GEMINI Discovery Metadata Standard is a defined element set for describing geo-spatial, discovery level metadata within the United Kingdom. It is: 2.0, 2009 and 2.1, 2010 AGI ownership/copyright "This document has been produced by Rob Walker of Rob Walker Consultancy Ltd with funding from the residue of the National Geospatial Data Framework (NGDF)." 2.2, 2012 "Amended following proposals from UK Location Programme Metadata Working Group" - "UK Location" was a Defra funded project to help UK conform to European Commission INSPIRE. 2.3, 2018 "In 2018, changes were made that had been requested by various user communities as well as the revised metadata technical guidance from INSPIRE." - these were funded by Defra. The contract with AGI required AGI to grant Defra (the Crown) a perpetual licence to us any intellectual property and to indemnify Defra against any claim that might be made in future. And Defra required GEMINI to be released under an open licence (OGL or CC-BY - AGI's choice) Contract Reference: ecm_43275 (22949) of June 2015 |
|
Thanks for digging in there Peter. So would it be AGI's IPR policy that would govern disclosure of necessary patents in contributions by the participants in those various working groups? Any idea what that policy is? |
|
Simon, I'll ask AGI, but I don't see myself as 'owning' this 'challenge' i.e. the adoption of GEMINI by OSB. The challenge owner listed above (from March 2021) is Lawrence Greenwood (who was in CDDO in at least up to April 2021), but in September 2021, it was Keiran Millard of Geospatial Commission who proposed it - and tells me he hasn't heard anything since (other than seeing the minutes of the September meeting, or via me) |
|
Also, Simon, I'm tempted to ask: given that GEMINI is textual guidance on how to use a small group of ISO standards, how would it be possible for that guidance to have allowed embedded patents to sneak in? It's basically about writing text into text fields, choosing wisely from fields with controlled choices, and providing URLs to link out to things that the user of the standard wants to link to! |
|
It would indeed be subtle, yes! The main avenue would be by selecting data expressions that imply the use of particular algorithms, workflows or preparatory processing. Since patents are a "strict liability" matter, there is no need for the associated inventions to be directly expressed in the standard - simply causing others to implement them when processing the data structure would be enough to support a prima facie case. (Also note ISO standards can and often do have associated SEPs...) |
|
Simon,
So, I think you're objection would be to any & everything that comes along! If the ISO/IEC/ITU approach doesn't satisfy, then what can? https://www.iso.org/iso-standards-and-patents.html |
|
Indeed, there is no reasonable way to examine a standard and determine whether any patent would be infringed by implementing it. That's why getting either a FRAND statement (as ISO does) or a much better RF statement (as W3C does, and consequently assures an open source implementation is feasible) from all contributors is essential, and why casually-implemented specifications and profiles are always more risky and will always make me ask a probing question! |
|
Some possible ways forward on @webmink's concern:
If FRAND/RF is essential, should we revisit all the previous recommendations, to check they include a FRAND/RF declaration?
I think it's only worth doing that detailed work if entirely necessary. *note: I've never been the challenge owner / proposer, "just an expert witness". Most recently, this was Keiran Millard of the Geospatial Commission |
|
@PeterParslow I agree with you that OSB needs a process both to help proposals gather and deliver the evidence to support an informally-created specification being treated as an open standard. RF terms are fundamental to a standard being implementable safely as open source - see https://opensource.org/osr/ - so any standard brought to OSB needs to satisfy the board that it can be safely implemented. In the case of GEMINI, the historic evidence suggests that few commercially-motivated parties have had the opportunity to inject IP a la Rambus so my evaluation is veering towards approval on this point. Do you know if Ordnance Survey has any IPR claims apart from copyright in the methods embodied in the specification? |
|
I would be quite confident to say that OS has no such claims, and I think I have the authority to do so :) |
|
Oh that's great! I guess we need that added to the OSB info package then? Probably need to also add assertions for any other entity that's contributed. I guess there must be a manifest of IPR declarations associated with the base ISO standard? Are there any other entities who should provide "no conflict" statements in place of RF assurances? |
|
& that puts us back where we started - all institutions who may have contributed in the 18 years of GEMINI's existence... |
|
Do you think there are many that are either for-profit or proxies for for-profit entities? My view is we have to reach a level of confidence rather than secure absolute proof. |
|
The "original author", although working under contract, was a freelance professional for much of the time, so I guess we should get him to declare that he never sneaked any patent stuff in. There was another freelance guy who was a member some years back, and managed the AGI /HR Wallingford project. OS is considered by many to be "for-profit" (although we have to either re-invest or give it to the Treasury, as our sole share holder). Astun Technology allow one of their staff to contribute, as part of their in kind support to AGI; GeoPlace did the same for a while in the past (i.e. in both cases, the company allows the person to work for AGI for part of their time, and as part of that, they contribute to GEMINI HR Wallingford did some work, under contract to AGI. HS/2 had an active member for a while, That's the records since 2014; I don't have anything before then, other than the published versions - and I would be a bit surprised if AGI does. |
|
AGI are open to creating "a contributors agreement / IPR clause (or similar) like the one that OGC has" (OGC = Open Geospatial Consortium; their IPR policy is not unlike W3Cs. Would that be useful? It could only apply to contributions made to GEMINI from now on? If it's too legalise, it could reduce the number of people willing to participate, because they may need to get their employer's lawyers to look at it! |
|
AGI have now added links from the GEMINI home page (https://www.agi.org.uk/uk-gemini/) to:
|
|
Regarding Q. 24. Is the documentation of the intellectual property rights publicly available (is there a clear and complete set of licence terms)? |
|
Thanks @PeterParslow and @KeiranMillard for this helpful document. My only quibble is that in point 1, the ISO process does not guarantee that there are no patents; rather, it causes all holders to declare them. So accompanying the process that led to the ISO standardisation would be statements by participants of the patents they hold, which would then be termed "Standard-Essential Patents"or SEPs. These are ultimately what are problematic to Open Source, as I have recently explained. This is unlikely to be a problem for GEMINI for the reasons the document articulates so I will be happy to accept this IP history as adequate, but for future submissions the secretariat should note that the ISO patent policy does not lead to patent-free standards and it's important to validate there are no SEPs whose FRAND licensing would prevent normal Open Source implementation. |
|
Agreed, webmink. That's why I included "ISO’s less than ideal words"! However, this is a challenge to the whole OSB process for applying the UK government Open Standards Principles. As we discussed, there are very few standards organisations that have a process that in itself proves that their publications meet your interpretation of Principle 2, so the OSB needs a thorough evidence collection/presentation/judging process for almost all proposed standards. |
|
I don't think that's true Peter. All current de jure and de facto SDOs have effective IP policies for identifying SEPs, and some (such as OASIS) have practices leading to those SEPs being licensed to implementers on an RF basis. IOSB does need to take care to ensure the SEP status of standards it reviews is clearly documented in the submission process in future though. |
|
Most people would consider ISO, IEC and ITU as "current de jure SDOs", yet as you say, their IP policy doesn't meet your criteria. Still - this is a discussion wider than the GEMINI proposal. |
|
That's not what I said. They very effectively identify SEPs and require FRAND licensing. But open source projects can't use FRAND-licensed SEPs, as OSB already understands. |
|
Thanks @PeterParslow and @KeiranMillard. The patent evidence and other supporting information, in my opinion, clearly address the issues that were discussed during the 2022-02-16 meeting. |
|
As per housekeeping practices we are closing this with the status as a recommended standard |
Title
The adoption of UK GEMINI 2.3 as an open standard for metadata relating to geographic/geospatial data
Category
Challenge Owner
The Geospatial Commission is an expert committee that sets the UK’s geospatial strategy and promotes the best use of geospatial data.
Short Description
“UK GEMINI” provides guidance on how to publish geographic metadata in a way that conforms to relevant ISO standards and the UK’s implementation of the European INSPIRE regulations. UK GEMINI (GEo-spatial Metadata Interoperability Initiative) is a specification for a set of metadata elements for describing geospatial data resources. It has been produced and is maintained by the AGI GEMINI working group. It is provided free of charge by the AGI for the benefit of the GI community, under a Creative Commons license.
User Need
The government’s Geospatial Commission (GC) was established to improve the quality of key, publicly held data and make it easier to find, access, and use. Since its inception in 2017, the GC has commissioned several projects as part of its Data Improvement Programme. The projects, and the resultant recommendations, repeatedly recognize and highlight the importance of the discovery metadata for geospatial data and the vital role it plays in making geospatial data more Findable, Accessible, Interoperable, and Reusable.
As part of the geospatial commission's data discoverability project, a review of the most common metadata standards was conducted. It was found that The UK GEMINI Discovery Metadata standard and guidelines, describe data in a way that conforms to relevant international standards and EU INSPIRE guidance, and was the best fit for the requirements of the 6 geospatial partner bodies and DEFRA.
Therefore in order to improve the FAIR rating of data and unify best practice across government, the UK GEMINI standard should be adopted as an open standard and used to describe geospatial data sets and services.
Expected Benefits
Simplification
An easier standard for various audiences to use: those creating metadata (for datasets, with a separate view/entry point to GEMINI for services), those creating metadata tools, and those validating metadata instances.
Reducing implementation choices UK GEMINI makes metadata creators’ jobs easier by streamlining choices – either with instruction, or (more often) with a ‘heavy hint’ on how to select what to do among INSPIRE and ISO19115:2003
Making explicit some things that are hidden
Some things are rather hidden in other standards, perhaps appearing in examples, or as subordinate paragraphs in other requirements. Gemini makes these explicit.
Closer alignment with the ISO standards
Gemini aligns closely with ISO standards, in that a valid GEMINI record is definitely a valid ISO 19115:2003 record (or ISO 19119, in the case of metadata for a service) and simplifies whenever possible.
UK specific things: data.gov.uk
Extra things are required just to make metadata work in the current UK data infrastructure, including with data.gov.uk. UK Gemini provides guidance in these areas.
Community requests
GEMINI incorporates suggestions from multiple user communities in the UK, evidence to support the openness of the GEMINI development/maintenance. The AGI GEMINI working group is a) open to people who are not AGI members
b) currently includes members from companies (Astun Technology, Centre for Ecology & Hydrology, Geoplace), academics (British Oceanographic Data Centre, Edinburgh University, BGS), and government organizations (HS2, OS).
For example
Conclusion
The aim is to make UK geographic metadata more consistent and better quality.
Location data is already pervasive and its benefits will continue to increase throughout the economy and across all regions – supporting economic recovery, attracting investment, creating jobs, and boosting UK exports in an environmentally sustainable way. Initial research carried out in 2018 by the Geospatial Commission suggested that location data has a potential economic benefit to the UK of up to £11 billion per year.
Part of realising this is ensuring that data and therefore metadata is Findable, Accessible, Interoperable, and Reusable (FAIR). The adoption of the UK GEMINI standard would be a major step in realising this goal.
Functional Needs
Adoption of the UK GEMINI 2.3 as an open government standard.
Raise awareness and highlight the importance of the discovery metadata for data and the vital role it plays in making data more Findable Accessible Interoperable and Reusable.
The text was updated successfully, but these errors were encountered: