-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multiple vertical Datum #79
Comments
Daniel, thank you for bringing this up. We have multiple areas in US waters with this same issue. Right now as a workaround, we have been creating two products from the same tile, with one in each of the two vertical datums. Then, we have to clip out the data in each tile to correspond to the vertical datum. This is an onerous and not an ideal solution, and we need something better for full compatibility with S104. We have been working on defining one vertical datum called our "National Charting Datum," which encompasses all currently used vertical datums we use for navigation. That way, the S102 rasters for each tile will be seamless as far as vertical datums from the mariner's perspective, and will ease compatibility with our S104 products. |
As noted by @anthonyklemm, the US is abstracting this problem into definitions of National Chart Height and National Chart Depth Datums. This reduces complexity for vendors when combining S-102 and S-104. If having multiple vertical datums in a single product is required, it may be more efficient to precompute which cells are governed by which datums and store this as an attribute in QualityOfBathymetryCoverage. This would remove intersecting cells with a vector in downstream applications, and remove any logic ambiguity. In the case of locks/gates, which was specifically mentioned in the proposal, a static S-102 product might best serve the mariner by reporting depths on the more depth conservative datum. Only when joined with S-104 would the mariner have depths relative to the real time water level. |
@GlenRice-NOAA I like the simplicity of your proposal. I had a similar comment: https://github.com/iho-ohi/S-102-Product-Specification/pull/90/files/7d677d636761598c212b578ee6a679084e1003f5#r1621861445 I will copy my comment here:
|
@giumas Since I am not formally part of the working group I am commenting here because it seems like the merits of the proposal should be discussed where the proposal is made (this issue) vs in the pull request (which addresses the technical merits of satisfying the proposal). As the chair perhaps @hasel001 has a different process in place. To be clear, I don't believe this proposal effects how we are building S-102 in the US. The problem is that with different countries solving these problems in different ways but with the same formats it will be difficult for vendors to implement maintainable solutions. @kusala9 may have some thoughts on this as well. Beyond the vendors, this complexity is also problematic for any other user that wants to access S-102 data through libraries like GDAL. The recent update adding S-102 to GDAL I think is really helpful and I am concerned that this proposal may break the GDAL model, at least as the driver is currently implemented. The original proposal concerns same resolution but overlapping coverage. I don't believe this is of value to the mariner since only one depth can be displayed at a time anyway. The tradition of navigational products is to provide the most conservative depth estimate for a location, and, in keeping with the principle of least surprise, navigational products should continue in this way. The right place to show the depth in places like locks is to have the water level adjusted in S-104 and added to S-102. This way S-102 provides for clear and safe navigation, and a more advanced product is available only if a timely S-104 is also available. If varying vertical datum information is needed this can be provided in QualityOfBathymetryCoverage since it should be 1:1 with each depth estimate reported. As far as the response here, there seems to be a some concern around overlapping but differing resolutions in the same file. The only value I can think of for overlapping coverage with differing resolution is to support overviews to speed data access. It would seem that any Bathymetry Coverage should have a corresponding QualityOfBathymetryCoverage or else the value of the bathymetry becomes ambiguous. The S-102 specification needs to deal with the generation of overviews if this is the case as how they are derived and used can have significant safety of navigation impact. If the problem here is really how to deal with overlapping coverage of differing resolution, then I think a preferable solution is to have a QualityOfBathymetryCoverage object that specifically describes the quality of the information provided and includes the vertical datum information as well. |
The more we go into this rabbit hole, the less I am convinced that having overlapping coverages at the same grid resolution in a h5 file is adding (enough) value to make acceptable the increase in complexity. By adding a reference to the S-101's SoundingDatum surface (https://github.com/iho-ohi/S-102-Product-Specification/pull/90/files#diff-c166417585b1423948d7df2d60a2ae3a4edf1917b6d3d7bbfc12dfb5f1ae9147R861), are we somehow forcing an approach in the ECDIS S-100 visualization that is likely out of the mandate of this PT? I believe that, for each display scale range, there should be 1 and only 1 depth value for each grid cell. To summarize:
|
@GlenRice-NOAA My only concern your proposed method is that I am not sure datum does not seem to me a QualityOfBathymetryCoverage asset. Datum is not really a measure of quality, rather an measure of adjustment to the data value. I think perhaps we need to find another object elsewhere to store this field. Otherwise, I appreciate the simplicity of the solution. Perhaps as a workaround to ensure we get the next version out on time is attach an supplemental file delineating the location of each datum and the transitory areas. |
apologies I haven't had time to look at this in more detail. My only comment at this stage is that the ECDIS OEMs require a specification which is clear, concise and without ambiguity. The current annexes in S-98 Annex C assume no overlapping data, but the user could be prompted to select a single dataset if need be. the OEM won't "compute" with the data, so they won't interpolate based on grid cell dimensions or parameters. I think adding in domain polygons could be done but would require a level of testing and development which we don't really have, given the imminent deadlines for S-102, S-164 and S-98 Annex C....That said, I recognise the problem but can't we fix this using multiple cells and masking the data values in each cell so that the boundaries of each datum area are covered?? |
Sticking with BSHs original proposal, if we:
We get a pretty simple and already well-defined solution (by @RohdeBSH and other BSH staff) and only lack the "partial grid cell" option originally proposed. This way we don't need to think about adding vertical datum attribute to quality of bathymetric data coverage either. |
@tfilppula, just to be sure that I understood your proposal properly. Roughly speaking, you want to put the content of multiple S102-PS-2.2 h5 files into one S102-PS-3.0 h5 file, using multiple feature instance groups and bathy coverages. Correct? If yes, with or without a shared quality of bathy coverage? |
@giumas |
@tfilppula, thank you for the clarifications. I believe that you are proposing 2 substantial differences (removal of domain extent polygon and not allowing overlapping depth data) from the original proposal that quite reduce complexity. Should it still be allowed to obtain the same result using multiple .h5 files with a single vertical datum? |
During Day 1 of S-102PT18, it was recognized that the topic of Multiple Vertical Datums can be effectively separated into discrete issues as follows:
Please consult with your organization and prepare to discuss at Wednesday's meeting what you've found. (Also, if I have made things ambiguous with my wording, please suggest clarifications, and I will do my best to implement them. Thank you.) |
Hi @ all, these are the answers from BSH:
The question does not arise. The BSH PR#90 proposal never asked to choose between one of the two options.
A vertical datum surface cannot be described by a bounding box. A bounding box can never represent the required level of detail of the real world.
This is a very simplified question for a highly complex problem. We would like to present something live on this point. |
Hello all, answers from FIHO:
S-102 should allow multiple features (coverages) to exist in the same file. This allows for greater flexibility for the producer, as producer can opt to either using multiple files or multiple coverages in one file.
PS should allow both, using either domain extent polygon or bounding box. Allowing also the use of domain extent polygon brings greater flexibility for the producers.
FIHO thinks that overlapping data should be considered as overlapping depth information (and uncertainty, where present). In other words: overlaps between Data-NoData (fill value) and NoData-NoData should be allowed. After all, its conflicting depth information we seek to avoid. In the border of two producing countries some overlapping data should be allowed, but it's best to keep overlaps to a minimum. |
Note 1: On 1751 UTC on 12 June, I changed the wording of the voting question to better achieve correctness and clarity. It is well in advance of the voting period, so there is ample time for everyone to re-read. Sorry for the confusion, and I hope everyone has a great rest of the week and weekend. Note 2: On 1615 UTC on 13 June, I corrected a typo in the voting closure time. It now reads 15 June vice 1500 June. During Day 2 of S-102PT19, this issue was discussed at length but remains to be resolved. I apologize for allowing ambiguity in the process up to this point, and I assure you that I won't allow it to happen in such a way again. Thank you for being patient with my leadership. The following procedure will be used to resolve the issue so we can move forward with the development of the S-102 Product Specification:
Should Ed. 3.0.0 of S-102 ALLOW multiple feature instances for BathymetryCoverage (PR#90)? Vote "Yes" to ALLOW the use of multiple feature instances for BathymetryCoverage.
That is to say, voting shall open at 13 June 2024, 1000 UTC, and voting shall close at 15 June 2024, 1200 UTC.
Name
Thank you for your participation and support. Please let me know if you have other questions or issues. Best regards, Lawrence Haynes Haselmaier, Jr. |
Note: I edited my comment based on rewording of the voting question. @hasel001, as you know, I was unable to join today meeting. So it may have been already clarified, but from your #79 (comment), it is unclear to me what happens in case that we allow the use of multiple feature instances for I am asking because the PR related to this ticket (#90) contains much more than just allowing the use of multiple feature instances for
In other words, I don't believe that voting for the current question can implicitly mean that we support also the other elements in the PR (in particular, the latter one), also because some of them are not even mentioned in the description of this ticket. |
@giumas I edited the question (about an hour ago according to GH) to read: Should Ed. 3.0.0 of S-102 ALLOW multiple feature instances for BathymetryCoverage (PR#90)? Vote "Yes" to ALLOW the use of multiple feature instances for BathymetryCoverage. You may still have questions, but I anticipate they will be different ones in this context. Can you take another look and ask again? |
@hasel001 I edited my comment based on the edited question. In short, I believe that my points are still valid. |
Hi @giumas , S-100 ed. 5.2.0 contains the answers.
It is not true, that this is a new key element. Please remember the use of
That's not what your reference said. The referenced paragraph is about the
It is not true, that this is a new key element. Please remember the attribute overriding is currently not prohibited in S-102. It is a concept of S-100 (Part 10c section 10c-9.7.1).
That is your personal opinion and you are welcome to share it. We have a different opinion. As already explained, the
But only in the widest sense. I would also have preferred to reference the MRN of the |
@RohdeBSH, it is quite explicit in the PR that "When using multiple vertical datums, the use of the domainExtent.polygon is mandatory, which means that the BoundingBox is not specified in this case." It is difficult to comment on a moving target if you keep changing opinion. In your own words: "And again the domainExtent.polygon can be implemented as bounding box (S-100 Table 8-8 Note 1)." |
@giumas If you don't want to use the multiple features for S-102, then don't. It won't change anything for you. The bounding box on the Feature Instance Group is only mandatory in the S-102. It has always been optional in the S-100 (S-100 Table 10c-12) => concept of If it still isn't clear after this last picture, then I can't help you any further. That's the simplified structure of #90. |
@RohdeBSH, I am just delivering what @hasel001 asked to do. Going back to the technical part of your comment, it is at least ambiguous whether "When using multiple vertical datums, the use of the domainExtent.polygon is mandatory, which means that the BoundingBox is not specified in this case." means that:
Hope that you are at least open to clarify such a statement to be aligned to what you put in the picture. |
@giumas Finally, the bounding box geometry is not gone, it just encoded somewhere else. |
@RohdeBSH, given that this image is correct: #90 is introducing 2 new ways of managing multiple vertical datums (both within a single product): one with polygon and one with bounding box. This translates to me that we need to provide guidance (e.g., on valid data overlap) on 3 variants of S-102 for this specific issue of multiple vertical datums:
The latter variant is the less defined in my opinion. For instance, what is the meaning that such a bounding box is "an independent vector geometry based on the SoundingDatum surface from S-101"? |
I also have a 4th & 5th ;) Would you like us to disallow bounding box geometry in |
Just a question to understand this better: |
Because the boundaries of your vertical datum area do not necessarily have to align with the grid axes. |
We do not support the mechanisms proposed at the start of this issue and the associated pull request. Our objection does not concern specific data types and their use. Our objection is based on how this proposal reflects on the purpose of S-102. Our objection should not reflect on the work BSH has done to make this proposal. On the contrary, we appreciate the effort that has been made to build a common understanding of how S-102 can be used. We believe that reporting multiple depths for the same cell and splitting cells is antithetical to the purpose of S-102. To our mind, S-102 is not a higher resolution version of S-101 bathymetry, but is meant to act as a digital twin to the bathymetry in the real world. If there is a perceived need to describe multiple depths in the same location, the product is not at sufficient resolution to support a meaningful description of the bathymetry. If a product is at sufficient resolution, changes in vertical datum to support integration with S-102 can be supported with the Raster Attribute Table as described here. Other parts of the S-100 framework note that overlapping values should be avoided, so we believe that the S-100 model supports this claim. We believe that supporting this proposal will create an antipattern in S-102 that, once part of the S-102 3.0 product specification, will be forever part of S-102. S-102 3.0 is not for test and evaluation purposes but is instead meant to be an operational version. As is noted by @kusala9, there is not sufficient time to test and evaluate this kind of change before the 3.0 release. Because the perceived problems targeted by the associated pull request can be solved by making suitably higher-resolution products and splitting into separate datasets, we believe that the right course of action at this time is to:
|
Good afternoon Project Team, After conferring with the S-100WG Chair; and after considering potential impacts to S-98, other S-100 specifications, etc.; I realize I have jumped the gun by forcing a vote on the Multiple Vertical Datums issue. In particular, it would be imprudent and inconsiderate of our Project Team to push ahead with a decision whose effects would be felt in a much larger radius than our own. Our overall objective is to develop a robust, useful, and interoperable product specification. To better support this objective, I am calling for us to change course as follows:
I understand that much effort from many contributors has been put forth toward this topic, and I immensely appreciate everyone’s participation. While I recognize the importance of fostering progress, I also recognize the gravity of the situation and the onus to reach a workable, conscientious solution. In other words, I know it’s important to work quickly, but I also know it’s important to get things right. Thank you for being patient with me as we navigate this trying process. As always, please let me know if you have questions, comments, or feedback in general for me. Thank you, and I hope you all have a great weekend. Best regards, Lawrence Haynes Haselmaier, Jr. |
After conferring again with the S-100WG Chair, I have decided to hold a VTC (via Google Meet) 27 June 2024 from 1300 to 1600 CEST. The meeting can be accessed here. This VTC's purpose is for us to resolve the Multiple Vertical Datums issue with a sustainable approach that considers downstream/cross-stream implications for OEMs, S-98, S-104, etc. Our intent is to work through the BSH proposal to ensure: Because the implied alternative for the BSH proposal (with the PS in its current state) is to break up datasets at vertical datum boundaries, we should include a clause explicitly describing this option. It is important to consider that we (S-102PT) will not perfectly change the PS on the first try. However, it is imperative that we reach consensus on a workable solution at this VTC and so lay the foundation for future progress. Please let me (Lawrence, hasel001@gmail.com) know if you have any questions or issues. Thank you, and I look forward to meeting with you soon. Best regards, Lawrence Haynes Haselmaier, Jr. |
Dear colleagues,
During the practical implementation of the S-102 and the tests with the pilots, the BSH came across an issue that should be discussed. This concerns the specification of the vertical datum within the S-102.
Current facts:
Problem description:
The BSH uses a regular product scheme for the creation of S-102 products. No regional topographical features can be taken into account within a regular product scheme, as otherwise a simple automatic creation of the S-102 products is no longer possible. We are therefore faced with the issue that an S-102 product contains topographical features that cannot be mapped using a single vertical datum. In Germany, this issue affects tidal areas and canals/rivers with locks.
In tidal areas, harbors or harbor basins are often separated from direct access to the sea by a gate/lock in order to ensure a defined water level in the harbor area at all times.
The entrance/exit to canals can also be separated by a lock. Here too, the lock serves to maintain a defined water level and/or to compensate for a height difference in the terrain. The same applies to rivers.
In the correspondingly separated areas, a different local vertical datum is typically used than in the sea.
The presence of areas with different vertical datum in an S-102 product can be considered ubiquitous.
Examples:
Germany, North Sea + Baltic Sea, entry/exit to the Kiel Canal:
Germany, Cuxhaven, "Fischerreibecken" (basin):
We also found similar patterns in a data set from SHOM (FR) for St. Malo.
Solution 1:
One solution could be to design the S-102 product in such a way that the cases described do not occur. The spatial extension of the products would then have to be manually checked along the entire coast and specially adapted.
However, this approach is not practicable.
Solution 2:
Another solution would be to divide the depth coverage according to the coverage of the vertical datum. A FeatureInstanceGroup would be created for each vertical datum within the FeatureContainerGroup (BathymetryCoverage) (e.g. BathymetryCoverage.01 & BathymetryCoverage.02). The BathymetryCoverage feature is thus represented by several instances. This procedure is described in S-100 Ed. 5.2.0 section 10c-9.6. The two coverages would then each represent a vertical datum.
S-100 Ed. 5.2.0 also provides a solution for marking the vertical datum for the respective depth coverage. Section 10c-9.7.1 describes the overwriting of attributes. The vertical datum is mentioned as an explicit example. For the solution, this means that the different vertical datum must be specified as an attribute in the FeatureInstanceGroup. This different vertical datum therefore overwrites the specification in the root group for this FeatureInstanceGroup only.
The question remains whether the two depth coverages may overlap. From the BSH's point of view, the answer is yes. Experience has shown that locks/gates do not follow the grid of the S-102 depth coverage. This vector data can also cross grid cells diagonally. In order to be able to reliably determine in a grid which depth applies on one side and on the other side of the lock/gate, it is necessary to correctly map the geometry of the boundary object. S-100 Ed. 5.2.0 offers the "domainExtent.polygon" approach for this in section 10c-9.7 Table 10c-11. This polygon defines the validity of the respective grid/FeatureInstanceGroup. The geometry of the lock/gate therefore coincides with the boundary of the polygon and forms a unique intersection line.
To ensure that a depth can be determined from a grid cell in every case, it is necessary for the depth coverages to overlap at least in the boundary area. A complete overlay is therefore also possible, but at the cost of the file size.
Recommendation
The BSH recommends the implementation of solution approach 2 and has prepared a corresponding dummy dataset. Unfortunately, we were not able to add the different vertical datum attribute to the dummy dataset. We have therefore faked the images and you will have to imagine it in the HDF5 file.
VerticalDatum_DummyDataset.zip
The text was updated successfully, but these errors were encountered: