Observation callsign changes for GPSRO

[Liam Hall (liam-hall239)]

We found a bug in which opsinputs did not output the rising and setting flags for GPSRO into varobs.

We added a quality flag for the phase of occultation, rising (1) or setting (0), on the end of the observation callsign and split the last 7 characters of the callsign to include this flag at the end of the callsign.

An example is found in /home/users/liam.hall/desktop/opsinputs_pr/varobs.txt starting from line 1418.
The expected output is a 16 digit callsign with the last character to be either a 1 or a 0 - the phase flag.

[Adam Maycock (adammaycock)]

Out of interest, how has this bug not caused a problem (difference from OPS). Does VAR do anything with the rising / setting flag?

Also, I assume this change will result in kgo differences in nightly sith / malak testing, so we'll need to run those tests and generate new kgo before merging this PR. If that's the case, we'll set the waiting for kgo update label on this PR.

[Liam Hall (liam-hall239)]

Hi Adam, it hasn't caused any actual issues anywhere. Before, opsinputs cut off the very last digit of the callsign so the rising and setting information was lost at this point and is needed for current RO work.
The work I am doing now needs this flag and so I have made changes to my own version of VAR which splits the data into the respective occultation phase and then can be plotted.

For context, please see the graphs in ssa-desroziers-2025.

[Chris Thomas (ctgh)]

Thanks for this fix. It looks like you need to add the new variable to the test netCDF file.

[David Davies (DJDavies2)]

One of the ctests is failing for me:

test_opsinputs_varobswriter_071_VarField_bendingangle

There are also some coding norms failures.

[Liam Hall (liam-hall239)]

I have just made a change to the initial commit to remove the else statement and include the variable in the constructor. I have also removed any white space from the file.

I have checked the varobs file to see if the change is implemented and it has, with the last digit of the callsign only showing 1s and 0s. It is located here: /home/users/liam.hall/desktop/opsinputs_pr/varobs_pr.txt for cycle 20250101T0000Z if you need a copy for the ctests.

[Michael Cooke (mikecooke77)]

/home/users/liam.hall/desktop/opsinputs_pr/varobs_pr.txt

I believe for the ctests you will need to recreate the file using the python script (https://github.com/MetOffice/opsinputs/blob/develop/test/generate_unittest_netcdfs.py) as part of this PR.

[Neill Bowler (neill-b)]

So, the change tells me that 140 files have been modified. Surely this is not right?

[Liam Hall (liam-hall239)]

I also thought that there were so many modified files.
When running the generate_unittest_netcdfs.py file, it gave: "PermissionError: [Errno 13] Permission denied: 'testinput/001_VarField_pstar.nc4' " so I deleted the existing file after checking my permissions and then it worked, creating this as well as modifying the rest.
The error never mentioned the other files.

[Neill Bowler (neill-b)]

It appears that this changeset is the rogue one: 89ff34b
I'd suggest that you revert that one, then only modify the files that need changes.

[Liam Hall (liam-hall239)]

So to make sure I have this right, just push the generate_unittest_netcdfs.py and the file that gave an error or just the unittest python file and of course GnssroStationIDMetOffice.cc?

[Michael Cooke (mikecooke77)]

Not sure what to make of the CI runner. Matt Shin (@matthewrmshin) is this because we are using an older version of the JCSDA container? Should we move this to just using the azure UKMO CI as it will only ever be run in this environment?

[Matt Shin (matthewrmshin)]

CI should now be using our own container running on our own self hosted runner.

[Matt Shin (matthewrmshin)]

Build is now failing with a compiler error:

/var/tmp/opsinputs/pr-261/opsinputs/src/opsinputs/GnssroStationIDMetOffice.cc:59:41: error: no matching function for call to 'missingValue(const int&)'
   59 |   const int missing = util::missingValue(missing);
      |                       ~~~~~~~~~~~~~~~~~~^~~~~~~~~
In file included from /tmp/build-and-test-j5MEtK/ioda/include/ioda/ObsDataVector.h:24,
                 from /var/tmp/opsinputs/pr-261/opsinputs/src/opsinputs/GnssroStationIDMetOffice.cc:24:

[Chris Thomas (ctgh)]

Thanks, looks good!

[David Davies (DJDavies2)]

There are failures in the ctest test_opsinputs_varobswriter_071_VarField_bendingangle.

[Chris Thomas (ctgh)]

Liam Hall (@liam-hall239) I think the new file doesn't contains MetaData/qualityFlags. I haven't checked it for sure but the failed ctest output indicates that may be the case.

[Liam Hall (liam-hall239)]

Hi Matt Shin (@matthewrmshin), after speaking to Chris, it looks like the CI failure may be due to the recent
mobbs update. Local build and tests pass successfully from my workflow.

Please let me know if you're happy for the PR to be merged.

[Matt Shin (matthewrmshin)]

I've just hacked the CI image, so let's try building one more time.

[Matt Shin (matthewrmshin)]

I've hacked the CI build script to use Makefile not Ninja. This should work. Final try.

[Chris Thomas (ctgh)]

Matt Shin (@matthewrmshin) Thanks for getting it over the line.

[Liam Hall (liam-hall239)]

Ahh success - Thank you all!