Skip to content

location_model_update_kas#123

Merged
jirhiker merged 26 commits intopre-productionfrom
location_model_update_kas
Sep 11, 2025
Merged

location_model_update_kas#123
jirhiker merged 26 commits intopre-productionfrom
location_model_update_kas

Conversation

@ksmuczynski
Copy link
Contributor

Why

This PR addresses the following problem / context:

  • Fields need to be added to the database model and associated files so that legacy data from NMAquifer can be migrated effectively and new data can be stored appropriately.

How

Implementation summary - the following was changed / added / removed:

  • Added new fields to the Location database model.
  • Updated the CREATE, RESPONSE, and UPDATE schema models to reflect the addition of new fields in the database model
  • Updated the POST, PATCH, and GET tests to reflect the addition of new fields
  • Updated the util.py transfer script file to include the new fields.

Notes

Any special considerations, workarounds, or follow-up work to note?

  • TODO: Create functions so that the state, county, and quad_name fields are auto-populated by the API.
  • TODO: confirm whether PointID should map to location.name or thing.name

db/location.py Outdated
Comment on lines 57 to 60
elevation_accuracy: Mapped[float] = mapped_column(nullable=True)
elevation_method: Mapped[str] = lexicon_term(nullable=True)
coordinate_accuracy: Mapped[float] = mapped_column(nullable=True)
coordinate_accuracy_unit: Mapped[str] = lexicon_term(nullable=True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will all elevation_accuracy values be in meters? If so, should elevation_accuracy be renamed elevation_accuracy_m? Or, should we add a field called elevation_accuracy_unit?

Same for coordinate_accuracy: will it always be in degrees since we are storing coordinate values in WGS84.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the NMAquifer documentation, elevation accuracy is in units of feet. I don't think we need a separate units field here. We can either append "_ft" to the field name or just note the units in the documentation describing the field. I'm kind of inclined to do both.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I generally prefer to avoid Hungarian Notation i.e. elevation_accuracy_m

Copy link
Contributor

@jacob-a-brown jacob-a-brown Sep 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as it's documented somewhere that should be okay. If the units are always going to be the same we don't need a separate ..._unit table and can note it. Then in the response schema we can return a units field that evaluates to the correct unit.

Should the elevation_accuracy always be meters because elevation is stored in meters? And coordinate_accuracy should always be in decimal degrees because lat/long are stored in decimal degrees?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that seems logical

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I've read, best practice and industry standard is to store coordinate accuracy as a single, consistent linear unit, with meters being favored.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Elevation in NMAquifer is stored in feet, so the elevation accuracy should also be stored in feet.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the transfer script we can convert elevation accuracy units from ft to meters

As for coordinate accuracy I'm okay to store that in meters, too, if that's the standard

elevation_accuracy: float | None = None
elevation_method: str | None = None
coordinate_accuracy: float | None = None
coordinate_accuracy_unit: str | None = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we decide to keep a coordiante_accuracy_unit field should this always be degree?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason I added a units field for coordinates is because the NMAquifer lookup table, LU_CoordinateAccuracy, has units of second, minute, and meters. I thought a one-to-one mapping would be the fastest way to migrate the data, but we could convert coordinate accuracy units into a standard unit during the migration. From what I've read, it seems best practice and industry standard is to store coordinate accuracy as a single, consistent linear unit, with meters being favored.

Converting arc-seconds to linear units is a little tricky because the linear distance of a longitude
arc-second depends on the latitude, so lat/long will need to be a variable in the conversion. Just something to be aware of.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll write a function for the second/minute conversion to meters and be sure to highlight it in the transfer PR for review. Hopefully Python has a function or library to do this already, but that may not be the case 🫰

Comment on lines +215 to +221
elevation_accuracy=row.AltitudeAccuracy,
elevation_method=row.AltitudeMethod,
# created_at=created_at,
# point_accuracy=row.CoordinateAccuracy,
# point_method=row.CoordinateMethod,
coordinate_accuracy=row.CoordinateAccuracy,
coordinate_method=row.CoordinateMethod,
nma_coordinate_notes=row.CoordinateNotes,
nma_notes_location=row.LocationNotes,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can update this when I merge your work with the lexicon updates, but now that we are not using lookup table codes I think we'll need to amend this section (and for other tables/fields that are lexicon/lookup tables) to get the MEANING from the LU tables. Otherwise they won't be found in the lexicon term table and we'll run into errors.

On that note, I'll wait to open a PR for lexicon so that I can take care of that in the transfer scripts where it is relevant.

@codecov-commenter
Copy link

codecov-commenter commented Sep 11, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

Files with missing lines Coverage Δ
db/location.py 94.73% <100.00%> (+0.79%) ⬆️
db/thing.py 100.00% <100.00%> (ø)
schemas/location.py 92.45% <100.00%> (+3.56%) ⬆️
tests/conftest.py 100.00% <ø> (ø)
tests/test_location.py 100.00% <100.00%> (ø)

@jirhiker jirhiker merged commit d39b956 into pre-production Sep 11, 2025
3 checks passed
@ksmuczynski ksmuczynski deleted the location_model_update_kas branch September 24, 2025 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants