stratum mislabled in AI gird shapefile #30
Comments
|
Ok. Is this the official shapefile for the GOA/AI group that is used to produce stock assessment products? Or is this a realignment that Mark did that hasn't been adopted by the group as the official stratum boundary file? |
|
I think the latter. Good point, probably worth checking with @Ned-Laman-NOAA. |
|
@Ned-Laman-NOAA Is the shapefile Emily shared the official shapefile for the AI? |
|
I'll add that I double checked that everything else (columns, observations, data dimensions, etc) were otherwise exactly the same between the shapefiles. I took a moment to recalculate the areas and perimeters of each grid from scratch (a great sanity-check suggestion form @sean-rohan-NOAA) and they are the same between the files. I feel confident that, between these two shapefiles, the only difference is in the specification of those few stratum. |
|
@Ned-Laman-NOAA @EmilyMarkowitz-NOAA Was there a final decision on this? Noticed it's been a month and a half since this issue was opened. |
|
No updates from me, as I am just the anticipated user. @Ned-Laman-NOAA were you able to talk with people on your end? |
|
Remind me (again) of the origin of the shapefile with the obviously funky strata? E.g., is it U:\alaska\constructs\zones\strata\aigrid? U:\alaska\constructs\zones\strata\ai_strata? Or does it originate from one of the Oracle tables on the AI schema? |
|
The AI stratum shapefile is U:\alaska\constructs\zones\strata\ai_strata,
which I used based on your recommendation in an email on May 24, 2022.
…On Tue, Apr 18, 2023 at 1:26 PM Ned Laman ***@***.***> wrote:
Remind me (again) of the origin of the shapefile with the obviously funky
strata? E.g., is it U:\alaska\constructs\zones\strata\aigrid?
U:\alaska\constructs\zones\strata\ai_strata? Or does it originate from one
of the Oracle tables on the AI schema?
—
Reply to this email directly, view it on GitHub
<#30 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/APULYIL647HLZPPLDUJIEI3XB32IPANCNFSM6AAAAAAVOADPAU>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Sean K. Rohan, PhD (he/him)
Research Fish Biologist
Bering Sea Bottom Trawl Survey Group
Resource Assessment and Conservation Engineering Division
NOAA Alaska Fisheries Science Center
7600 Sand Point Way NE
Seattle, WA 98115
***@***.***
|
|
In that recommendation, I cited both ai_strata & aigrid as the authorities for this information. I'm now realizing that those two shapefiles aren't even in synch. The ai_strata shapefile does not include Bowers Ridge as the aigrid shapefile does. I believe that some of the stratum discrepancies are on Bowers Ridge but we haven't sampled out there on an Aleutian survey for decades. Let's first agree on whether or not to include Bowers Ridge in your navmaps package to start narrowing down where we need to address these issues. |
|
My suggestion is that Bowers Ridge shouldn't be included because the strata
out there aren't included in index calculations and aren't in the standard
survey. Based on that logic, they could also be dropped from the survey
grid.
As an aside, Bowers Ridge also shouldn't be included in the VAST AI
extrapolation grid but was in the grid the last time I checked. Maybe it's
changed.
…On Tue, Apr 18, 2023 at 4:26 PM Ned Laman ***@***.***> wrote:
In that recommendation, I cited both ai_strata & aigrid as the authorities
for this information. I'm now realizing that those two shapefiles aren't
even in synch. The ai_strata shapefile does not include Bowers Ridge as the
aigrid shapefile does. I believe that some of the stratum discrepancies are
on Bowers Ridge but we haven't sampled out there on an Aleutian survey for
decades. Let's first agree on whether or not to include Bowers Ridge in
your navmaps package to start narrowing down where we need to address these
issues.
—
Reply to this email directly, view it on GitHub
<#30 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/APULYIICJRJRF3U2QRKC5TLXB4PKHANCNFSM6AAAAAAVOADPAU>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Sean K. Rohan, PhD (he/him)
Research Fish Biologist
Bering Sea Bottom Trawl Survey Group
Resource Assessment and Conservation Engineering Division
NOAA Alaska Fisheries Science Center
7600 Sand Point Way NE
Seattle, WA 98115
***@***.***
|
|
There is a copy of the aigrid shapefile with Bowers Ridge clipped out here (G:\AI-GOA\shapefiles\aigrid_clipped) |
|
Ok. So the station grid with Bowers Ridge clipped out
(G:\AI-GOA\shapefiles\aigrid_clipped) is the official survey grid shapefile
for the AI?
…On Tue, Apr 18, 2023 at 4:51 PM Ned Laman ***@***.***> wrote:
There is a copy of the aigrid shapefile with Bowers Ridge clipped out here
(G:\AI-GOA\shapefiles\aigrid_clipped)
—
Reply to this email directly, view it on GitHub
<#30 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/APULYIPI4HVAI5WEIBCDBYLXB4SGZANCNFSM6AAAAAAVOADPAU>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Sean K. Rohan, PhD (he/him)
Research Fish Biologist
Bering Sea Bottom Trawl Survey Group
Resource Assessment and Conservation Engineering Division
NOAA Alaska Fisheries Science Center
7600 Sand Point Way NE
Seattle, WA 98115
***@***.***
|
|
The shapefile here (G:\AI-GOA\shapefiles\aigrid_clipped) is literally the shapefile from here (U:\alaska\constructs\zones\strata\aigrid) with Bowers Ridge clipped out. |
|
Hi Ned, It looks like the aigrid_clipped shapefile you reference has grid cells with strata 793 and 794 that are west of the boundaries for strata 793 and 794:
Therefore, I still think it would make sense to adopt Em's suggestion to change stratum designations for grid cells in the shapefile, as described earlier in this thread:
|
|
Sean, It sounds like you're right and we should be using the corrected and clipped shapefile Emily references. Here is where my hesitancy lies. Before we commit to making the change, we need to have written documentation detailing how the corrected stratum labels were assigned and applied. Then, there is a whole system of shapefiles, Oracle tables, and public-facing databases that will need to receive these changes to Aleutian stratum areas. Before that happens, there nees to be a coordinated communication campaign to inform data users about the changes. Clear documentation of how and why these changes are being made will form the basis for communication and rollout of the changes. In short, I don't think we can simply swap shapefiles in akgfmaps and call it good without doing our diligence among all of our sources of data and data users. In fact, I think all of the other changes need to be in place and clearly communicated before your package distributes the new map. Thoughts? |
|
Would it be helpful to add a page documenting this change to an akgfmaps pkgdown page or to the gap_products quarto book? @sean-rohan-NOAA I could help create the former (small lift, though you might already have experience) and/or latter (create a page/section for you to edit) if there is interest. Here is what a pkgdown page for the gapindex package looks like. |
|
It is my sense that the akgfmaps package is rapidly becoming the face of our spatial information so I think it would be helpful to add this type of documentation alongside the package. I remain hesitant to try to work backwards from the package to our internal resources when trying to disseminate these changes because the rate of change will be much much slower for the latter. |
|
My first impression is that we don't need a coordinated campaign to
communicate with end users about changes to shapefiles beyond what is
already being done because that isn't consistent with how folks have been
using or obtaining shapefiles. Until recent times, many (most?) folks have
been using their own bespoke versions of shapefiles or making changes to
shapefiles during their data wrangling to handle errors with shapefiles.
Bespoke shapefiles have been used in an operational capacity. There is at
least one case where an 'official' shapefile with errors has been used for
both research and production (to generate the AI extrapolation grid that
has been the default in VAST-- includes Bowers Ridge). There were
discrepancies between survey areas for design and model-based indices of
abundance. There is a another case where stratum areas were likely
incorrectly transcribed to racebase so there is a discrepancy between
stratum areas that can be calculated from official shapefiles and official
stratum areas in racebase.
In the last couple of years, folks have been switching to using the
akgfmaps package. I think that's because it simplifies their R-based
workflows and ensures that they are using the same shapefile as others. A
core principle for the package is that the information in the shapefiles
should match the information in racebase.
Shapefiles in akgfmaps are used to produce the following:
- Survey data reports
- Presentations
- Survey charts
- NRT temperature maps
- Stratum area (EBS/NBS)
- Extrapolation grids for VAST (EBS/NBS)
- Novel research products
One benefit of the package is that a user can determine which version of a
shapefile they're using based on which version the akgfmaps package they're
using. To communicate about changes, new releases of akgfmaps include
change notes with versioned releases and changes are also logged in the
NEWS file.
Currently, the AI survey grid shapefile is often not being used because it
is not reliable since it has stratum errors. Consequently, package users
are creating their own bespoke version of the shapefile to handle the data
errors, which I suspect leads to inconsistencies.
…On Mon, Oct 2, 2023 at 2:51 PM Ned Laman ***@***.***> wrote:
Sean,
It sounds like you're right and we should be using the corrected and
clipped shapefile Emily references.
Here is where my hesitancy lies. Before we commit to making the change, we
need to have written documentation detailing how the corrected stratum
labels were assigned and applied. Then, there is a whole system of
shapefiles, Oracle tables, and public-facing databases that will need to
receive these changes to Aleutian stratum areas. Before that happens, there
nees to be a coordinated communication campaign to inform data users about
the changes. Clear documentation of how and why these changes are being
made will form the basis for communication and rollout of the changes.
In short, I don't think we can simply swap shapefiles in akgfmaps and call
it good without doing our diligence among all of our sources of data and
data users. In fact, I think all of the other changes need to be in place
and clearly communicated before your package distributes the new map.
Thoughts?
—
Reply to this email directly, view it on GitHub
<#30 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/APULYIMYJ4MHAAENM6WW7XTX5MZMDAVCNFSM6AAAAAAVOADPAWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBTHAZDAMZXGA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Sean K. Rohan, PhD (he/him)
Research Fish Biologist
Bering Sea Bottom Trawl Survey Group
Resource Assessment and Conservation Engineering Division
NOAA Alaska Fisheries Science Center
7600 Sand Point Way NE
Seattle, WA 98115
***@***.***
|
|
My preferred approach to removing Bowers Ridge from the standard AI grid
would be to keep all of the cells in the shapefile itself and filter out
Bowers Ridge cells when folks retrieve AI layers using get_base_layers().
Would still need to correct erroneous stratum 793 and 794 cells.
…On Mon, Oct 2, 2023 at 3:23 PM Ned Laman ***@***.***> wrote:
It is my sense that the akgfmaps package is rapidly becoming the face of
our spatial information so I think it would be helpful to add this type of
documentation alongside the package. I remain hesitant to try to work
backwards from the package to our internal resources when trying to
disseminate these changes because the rate of change will be much much
slower for the latter.
—
Reply to this email directly, view it on GitHub
<#30 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/APULYIKBQ7OJXL2IAOIWNNTX5M5D5AVCNFSM6AAAAAAVOADPAWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBTHA2TENZWGM>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Sean K. Rohan, PhD (he/him)
Research Fish Biologist
Bering Sea Bottom Trawl Survey Group
Resource Assessment and Conservation Engineering Division
NOAA Alaska Fisheries Science Center
7600 Sand Point Way NE
Seattle, WA 98115
***@***.***
|
|
@sean - take a look at my attempt to stratum correct the aigrid shapefile here G:\AI-GOA\shapefiles\aigrid_stratum_corrected. I haven't had a chance to do the same from ai_strata but will get there if this attempt looks like an improvment. |
|
Reviewing the shapefile the strata look correct for the AI. However,
it looks like there are a couple of new fields in the shapefile named
Field1 and Field2.
- Field1 appears to be the same as AIGRID_.
- Field2 contains 1017 'Y's and 56 'N's and 9324 '[NULL]'s.
Can Field1 and Field2 be omitted from the shapefile?
…On Thu, Oct 5, 2023 at 4:38 PM Ned Laman ***@***.***> wrote:
@sean <https://github.com/sean> - take a look at my attempt to stratum
correct the aigrid shapefile here
G:\AI-GOA\shapefiles\aigrid_stratum_corrected. I haven't had a chance to do
the same from ai_strata but will get there if this attempt looks like an
improvment.
—
Reply to this email directly, view it on GitHub
<#30 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/APULYIK74EKIQXRZFINXV5TX55AGVAVCNFSM6AAAAAAVOADPAWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBZG44TMNJXGU>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Sean K. Rohan, PhD (he/him)
Research Fish Biologist
Bering Sea Bottom Trawl Survey Group
Resource Assessment and Conservation Engineering Division
NOAA Alaska Fisheries Science Center
7600 Sand Point Way NE
Seattle, WA 98115
***@***.***
|
|
Field1 and Field2 can be omitted from the shapefile. |
|
Alright, I removed Field1 and Field2 from the shapefile and pushed to the slope branch. Now just waiting for tests to finish. |
Issue
A few stratum seem to be mislabeled in the AI gird shapefile currently included in akgfmaps. I had encountered this issue last year when I was putting together the survey progression/temperature maps and Mark Z was able to provide me with a new shapefile where these were corrected.
Suggestion
Replace this shapefile with this shapefile.
Notes
Differences between the shapefile in akgfmaps and survey-live-temperature-map
As a short hand, "r_" is grid data provided from survey-live-temperature-map and "a" is grid data provided from akgfmaps
When I plot bounding boxes around the corresponding INFPC regions (ID'ed by stratum via
goa.goa_strata), you can see that the stratum are in the issue (this is how I noticed the error in the shapefile).Using the AI Grid shapefile from akgfmaps:
Using the shapefiles in survey-live-temperature-map's shapefile
The text was updated successfully, but these errors were encountered: