age comp data

[BenWilliams-NOAA]

I'm not finding all of the age comps in the dataset
for example there should be comps for GOA POP prior to 1991, I'm not filtering this by area at all and cannot seem to retrieve the appropriate data - thoughts?

remotes::install_github("afsc-assessments/afscdata")
db = afscdata::connect("afsc")
afsc_species = 30060
dplyr::tbl(db, dplyr::sql("gap_products.agecomp")) %>% 
             dplyr::rename_with(tolower) %>% 
             dplyr::filter(species_code %in% afsc_species) %>% 
              dplyr::arrange(year)

  survey_definition_id area_id  year species_code   sex   age population_count length_mm_mean
                  <dbl>   <dbl> <dbl>        <dbl> <dbl> <dbl>            <dbl>          <dbl>
 1                   52   99904  1991        30060     1    -9           212445           408.
 2                   52   99904  1991        30060     1     1           467403           141.
 3                   52   99904  1991        30060     1     2          1874534           178.
 4                   52   99904  1991        30060     1     3          7344777           188.
 5                   52   99904  1991        30060     1     4          8385599           217.
 6                   52   99904  1991        30060     1     5         20979668           256.
 7                   52   99904  1991        30060     1     6         17513817           269.
 8                   52   99904  1991        30060     1     7         62769608           297.
 9                   52   99904  1991        30060     1     8         39032702           302.
10                   52   99904  1991        30060     1     9         28449398           327.
# ℹ more rows
# ℹ 1 more variable: length_mm_sd <dbl>
# ℹ Use `print(n = ...)` to see more rows

[zoyafuso-NOAA]

Hi @BenWilliams-NOAA ,

Thanks for testing out the tables. This is related to an internal discussion our working group has been having about the temporal stanzas to apply in each survey region. For the GOA and AI regions, there are data in our historical tables that go back to the 1980s but for projects like EFH and ModSquad, we only use data from the 90s-on (or at least are switching to this filter) because our team is more comfortable with the consistency of the standard survey protocol post-1990. For GOA the starting year is 1993-on and for AI the starting year is 1991-on. @Ned-Laman-NOAA would have more details on the issues with the data in the 1980s.

The tables are not set in stone at this point, so if you use data from pre-1990 in your analyses, this is great feedback that we can integrate into our discussions and maybe restore in future reruns of these tables.

[BenWilliams-NOAA]

Thanks for the feedback @zoyafuso-NOAA
Definitely many (most?) assessments are excluding the 1980s data, though some continue to include it.
I believe all that exclude those data include data from 1990.
As the data exclusions are currently setup the queries will not work for most all assessment authors, at least in the GOA.

[zoyafuso-NOAA]

Oh interesting, good to know. @Ned-Laman-NOAA , are we okay with adjusting the temporal stanza for the GOA GAP_PRODUCTS tables to 1990-current instead of 1993-current? I may misread Table 1 from the GOA EFH doc incorrectly, using 1993 as the cutoff.

[mkapur-noaa]

Thanks Ben for starting this issue and Zack & co for quick response. For what it's worth, regardless of what's decided re: validity of 1990 values, for continuity purposes it's often necessary for us to include these data. This is the case for this year for POP (the 2021 assessment had the 1990 survey age comps, so I need to have them for the bridged model). Just wanted to share that as you continue your internal discussions; many assessments will complete "updates" where identical data coverage is necessary. Thanks!

[Lewis-Barnett-NOAA]

The nets and vessels changed a lot, with the transition being complete in 1991. I talked with Ned informally about 1990 GOA data, which was done with about 70% of the effort being standardized to contemporary methods, the rest with prior methods. Ned can give the official decision on that, but as of now I'm planning to continue to use 1990 GOA bottom trawl data for modsquad products.

…

On Wed, Sep 27, 2023 at 9:51 AM Maia Kapur ***@***.***> wrote: Thanks Ben for starting this issue and Zack & co for quick response. For what it's worth, regardless of what's decided re: validity of 1990 values, for continuity purposes it's often necessary for us to include these data. This is the case for this year for POP (the 2021 assessment had the 1990 survey age comps, so I need to have them for the bridged model). Just wanted to share that as you continue your internal discussions; many assessments will complete "updates" where identical data coverage is necessary. Thanks! — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKMJP3ZARAO7U4ALNVOIR3X4RKRPANCNFSM6AAAAAA5GZOBMM> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- Lewis Barnett, PhD (he/him/his) Research Fish Biologist NOAA Fisheries, Alaska Fisheries Science Center 7600 Sand Point Way NE, Bldg 4 Seattle, Washington 98115 Google Voice: (206) 526-4111

[BenWilliams-NOAA]

Really appreciate the discussion all! To further what Maia was saying, there is an additional layer that as assessment authors we pass inclusion/exclusion of data past the plan team and ssc before taking action. Dropping or including these data can sometimes have a large influence (as I'm sure the modsqaud is aware!)

[Ned-Laman-NOAA]

Great discussion all, and very helpful to our team developing these products.

Lewis and I did discuss the window of standardized GOA data and I agree with starting the stanza with 1990 for data analyses.

As Lewis indicated, the majority of the 1990 data were standardized with regards to trawl gear and sampling protocols so that they align well with what we currently implement at sea. Pre-1990, the gear and sampling protocols were evolving and standardization was less of "a thing" during that period.

[zoyafuso-NOAA]

So the partial decision is to start the GOA time series for GAP_PRODUCTS tables from 1990. After some offline discussion, Ned will get back to us on whether it is still appropriate to include data from the 1980s for both AI and GOA. Thanks all.

[zoyafuso-NOAA]

Hi @BenWilliams-NOAA and @mkapur-noaa,

Sorry for the delay, we haven't forgotten about this issue. @Ned-Laman-NOAA has a response (we talked about this this afternoon), this is just a ping for him to nip this issue in the bud.

[Ned-Laman-NOAA]

Hi all.

I spent some in our codebooks that describe our survey gear as recorded in RACEBase and had some conversations with our Net Shed Supervisor (Shawn Russell) about the trawl gear we're currently deploying in the Gulf of Alaska and Aleutian Islands.

The short version of what I found is that the stanza for standardized Gulf of Alaska bottom trawl surveys is 1990 to present and for the Aleutian Islands is 1991 to present.

Some confusion arises because the footrope configuration we're presently using has on the Poly Nor'Eastern trawl been in use since 1987, but we don't have sufficient documentation about sampling protocols from those late 80's co-op surveys to be confident that on deck catch processing is comparable to what we are currently doing on our surveys. Therefore, even though there are some trawls in the late 80's with comparable trawl configurations, we have decided, in the absence of confirmatory evidence, to be conservative with our interpretation of standardization and set the start of the two time series as I've indicated above.

For those instances where a stock author needs to have access to pre-1990 data when building bridging models or otherwise for comparison, I recommend requesting those data sets through our data request repository so that we can capture those needs and more efficiently respond to them in the future.

N

[mkapur-noaa]

Thank you for the detailed documentation and thought to this process. This sounds sensible to me. My only thought -- and this might be more for @BenWilliams-NOAA and other package-writing folks -- is that it would be useful for a message() to pop up confirming/reminding folks that those years will NOT be pulled, even if they are asked for, under the system defaults. With lots of automation it is easy to not catch that certain years are suddenly not in an input file.

[Ned-Laman-NOAA]

That's a great comment, Maia!
It can be tough to figure out what happened when something is just absent.

[zoyafuso-NOAA]

GAP_PRODUCTS should now have 1990 data for GOA and future production runs will use 1990 as the start date for GOA and 1991 for AI. Thanks all!