Table Uncertainty Reporting

[JZahner1]

It appears that process_table, and functions dependent on it, report estimate and uncertainty but not the the upper and lower estimates. These are usually more informative than the raw uncertainty values. I think that estimate_lower and estimate_upper should be output by process_table by default rather than uncertainty.

Allowing users to optionally report uncertainty in addition to, or instead of, estimate_lower/estimate_upper, would probably be valid.

I think this is a fairly simple fix.

[Schiano-NOAA]

How do you think it would be best displayed? Couldn't the users easily calculate the upper and lower bounds using the uncertainty from the model?

We face issues where tables can become quite clunky when too many columns are present because headers get repeated to meet accessibility requirements. I think an example of what you are thinking would be helpful!

[JZahner1]

Sure, the user could calculate them themselves; but its a calculation that's already built into the data provided to process_table. Seems like something that should be provided for convenience (also to ensure mistakes dont get made in calculating the upper/lower values).

Large tables being clunky is a universal problem. When you refer to "repeating headers" you're referring to using the full "stddev_spawning_potential_ratio_ratio" style names correct? If so, is there a reason that has become the accepted way to meet accessibility requirements? It seems to me like assessment authors would always change those header names to something more human readable before they became part of a report.

I would be expecting to see a table that styled something like below (data is very fake):

Year	Estimate	Lower	Upper	Uncertainty
1	0.68	0.60	0.75	0.02
2	0.65	0.58	0.7	0.02
3	0.60	0.55	0.64	0.01
4	0.55	0.46	0.65	0.03
5	0.51	0.38	0.65	0.05

Obviously if there are tons of grouping variables (fleets, sexes, regions, etc.) this starts to get more complicated.

This table could potentially be simplified as something like:

Year	Estimate (uncertainty)	Lower - Upper
1	0.68 (0.02)	0.60 - 0.75
2	0.65 (0.02)	0.58 - 0.70
3	0.60 (0.01)	0.55 - 0.64

Just a thought.

[kellijohnson-NOAA]

Instead of storing the intervals as columns would it be okay to have helper functions to easily return the intervals, instead of having them calculated deep within plotting or table code. That way the user would have access to them easily using the same code that tables and figures use? This would help for the scenario when people want non-standard intervals, e.g., 25 and 75% rather than 5 and 95%.

[JZahner1]

@Schiano-NOAA are the intervals being read directly from the model outputs, or are they calculated internally based on the estimate and uncertainty (value + type)? If they're being derived internally, I think adding support for non-standard intervals is a simple and valuable feature.

[JZahner1]

@kellijohnson-NOAA @Schiano-NOAA I played around, and I do think it would be relatively simple to create a helper function that would allow for plotting/reporting of non-standard intervals. It basically requires doing something like:

interval_info <- list(label = list(dist="...", alpha="..."))
prepared_data <- filter_data(...) |> calculate_intervals(interval_info)

The user would have to define interval_info (probably as a function argument) if they want to customize the interval size, and would be able to do so differently for each label in the data (e.g., different intervals for SSB vs SPR if we were plotting kobe plots). This could be simplified down by requiring all quantities to use the same interval sizes. I remain unsure if its possible to allow the user to define a specific distribution to calculate intervals from, since some of them may require transforming the mean or uncertainty.

[Schiano-NOAA]

I am going to convert this to a discussion to get more input from people. This would require an overhaul of the current tables, but if it's something other regions would want, we can determine the best path forward.

Note: Many tables do need groupings which can be difficult and become hard to read.

[Schiano-NOAA]

@nmfs-ost/asar-collaborators thoughts?

[JZahner1]

Looking at some of the other tables, I do agree that we would need to carefully think about reporting upper/lower values as a default. For example:

Do upper/lower columns need to be reported for both observed and predicted values?
Should columns be reported as: "observed estimate", "observed lower", "observed upper", "predicted estimate", "predicted lower", etc. or as "observed estimate", "predicted estimate", "observed lower", "predicted lower", etc.

I think tables in the style of | "estimate (uncertainty)" | "(lower - upper)" | is probably the cleanest way to handle reporting the uncertainty intervals, but it can definitely get complicated.

[kellijohnson-NOAA]

We should also be thinking about this in a Bayesian context where normality of the posterior distribution cannot be assumed. In those instances the uncertainty would be summarized across thousands of report files rather than a single uncertainty value applied to a point estimate.

[JZahner1]

Is there something specific that needs to be considered in terms of table formatting in the bayesian context? "uncertainty" (as a standard deviation or standard error or CV), might not make sense anymore if normalcy cant be assumed, but displaying lower-upper intervals should be robust to the shape of the posterior, I would think.

[kellijohnson-NOAA]

Credibility intervals can be calculated depending on what you want the interval to be, so very similar to what you are desiring but I don't know if that interval level is standard.