Historical vectors should be omitted from the dropdown

[oysteoh]

Should just be default added to the summary vector

[pinkwah]

Note that checking for names ending with H and assuming it's historical is potentially an awful heuristic. See equinor/resdata#708

Julius suggested to check if a vector whose name ends with H also has a counterpart without H, in which case we can assume it's historical. Should double-check though.

[asnyv]

FYI:
For a plotting solution, that would probably work well @dotfloat. There is a risk that a user will only export the historical version (as you have to specify which vectors to export in the data deck, and you might include just e.g. FOPTH and not FOPT, though I wouldn't think it is common). But in this case, it probably makes sense to add the historical vector to the dropdown, as otherwise it wouldn't be possible to plot at all.

So checking for a counterpart is probably a good way to confirm that a vector is (very likely) to be historical, but not safe to be used to exclude the fact that it could be.

Another tricky thing we have encountered in webviz-subsurface is that there are use cases for uncertainty in historical data (for example the distribtution of produced volumes between wells, measurement errors in gauges and etc). In that case, the historical data is an ensemble in itself. In webviz-subsurface we currently only plot the historical data for 1 realization and assume that it is descriptive for the ensemble, but if you are unlucky, it could be the P99 historical vector... Just to add to the mess and confusion 😉

[eivindsm]

I am in favor of filtering out the historical vectors from the drop down list, alternatively as a defaulted option (eg radio button). As Asgeir points out it is possible to construct cases for which the historical vectors differ over an ensemble but as of now these are non existing. I also agree that the heuristic is terrible, but Julius' suggestion would make it robust. Out of the box only historical vectors have first identifier ending with an H, however it is possible to define user defined summay vectors and I think these are allowed to end on an H.

[asnyv]

@eivindsm When you say non-existing, what do you mean? Allocation uncertainty is not that uncommon to use?

You also have a few vectors that are not user defined, but still ends with an H. Some of the examples I found in a quick search (was quite a few more):
MONTH, AAQENTH (E300 only), SPRDH, SRRDEPTH, SRREXCH, CRREXCH, TCPUH, TCPUSCH

Most of these examples are quite obscure, but the general assumption of *H==historical is for sure not valid-

[markusdregi]

My next question is then; do we want to persist the historical vectors at all by default? Or could it even be an option that they never hit storage?

[asnyv]

would say: If you haven't added any uncertainty to it, and you are in history, it is for sure value in it. But needs some discussion. Think in the current plotter, the hated refcase is used to plot the historical vectors?

[eivindsm]

We should be able to plot it, the question is whether we need to store n-ens copies. As Asgeir suggests there could be situations for which they are not identical over the realizations. If we are to support the sort of functionality prototyped in nyb-viz we also need access to them. I'd say, yes we should persist them.

[eivindsm]

As Asgeir points out, the heuristics is terrible. It also seems that I have to modify my claim that uncertainty in allocation is nonexistent. My suggestion would be to have a radio button to filter *H vectors out from the drop down list similar to the option we already have filtering out responses without corresponding observations.

[oysteoh]

Implementing a version of this with the suggested checkbox filtering out all keys ending on H