TST: utils/extract unit tests that are failing

[aburrell]

Tests that still fail in test_utils_extract.py:

• TestUtilsExtractModObs::test_success
• TestUtilsExtractModObs::test_bad_kwarg_input[method-not_a_method-interpn only understands the methods]
• TestUtilsExtractModObs::test_bad_kwarg_input[sel_name-bad_val0-No model data keys to interpolate]

Expected behavior
These tests should pass. I think they need a better model instrument to work (not the same time cadence as the data instrument).

[jklenzing]

Found the cause of test_success

pysatModels/pysatModelUtils/tests/test_utils_extract.py

Line 114 in 471dfca

for label in self.model.data_vars.keys():

Should be self.model.data.data_vars.keys():, as model.data is the Dataset object.

[jklenzing]

I suspect that the route cause of the rest is a change in syntax needed since we are now using the pysat_testing2d_xarray object.

[jklenzing]

In the "method", "not_a_method" bad kwarg test, is the error message generated by scipy? or the local package? Can't find where this error message is generated.

[jklenzing]

For test_bad_kwarg_input[sel_name-bad_val0, these are the lines that are tested:

pysatModels/pysatModelUtils/utils/extract.py

Lines 601 to 605 in 471dfca

	# Remove any model coordinates from the modelled data to interpolate
	sel_name = sel_name[[mdat not in mod_name for mdat in sel_name]]
	if len(sel_name) == 0:
	raise ValueError('No model data keys to interpolate')

However, running with sel_name='Epoch' in the test, Epoch is not part of the model object. I suspect that the intention of the test can be satisfied by using sel_name='slt' instead. This is compatible with the new xarray test object. Is this correct?

[aburrell]

n the "method", "not_a_method" bad kwarg test, is the error message generated by scipy? or the local package? Can't find where this error message is generated.

It's generated by the scipy interpolation routine.

However, running with sel_name='Epoch' in the test, Epoch is not part of the model object. I suspect that the intention of the test can be satisfied by using sel_name='slt' instead. This is compatible with the new xarray test object. Is this correct?

I think so

[jklenzing]

Close to tracking down the scipy related test failure.

pysatModels/pysatModelUtils/utils/extract.py

Lines 684 to 691 in 471dfca

	# Construct the data needed for interpolation, ensuring that
	# the types are appropriate
	values = model[indices][mdat].data
	points = [model.coords[kk].data for kk in dims if kk in mod_name]
	get_coords = True if len(points) > 0 else False
	idims = 0
	while get_coords:

Essentially, get_coords (as generated at Line 688) is eternally false for the test instrument and model as defined, interpn is never called, and the error is never triggered. Haven't tracked down the root yet.

pysatModels/pysatModelUtils/utils/extract.py

Lines 626 to 637 in 471dfca

	tm_sec = (mod_datetime[1:] - mod_datetime[:-1]).min()
	tm_sec /= np.timedelta64(1, 's')
	ti_sec = (inst.index[1:] - inst.index[:-1]).min().total_seconds()
	min_del = tm_sec if tm_sec < ti_sec else ti_sec
	# Determine which instrument observations are within the model time
	# resolution of a model run
	mind = list()
	iind = list()
	del_sec = abs(mod_datetime-inst.index[:, np.newaxis]).astype(float) * 1.0e-9
	for inst_ind, mod_ind in enumerate(del_sec.argmin(axis=1)):
	if del_sec[inst_ind, mod_ind] <= min_del:

Also, for a test instrument with only one time stamp, min_del results in a nan value, which means the if statement at line 637 is never triggered. Running the test instrument with sat_id='2' or greater fixes this part of the problem, but the problems in Lines 684-691 remain.

[aburrell]

Essentially, get_coords (as generated at Line 688) is eternally false for the test instrument and model as defined, interpn is never called, and the error is never triggered. Haven't tracked down the root yet.

The root is that the model times need to be at a slower cadence than the instrument times.

[jklenzing]

Trying this with the 15 minute cadence version of the model. That may be too slow, though.

[jklenzing]

Closing with merge of #29.