tests.system.aiplatform.test_autologging.TestAutologging: test_autologging_with_autorun_creation failed

[flaky-bot]

Note: #1950 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.

---

commit: bf1be87
buildURL: Build Status, Sponge
status: failed

Test output

self = Index(['experiment_name', 'run_name', 'run_type', 'state', 'param.n_jobs',
       'param.copy_X', 'param.fit_intercept...rmse',
       'metric.training_r2_score', 'metric.training_mse',
       'metric.training_score'],
      dtype='object')
key = 'metric.training_mean_absolute_error', method = None, tolerance = None
def get_loc(self, key, method=None, tolerance=None):
    """
    Get integer location, slice or boolean mask for requested label.

    Parameters
    ----------
    key : label
    method : {None, 'pad'/'ffill', 'backfill'/'bfill', 'nearest'}, optional
        * default: exact matches only.
        * pad / ffill: find the PREVIOUS index value if no exact match.
        * backfill / bfill: use NEXT index value if no exact match
        * nearest: use the NEAREST index value if no exact match. Tied
          distances are broken by preferring the larger index value.

        .. deprecated:: 1.4
            Use index.get_indexer([item], method=...) instead.

    tolerance : int or float, optional
        Maximum distance from index value for inexact matches. The value of
        the index at the matching location must satisfy the equation
        ``abs(index[loc] - key) <= tolerance``.

    Returns
    -------
    loc : int if unique index, slice if monotonic index, else mask

    Examples
    --------
    >>> unique_index = pd.Index(list('abc'))
    >>> unique_index.get_loc('b')
    1

    >>> monotonic_index = pd.Index(list('abbc'))
    >>> monotonic_index.get_loc('b')
    slice(1, 3, None)

    >>> non_monotonic_index = pd.Index(list('abcb'))
    >>> non_monotonic_index.get_loc('b')
    array([False,  True, False,  True])
    """
    if method is None:
        if tolerance is not None:
            raise ValueError(
                "tolerance argument only valid if using pad, "
                "backfill or nearest lookups"
            )
        casted_key = self._maybe_cast_indexer(key)
        try:


          return self._engine.get_loc(casted_key)


.nox/system-3-10/lib/python3.10/site-packages/pandas/core/indexes/base.py:3802:

pandas/_libs/index.pyx:138: in pandas._libs.index.IndexEngine.get_loc

???

pandas/_libs/index.pyx:165: in pandas._libs.index.IndexEngine.get_loc

???

pandas/_libs/hashtable_class_helper.pxi:5745: in pandas._libs.hashtable.PyObjectHashTable.get_item

???


???

E   KeyError: 'metric.training_mean_absolute_error'

pandas/_libs/hashtable_class_helper.pxi:5753: KeyError
The above exception was the direct cause of the following exception:
self = <tests.system.aiplatform.test_autologging.TestAutologging object at 0x7f1356d09b70>

shared_state = {'bucket': <Bucket: tmpvrtxsdk-e2e-cd1443d1-8135-4fdc-ba1b-29e6e10aae41>, 'resources': [<google.cloud.aiplatform.tenso...cd1443d1-8135-4fdc-ba1b-29e6e10aae41', 'storage_client': <google.cloud.storage.client.Client object at 0x7f1252011cf0>}
def test_autologging_with_autorun_creation(self, shared_state):

    aiplatform.init(
        project=e2e_base._PROJECT,
        location=e2e_base._LOCATION,
        experiment=self._experiment_autocreate_scikit,
        experiment_tensorboard=self._backing_tensorboard,
    )

    shared_state["resources"] = [self._backing_tensorboard]

    shared_state["resources"].append(
        aiplatform.metadata.metadata._experiment_tracker.experiment
    )

    aiplatform.autolog()

    build_and_train_test_scikit_model()

    # Confirm sklearn run, params, and metrics exist
    experiment_df_scikit = aiplatform.get_experiment_df()
    assert experiment_df_scikit["run_name"][0].startswith("sklearn-")
    assert experiment_df_scikit["param.fit_intercept"][0] == "True"


  assert experiment_df_scikit["metric.training_mean_absolute_error"][0] > 0


tests/system/aiplatform/test_autologging.py:162:

.nox/system-3-10/lib/python3.10/site-packages/pandas/core/frame.py:3807: in getitem

indexer = self.columns.get_loc(key)

self = Index(['experiment_name', 'run_name', 'run_type', 'state', 'param.n_jobs',

'param.copy_X', 'param.fit_intercept...rmse',

'metric.training_r2_score', 'metric.training_mse',

'metric.training_score'],

dtype='object')

key = 'metric.training_mean_absolute_error', method = None, tolerance = None
def get_loc(self, key, method=None, tolerance=None):
    """
    Get integer location, slice or boolean mask for requested label.

    Parameters
    ----------
    key : label
    method : {None, 'pad'/'ffill', 'backfill'/'bfill', 'nearest'}, optional
        * default: exact matches only.
        * pad / ffill: find the PREVIOUS index value if no exact match.
        * backfill / bfill: use NEXT index value if no exact match
        * nearest: use the NEAREST index value if no exact match. Tied
          distances are broken by preferring the larger index value.

        .. deprecated:: 1.4
            Use index.get_indexer([item], method=...) instead.

    tolerance : int or float, optional
        Maximum distance from index value for inexact matches. The value of
        the index at the matching location must satisfy the equation
        ``abs(index[loc] - key) <= tolerance``.

    Returns
    -------
    loc : int if unique index, slice if monotonic index, else mask

    Examples
    --------
    >>> unique_index = pd.Index(list('abc'))
    >>> unique_index.get_loc('b')
    1

    >>> monotonic_index = pd.Index(list('abbc'))
    >>> monotonic_index.get_loc('b')
    slice(1, 3, None)

    >>> non_monotonic_index = pd.Index(list('abcb'))
    >>> non_monotonic_index.get_loc('b')
    array([False,  True, False,  True])
    """
    if method is None:
        if tolerance is not None:
            raise ValueError(
                "tolerance argument only valid if using pad, "
                "backfill or nearest lookups"
            )
        casted_key = self._maybe_cast_indexer(key)
        try:
            return self._engine.get_loc(casted_key)
        except KeyError as err:


          raise KeyError(key) from err


E               KeyError: 'metric.training_mean_absolute_error'
.nox/system-3-10/lib/python3.10/site-packages/pandas/core/indexes/base.py:3804: KeyError

[flaky-bot]

Closing as a duplicate of #2796