Analysis fails to run after serializing data retrieved from database

[kevinsung]

Informations

• Qiskit Experiments version: 53c3d21
• Python version: 3.9.10
• Operating system: Arch Linux

What is the current behavior?

The following steps result in an error:

1. Run an experiment and save it to the database service
2. Retrieve the data from the database service
3. Serialize the data
4. Deserialize the data
5. Run analysis on the data

Steps to reproduce the problem

import json
from qiskit import IBMQ
from qiskit_experiments.framework import ExperimentEncoder, ExperimentDecoder, ExperimentData
from qiskit_experiments.library import StandardRB

IBMQ.load_account()
provider = IBMQ.get_provider(hub='ibm-q-internal', group='deployed', project='default')
backend = provider.get_backend('ibmq_belem')

exp = StandardRB(
    qubits=(0,),
    lengths=list(range(1, 5)),
    seed=123,
    backend=backend,
)
data = exp.run()
data.save()
data.block_for_results()

retrieved_data = ExperimentData.load(data.experiment_id, provider.service('experiment'))
deserialized = json.loads(json.dumps(retrieved_data, cls=ExperimentEncoder), cls=ExperimentDecoder)
deserialized = exp.analysis.run(deserialized)

/home/kjs/projects/qiskit-experiments/qiskit_experiments/framework/json.py:567: UserWarning: Could not deserialize instance of class <class 'qiskit.providers.ibmq.experiment.ibm_experiment_service.IBMExperimentService'> from settings {}. 
The following exception was raised:
Traceback (most recent call last):
  File "/home/kjs/projects/qiskit-experiments/qiskit_experiments/framework/json.py", line 338, in _deserialize_object
    return cls(**settings)
TypeError: __init__() missing 1 required positional argument: 'provider'

  return _deserialize_object(obj_val)
/home/kjs/projects/qiskit-experiments/qiskit_experiments/framework/json.py:567: UserWarning: Could not deserialize instance of class <class 'qiskit.providers.ibmq.ibmqbackend.IBMQBackend'> from settings {}. 
The following exception was raised:
Traceback (most recent call last):
  File "/home/kjs/projects/qiskit-experiments/qiskit_experiments/framework/json.py", line 338, in _deserialize_object
    return cls(**settings)
TypeError: __init__() missing 4 required positional arguments: 'configuration', 'provider', 'credentials', and 'api_client'

  return _deserialize_object(obj_val)
/home/kjs/projects/qiskit-experiments/qiskit_experiments/framework/json.py:567: UserWarning: Could not deserialize instance of class <class 'datetime.datetime'> from settings {}. 
The following exception was raised:
Traceback (most recent call last):
  File "/home/kjs/projects/qiskit-experiments/qiskit_experiments/framework/json.py", line 338, in _deserialize_object
    return cls(**settings)
TypeError: function missing required argument 'year' (pos 1)

  return _deserialize_object(obj_val)

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [1], in <module>
     20 retrieved_data = ExperimentData.load(data.experiment_id, provider.service('experiment'))
     21 deserialized = json.loads(json.dumps(retrieved_data, cls=ExperimentEncoder), cls=ExperimentDecoder)
---> 22 deserialized = exp.analysis.run(deserialized)

File ~/projects/qiskit-experiments/qiskit_experiments/framework/base_analysis.py:145, in BaseAnalysis.run(self, experiment_data, replace_results, **options)
    143 # Make a new copy of experiment data if not updating results
    144 if not replace_results and _requires_copy(experiment_data):
--> 145     experiment_data = experiment_data.copy()
    147 experiment_components = self._get_experiment_components(experiment_data)
    149 # Set Analysis options

File ~/projects/qiskit-experiments/qiskit_experiments/framework/experiment_data.py:173, in ExperimentData.copy(self, copy_results)
    172 def copy(self, copy_results=True) -> "ExperimentData":
--> 173     new_instance = super().copy(copy_results=copy_results)
    175     # Copy additional attributes not in base class
    176     if self.experiment is None:

File ~/projects/qiskit-experiments/qiskit_experiments/database_service/db_experiment_data.py:1475, in DbExperimentDataV1.copy(self, copy_results)
   1473 with self._analysis_results.lock:
   1474     new_instance._analysis_results = ThreadSafeOrderedDict()
-> 1475     new_instance.add_analysis_results([result.copy() for result in self.analysis_results()])
   1476 with self._figures.lock:
   1477     new_instance._figures = ThreadSafeOrderedDict()

File ~/projects/qiskit-experiments/qiskit_experiments/database_service/db_experiment_data.py:865, in DbExperimentDataV1.analysis_results(self, index, refresh, block, timeout)
    863 if block:
    864     self._wait_for_analysis(timeout=timeout)
--> 865 self._retrieve_analysis_results(refresh=refresh)
    866 if index is None:
    867     return self._analysis_results.values()

File ~/projects/qiskit-experiments/qiskit_experiments/database_service/db_experiment_data.py:827, in DbExperimentDataV1._retrieve_analysis_results(self, refresh)
    825 # Get job results if missing experiment data.
    826 if self.service and (not self._analysis_results or refresh):
--> 827     retrieved_results = self.service.analysis_results(
    828         experiment_id=self.experiment_id, limit=None, json_decoder=self._json_decoder
    829     )
    830     for result in retrieved_results:
    831         result_id = result["result_id"]

AttributeError: 'dict' object has no attribute 'analysis_results'

What is the expected behavior?

It should work.

Suggested solutions

[kevinsung]

The bug is triggered by copying the data, i.e. deserialized.copy().

[kevinsung]

It's triggered because deserialized.service is a dict, not a DatabaseServiceV1. I guess it's related to the warning about how the service could not be deserialized.

[ItamarGoldman]

After checking it I think we need to implement __json_encode__ methods in qiskit.providers.ibmq.experiment.ibm_experiment_service.IBMExperimentService class and qiskit.providers.ibmq.ibmqbackend.IBMQBackend.

If we remove the service from the json, the analysis is working.
To solve the problem I will use the method that is used in job_id .

[ItamarGoldman]

I want to understand the use case for dumping the experiment data to a json.
The use case that I thought of are for sharing and moving experiment data without accessing experiment_db.

The _service attribute is intended for communication with the database.
For the use I see for creating the json file, there is no relevance to this property and therefore I do not see the need to deserialize it.
In addition, I don't see the meaning in serializing it as it will not have any meaning because the session doesn't exist anymore.
It could be that this problem will fix it self after the merge of the updated service that @gadial is working on.
Regardless, I suggest the following solution (in __json_encode__):

json_value = {}
for att in [
	"_metadata",
	"_source",
	"_service",
	"_backend",
	"_id",
	"_parent_id",
	"_type",
	"_tags",
	"_share_level",
	"_notes",
	"_data",
	"_analysis_results",
	"_analysis_callbacks",
	"_deleted_figures",
	"_deleted_analysis_results",
	"_created_in_db",
	"_extra_data",
]:
	# the attribute self._service in charge of the connection and communication with the
	#  experiment db. It doesn't have meaning in the json format so there is no need to serialize
	#  it.
	if att == "_service":
		warnings.warn("'service' attribute isn't serializable.)
		continue
	value = getattr(self, att)
	if value:
		json_value[att] = value

@chriseclectic, I would like to hear your opinion.

[kevinsung]

@ItamarGoldman My use case for dumping the experiment data to JSON is data archival. For example, suppose the data is used to support results published in a research paper. Then the data should be made publicly available so the results of the paper can be verified and reproduced.

[chriseclectic]

I think the main issue here is that Service, Providers, and Backends are not properly JSON serializable, and can't easily be made JSON serializable because of the token authentication required for loading them.

What happens now is these thing get serialized as an object, but deserialization fails and the partially deserialized dict gets set as the value in the objects containing them, which then leads to errors when other parts of code that interacts with them. In the BaseExperiment class if it gets a partially deserialized backend it just ignores it, but I don't think ExperimentData does the same sort of handling.

One possible work around when deserializing these objects directly would be to display a warning message showing the saved metadata (backend / service name etc) and asking user to re-set these values, but then set the value to None so that the loaded object should still work (assuming it can work without a backend/service). Eg something like "Experiment was saved using (name) service which cannot be automatically loaded. Please reset service attribute if required"

[ItamarGoldman]

@kevinsung The service attribute responsible for the communication with the cloud. The documentation of the class is as following:

This class is the main interface to invoke IBM Quantum
experiment service, which allows you to create, delete, update, query, and
retrieve experiments, experiment figures, and analysis results.

Because it is not experiment-related, experiment results can be restored without the need for service serialization.
I will do as @chriseclectic suggested. I will set service to be None in for encoding and raise a warning. This will not affect your usage.

[eliarbel]

Can this be closed?

[ItamarGoldman]

Yes. Closing the issue.