You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In this PR, it's found that it's possible for the maxcut benchmark to produce expected distributions with norm=0. At line 139 of get_expectation, there is a step:
# scale to number of shots
for k, v in counts.items():
counts[k] = round(v * num_shots)
Correct me if I'm wrong, but this is used to compare the results against a discrete approximation to the theoretical distribution, possibly to not penalize a result list that does not contain any counts for bitstrings that have very small probability mass (so that you'd expect 0 appearances at the given shot count) and also just to peg an integer number of results for each bitstring as ideal. This means for the case you describe, the only way to run the problem instance throwing the error is to increase the number of shots until the discretized expected distribution has at least a single nonzero element. I worry this has its own issues, because a significant distortion between the original distribution and the discretized distribution causes a distortion in the actual fidelity calculation. You could conceivably be comparing the results to a discrete distribution with whacky finite-size effects that make it look very different from the distribution that an ideal quantum computer is pulling from. This should only happen when the theoretical distribution is very wide and mostly but not perfectly flat, (I think), but it's worth considering.
Just wanted to share these thoughts... I don't think we use this kind of step in other benchmarks? It seems odd that we'd calculate fidelity against discretized distributions for some benchmarks and continuous exact distributions for others. Maybe we should perform a check that something like this doesn't happen in maxcut_benchmark.py or pass the exact distribution even if the fidelity moderately underperforms at low shot counts?
The text was updated successfully, but these errors were encountered:
In this PR, it's found that it's possible for the maxcut benchmark to produce expected distributions with norm=0. At line 139 of
get_expectation
, there is a step:Correct me if I'm wrong, but this is used to compare the results against a discrete approximation to the theoretical distribution, possibly to not penalize a result list that does not contain any counts for bitstrings that have very small probability mass (so that you'd expect 0 appearances at the given shot count) and also just to peg an integer number of results for each bitstring as ideal. This means for the case you describe, the only way to run the problem instance throwing the error is to increase the number of shots until the discretized expected distribution has at least a single nonzero element. I worry this has its own issues, because a significant distortion between the original distribution and the discretized distribution causes a distortion in the actual fidelity calculation. You could conceivably be comparing the results to a discrete distribution with whacky finite-size effects that make it look very different from the distribution that an ideal quantum computer is pulling from. This should only happen when the theoretical distribution is very wide and mostly but not perfectly flat, (I think), but it's worth considering.
Just wanted to share these thoughts... I don't think we use this kind of step in other benchmarks? It seems odd that we'd calculate fidelity against discretized distributions for some benchmarks and continuous exact distributions for others. Maybe we should perform a check that something like this doesn't happen in
maxcut_benchmark.py
or pass the exact distribution even if the fidelity moderately underperforms at low shot counts?The text was updated successfully, but these errors were encountered: