While looking at #5507 I noticed CircuitOperation uses with_... functions to load the data from json, which means unnecessarily reconstructing and revalidating the whole subcircuit for each argument.
https://github.com/quantumlib/Cirq/blob/master/cirq-core/cirq/circuits/circuit_operation.py#L477-L484
Seems like this should instead call the constructor directly with all arguments.