Refactor shape_inference_test for old Op versions PR1

[guoyuhong]

Description

#4160

Motivation and Context

Shape inference tests for older opset versions are not supported before this PR. This refactor would help to test old-versioned ops.
This PR introduces a function all_versions_for to help generate test functions for all versions of one Op. The users can use @parameterized.expand(all_versions_for("${op_name}")) to annotate the test function to apply the test function to all op versions. The test function should have the arguments like def test_op(self, version_name, op_version). version_name is usually unused but it's better to keep that to make the parameterized test name more readable. For example, with this version_name, the test name will have a version suffix showed as follows.

[guoyuhong]

@jcwchen Do you have time to take a look at this change?

[jcwchen]

Thank you for the PR! I like the idea, but IIRC (I tried to test it before), we will probably need to modify quite a few existing tests to accommodate old opset versions. Still, it's great to have a starting point now and we can decouple the modified work if needed.

[guoyuhong]

@jcwchen I refined my code and applied this pattern to several Ops. For most Ops, the one test case fits all versions of this Op. Until now, I found there are Ops that have some incompatible tests. Here is a simple summary:

1. Reshape's allowzero since Version 14, so test_reshape_static_shape_allowzero before Version 14 is skipped.
2. Upsample is deprecated from Version 10. I added special test case for Version 10.
3. Upsample's test case for Version 7 and Version 9 are different. I copied test from the old code from Version 7.
4. Squeeze's test case for Version 11 and Version 13 are different. I copied test from the old code from Version 11.

By the way, there seems a lot of work to do. Is it better to split this change to several PRs?

[guoyuhong]

@jcwchen I will apply this test scheme to all other test functions in a later PR and try to fix the Reshape issue when op verison <= 5 after that.

[adityagoel4512]

When making updates to shape inference logic, should we reflect these changes in older shape inference implementations (e.g.

onnx/onnx/defs/tensor/old.cc

Line 5032 in 925840b

.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {

)?

[guoyuhong]

When making updates to shape inference logic, should we reflect these changes in older shape inference implementations (e.g.

onnx/onnx/defs/tensor/old.cc

Line 5032 in 925840b

.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {

)?

@adityagoel4512 I'm not sure. @jcwchen what do you think about this question?

[jcwchen]

When making updates to shape inference logic, should we reflect these changes in older shape inference implementations (e.g.

Good question... It's really case by case. I would say if there is a critical bug in old shape inference, we should. A common scenario is all opset versions do not behave correctly in some corner cases and ONNX should fix them consistently among all versions. Another common scenario is if some situations only happen in the latest opset version (e.g., new attribute or new documentation) rather than older ones, we don't need to update/add such a logic in older opest versions. Also we need to be very careful about some shape inference changes might break runtime backward compatibility.

[adityagoel4512]

That makes sense to me, I suppose it'll require careful reviews then.

[gramalingam]

Agree with Jacky. If the fix makes sense for older op versions, ideally we should make the change to older version too.