Introduce @FlakyTest annotation for marking a few tests as flaky

[sijie]

FEATURE REQUEST

1. Please describe the feature you are requesting.

Add an annotation @FlakyTest for marking a few tests as flaky. So we can bring CI back to normal while at the same time working on fixing flaky tests.

2. Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?

nice-to-have

3. Provide any additional detail on your proposed use case for this feature.

N/A