Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The CI of cpp tests is broken #11549

Closed
BewareMyPower opened this issue Aug 4, 2021 · 2 comments
Closed

The CI of cpp tests is broken #11549

BewareMyPower opened this issue Aug 4, 2021 · 2 comments
Labels
type/bug The PR fixed a bug or issue reported a bug

Comments

@BewareMyPower
Copy link
Contributor

BewareMyPower commented Aug 4, 2021

Describe the bug
Recently I found a bug in C++ client that was introduced from #11492 (I'll fix it soon). However, the point is, the related C++ unit tests should fail but the CI passed. See https://github.com/apache/pulsar/runs/3188401433:

     176 ms: ./main AuthPluginToken.testTokenWithHttpUrl (try #1)
      89 ms: ./main AuthPluginToken.testTokenWithHttpUrl (try #2)
      35 ms: ./main BasicEndToEndTest.testV2TopicHttp (try #1)
      27 ms: ./main BasicEndToEndTest.testV2TopicHttp (try #2)
      27 ms: ./main BasicEndToEndTest.testHandlerReconnectionLogic (try #1)
      25 ms: ./main BasicEndToEndTest.testHandlerReconnectionLogic (try #2)
    1519 ms: ./main ClientTest.testConnectTimeout (try #1)
    1519 ms: ./main ClientTest.testConnectTimeout (try #2)
      28 ms: ./main ProtobufNativeSchemaTest.testSchemaJson (try #1)
      34 ms: ./main ProtobufNativeSchemaTest.testSchemaJson (try #2)
     107 ms: ./main ProtobufNativeSchemaTest.testAutoCreateSchema (try #1)
      69 ms: ./main ProtobufNativeSchemaTest.testSchemaIncompatibility (try #1)
      86 ms: ./main ProtobufNativeSchemaTest.testAutoCreateSchema (try #2)
      97 ms: ./main ProtobufNativeSchemaTest.testSchemaIncompatibility (try #2)
      91 ms: ./main ProtobufNativeSchemaTest.testEndToEnd (try #1)
      82 ms: ./main ProtobufNativeSchemaTest.testEndToEnd (try #2)
    9244 ms: ./main KeySharedConsumerTest.testOrderingKeyPriority (try #1)
    1059 ms: ./main ConsumerTest.testBatchUnAckedMessageTracker (try #1)
     121 ms: ./main ClientDeduplicationTest.testProducerDeduplication (try #1)
      73 ms: ./main ClientDeduplicationTest.testProducerDeduplication (try #2)
      40 ms: ./main AuthPluginTest.testTlsDetectHttps (try #1)
      40 ms: ./main AuthPluginTest.testTlsDetectHttps (try #2)

It looks like the current CI of cpp tests allow the failed tests and bugs could be introduced with CI passed.

To Reproduce

Expected behavior
CI should fail if tests still failed after retrying.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]

Additional context
Add any other context about the problem here.

@BewareMyPower BewareMyPower added the type/bug The PR fixed a bug or issue reported a bug label Aug 4, 2021
@BewareMyPower
Copy link
Contributor Author

BewareMyPower commented Aug 4, 2021

I found the broken CI was introduced from #10309, do you know why the CI passed even if the tests failed? @lhotari

BTW, I'm going to fix these failed tests ASAP.

@BewareMyPower
Copy link
Contributor Author

It's fixed by #11575

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
Development

No branches or pull requests

1 participant